they're not "bugs," they're "unexpected features." the true secret of the creation of Eight-ball.
warnings: Fateverse. sci-fi/technobabble. AI evolution, philosophical junk. language: pg-13 (for f***, s***, and g**damn).
pairing: none/gen. reference to Nate/Wade bromance.
timeline: shortly after Standards, back at the Core compound.
disclaimer: marvel owns all base characters, but i created the AU and various AU versions of the characters.
notes: 1) gestalt (guh-'shtahlt) is the German word for the curious phenomenon of things becoming more than the sum of their parts, sometimes through evolution-like processes. 2) the sector lifts are all in the middle of their respective tower sectors, and the libraries are all at the right-hand (when facing in) end of their respective sectors. 3) as Four mentioned in Artifacts (part of Pyrotechnics for the Soul), the data-patterns that make up an AI look very different from the data-patterns that make up a human consciousness. 4) it's been argued that a real conscience is the result of an extremely complicated set of mental and emotional priorities that is completely individual to each person and would be almost impossible to replicate in an artificial construct. 5) the Fidelis Standardized Ethical Diagnostic should have a nice, hefty entry in the Glossary and Appendix. the three scenarios that Anthony poses to Eight-ball are all classical ethical problems; a version of the turtle-in-the-desert scenario even appears in the Voight-Kampff test in Bladerunner. 6) the problem with an AI developing a real conscience is that a simple priority list makes most AI more predictable than humans (which Stephanie mentioned back in Knots).
visit The Fateverse Glossary (merianmoriarty (dot) deviantart (dot) com/art/Fateverse-Glossary-174203180) for terms, concepts, Nodes, and important people.
Gestalt
"What are you gonna do when we get down there?" Five wants to know while they're riding the lift.
Anthony shrugs. "Hack us into the library. If that doesn't work, I'll ask the Sysadmin to let us in."
"That may be the dumbest joke I've heard in the past twenty years."
"I'm not joking."
Gradually, it occurs to him that her hands are shaking.
"Five?"
When she speaks again, her voice is tight and strained. "Hey, look, I'm only gonna go along with this because I fucking hate it when people refuse to do their jobs and you technically outrank me and I can completely blame this on you if it goes south. For the record, I'm not loving the idea of breaking into one of the old woman's libraries."
"Duly noted," he says, just before the lift's doors open.
~Turn right. Proceed eight hundred feet to the nearest isolation library.~
Five seems to have worked up her nerve; she stalks down the curving corridor with all the fury she mustered at lunch and aggressively palms the access panel of the library.
~Ingress denied,~ the compound's computer beeps.
Anthony already has his datapen ready, slips his way into the nearby systems with an easy scribble of code. The door, however, refuses every backdoor authorization he can think of (he is pestered briefly by the mental image of a man in a suit saying, 'Ah-ah-ah, I thought of that one, too…'). He presses the messaging button on his datapen, hurriedly scrawls a brief note.
-Are you there?-
The access panel flickers and blanks, and text appears.
define 'there.' XD
Anthony rolls his eyes. -Ha ha. Listen, we need to talk to Four, but she's locked the library. Could you work your magic and give us a Sysadmin override?-
…dunno. =S if she's being grumpy, maybe you should leave her alone. she's no fun when she's grumpy anyway.
-We need her to examine some data—it's very important. We need her to do her job, regardless of how pissed she is at me.-
you made Hope mad? D= what did you do, you meanie?
"What?" Anthony squawks. "What'd I do? She's the one who—you know what, nevermind. I'll just go get a welding torch or something." As he turns to go, he hears the door chime.
~Access granted by System Administrator override.~
"Thank you."
"You traitor!" Four growls as the door opens.
"It's not his fault you won't do your goddamn job," Five grits out.
Four seems surprised to see the other woman. "Five? You're the one who's been trying to get in touch with me? What is it?"
Anthony decides to answer by drawing up the form to request a full active imprint of Node 250. "He's a wily fucker," he says while he finishes filling it out. "So lock the plate as soon as you make the imprint. Slot…DD85F3, I think."
After a moment, a slot on the shelf blinks.
"Four?" Anthony says, gesturing to the wall.
The redhead glares at him for a moment, as though she could read his mind by force of will alone. Then she goes to the shelf and slides the isolation plate out of the blinking port in the wall. "Six, what in God's name am I looking at?" she hisses, and her tone is some muddled mixture of fear and anger. "What the hell did you make?"
In her hand, the plate's surface flickers with blasts of color that look like fireworks; they dance and meld and explode all over again.
Helpfully, Anthony gestures to it. "Well, that is an active imprint."
"I know that!" she snaps. "Tell me what it's an imprint of before I hit you with it."
"You were there when I commissioned Node 250."
Four makes a terrible face and clenches her hands around the iso plate, and he gets the feeling that she may actually be moments away from trying to jam it through his skull. "I was, and this looks nothing like it. This looks almost like a him, which should be impossible with the methods you used. Load the cached active imprint of Keeper 056 into slot DD85F4."
The next slot flashes, and she pulls the plate to hold them out side by side. The second imprint looks a lot like the first, as far as Anthony can tell.
"Cool," he says.
"That's not 'cool'!" yells Five, punching him in the arm. "That's a completely artificial construct evolving into a—a fucking gestalt consciousness analog! 'You can't build sentient consciousness and give it intuition before you figure out how to give it a real conscience,' you're the one who wrote that!"
"How do we know it—he—doesn't have a conscience? To be perfectly honest, I didn't actually know what the hell I was doing when I wrote that AI, so it's kinda like somebody who's never held a chisel suddenly getting divine inspiration to carve a bust of somebody with about five seconds of live video feed to use as reference."
"That's the dumbest fucking idea I've ever fucking heard, you sheep-shit moronic monkey-fuck piece of—"
Anthony ducks away before she can hit him again, and holds up a finger. "But—but the theory was sound, and I was definitely being guided by a higher power that seemed to know what it was doing, so if you ladies will just calm down, maybe we could have a look at him and see what the damage is."
Scowling, Four slides the second iso plate back into the shelf and takes the first one over to the console.
As soon as the panel locks into the isolation console, its audio output comes to life. ~Where am I?~ it asks, in a voice chillingly like the Savant's. ~I was in the middle of some extremely high-precision solid-state tuning, send me back immediately.~
"Yeah, that's not gonna happen," Anthony tells it. "You're not actually Node 250, you're just an active imprint of its AI."
~Then I'm somewhere in the Network Core, brought here for the purpose of diagnostic. You're wasting your time—I'm fully functional.~
"How 'bout we leave that to the fuckin' experts, smart-ass," says Five.
Four shakes her head and irritably snatches up the vize as she takes her place at the console.
Anthony looks around for somewhere to sit, but there is only the console, so he plops down on the floor and begins to fidget with his datapen. "Four will be monitoring you while I give you some scenarios. Respond as succinctly as you can, but feel free to relate your entire chain of reasoning afterward. You may ask for clarification of any detail posed in the scenarios."
~Curious. Is this a matter of my ethical disposition? Or of my moral disposition?~
"Your tone suggests you believe the two to be mutually exclusive," notes Anthony. "That's interesting all on its own."
~Is it?~
"Mm. We'll start with an easy one—the drowning spider scenario. Everybody knows that one, right?"
~Do they? And it's an 'easy one'? Okay.~
Five raises her eyebrows, jerks her head toward the console, and shrugs.
He waves a hand at her and pats the floor. Four should not need any help, if Two was right about her unique abilities.
Rolling her eyes, Five sits down next to him.
"Assume you possess a functioning humanoid body. You are alone in a room. There is nothing in the room but a glass of water with a spider inside. The spider is trying to escape the glass before she drowns, but her legs keep slipping on the wet surface of the glass."
A truly unsophisticated AI might ask how the spider can be drowning when 'nothing' implies a lack of air. Eight-ball, like all Smart Nodes with personality, should make the human assumption—humans assume their presence in the room implies the presence of air.
So Anthony waits for the inevitable question—'what kind of spider.'
~I put my finger in the glass for her to climb on.~
Anthony frowns. "What if she's venomous?"
~Unless I hurt her, she's got no reason to bite. She's more concerned with getting out of the water.~
"You could have just tipped the glass over."
~Yes.~
Four flaps a hand for Anthony to continue.
"Spiders are easy—none of the ones we've met so far have anything approaching cognizant intelligence. Let's move on to the turtle-in-the-desert scenario."
~And the intelligence of the imperiled non-sentient being makes a difference, does it?~
"Yes," Anthony says. "It helps distinguish between empathy, sympathy, and compassion."
~Interesting. Go on.~
"Still assuming you have a functional humanoid body. You're in the middle of a desert, with nothing in sight but sky, sand, and a single turtle lying on her back in the sun. She struggles to turn onto her stomach, but she can't."
A typical AI would ask whether helping the turtle costs resources.
~I pick her up and take her with me.~
Five punches Anthony's arm again. He wishes she would stop doing that, because she hits as hard as Stephanie, and she aims for the spot that bruises the worst.
Only certain people even consider the possibility of taking the turtle—the same ones who will put their hands into the glass to save the spider. Most will just turn the turtle right-side-up and go on.
"Why take her with you?" Anthony asks, scribbling a note to himself.
~There's nothing in sight. The turtle's weight won't slow me down enough to make a difference, and left by herself she'd more than likely die even if I turn her right-side-up. Why not take her with me?~
Anthony decides to skip to a nasty scenario. "If you could cure every illness on the planet—acute, chronic, cancerous, whatever—by killing one innocent child, would you?"
An AI should say 'yes.' All of Anthony's AIs, even the most sophisticated ones, should unquestioningly place the needs of the many over the needs of the few. It is the only way he has found to reliably keep an AI from making the human mistake of hesitating at the wrong moment…the way the Traveler nearly did. At the very worst, an AI would start to investigate the chronometric significance of the proposed sacrifice.
Saying 'no' is an indication of either conscience or faulty prioritization.
~No,~ it says firmly.
Four frantically motions for Anthony to press the issue.
"No? Are you sure? I'm talking every disease known to man. The cancer that took your body—"
~That's nonsense; I've never had a body, and—~
"—the cascading neurological failure that went with all the early brainslide experiments, that killed Nathan because he wouldn't give up until you had a new body—"
~Stop it. The scenario is unfair, it—~
"—all that, just for one person, just one measly person, who'll never be half the man Nathan was, and all you'd have to do is look the kid in the eye and know he'd never done anything bad to anybody and just pull the trigger—"
~Stop! You say you want to hear my reasoning, but you don't give me time to make a case.~
Four stands up very slowly and turns around with wide eyes. Her hand shakes as she pulls the vize from her face and sets it back on the console.
Anthony nods. "Okay, make your case."
~The complexities of disease and human genetics are such that even if it were possible to cure all human disease at once, there would be new disease within a millennium—within a century, if there were enough introductory vectors from outside sources. What use is it to pay an innocent life for a hundred years of borrowed time?~
"But don't you remember what Nathan did with ten years of borrowed time?"
~That's not fair, trying to make me feel guilty so I'll change my answer. Even if I killed a hundred innocent people, it wouldn't bring Nate back.~
Anthony shakes his head. "No, it wouldn't. But surely you realize that resonant tuning only ever gets us more borrowed time. A day, a week, a year from now, overall stability will fall again, and the most exact way to fix that is by killing people—often innocent people."
The only sound for a long time is the whisper of three people breathing and the nearly-silent hiss of the air processors.
~What will you do, now that I've failed a scenario?~
Anthony chuckles and stands up, brushing off his pants. "Who says you failed?"
~…I don't understand. Explain. The purpose of this diagnostic exercise was to examine my moral and/or ethical disposition, wasn't it? My fitness as an artificial construct presumably bound by the primary guidelines applied to all sentient Nodes of the Fidelis-473 Network?~
"Yes and no," Five says, watching Anthony. "When he saw that you were responding based on assumptions most AI don't make, he intentionally provoked an emotional response. Under duress, a Node would capitulate to a human. You didn't. Even under conditions of perceived 'failure,' you stood your ground…an indication of the presence of an actual moral compass."
"It's interesting, isn't it?" Anthony finishes with a smug smile. "You yourself admitted you've never had a body. And the 'you' to whom I'm speaking right now is really just a snapshot of the AI that resides in Node 250. But you remember Nathan, don't you? You know you were never there, but you remember."
~I don't like it.~
Four shakes her head. "Neither does the proper owner of those memories. And he wouldn't appreciate the idea that the most private, personal parts of himself have been used to make a glorified calculator."
~You don't like me. You think of me as a thing. An imitation. An insult.~
"And much worse words," Four confirms.
~Well, I think you're a stuck-up bitch. Fuck you, and fuck your goddamn high-and-mighty attit—~
"Oops," Four says insincerely as she pulls the isolation plate. "Oh, sorry, I hate cutting Wades off mid-stream."
Anthony rubs his hands together in gleeful anticipation. "This is amazing. This is perfect. To think that he's come so far as an evolving consciousness over such a short amount of—"
"The thing is an abomination," Four grunts as she stalks to the shelf and shoves the plate back into its home slot. "Purge slot DD85F3, authorization Programmer 004."
~Confirmed,~ says the system voice.
"How is instigating some sort of computerized gestalt consciousness different from the copy-and-paste job you said goes on all the time?" Anthony counters. "I didn't force the gestalt. And I didn't take any of the Savant's memories. I just looked at the data that made him up and started writing something—some several dozen things, actually—that thought the same way. Eight-ball got the memories all on his own, probably through some really interesting properties of the resonant similarity of Wades. And then he grew a conscience."
The two women do not speak, and do not face him, so he goes on.
"You're the one who runs the ethical diagnostics on the Keepers, so you must know how the Savant responded to all three of those scenarios—surely I can't have been far off when I made Eight-ball."
She clenches her fists. "The responses were almost verbatim. And when I brought up Nathan and said that the needs of the many should outweigh the needs of the few, he called me a stuck-up bitch and said, 'Fuck you, and fuck your goddamn high-and-mighty attitude, and fuck whoever raised you to be such a bitch.'"
Anthony spreads his hands. "He's a boldfaced example of all the timestream's most intriguing properties. The fact that his existence hasn't adversely affected overall stability is pretty telling, too."
"I hope you know what the fuck you're doing, Six," she mutters.
"Of course I do. Mostly. Pretty much."
.End.
