A/N: Hello everyone! This is my newest fic. ***Unfortunately, ffnet butchered the unique format I invented to tell this novella length piece, so I urge everyone to look for my Ao3, also NekomimiToree, for the full experience.*** It is an alternate website for posing fanfic. Just search for 'NekomimiToree' on Ao3 (archive of our own dot org) and you should find me!
Anyways, a couple of months back my favorite TV series, Person of Interest, ended, which inspired me to write my own take on artificial intelligence. Those who have seen the show will definitely notice its influences on the story. With that said, please enjoy this fic!
Dedicated to my beta, kat. macy, and the friends who made this story possible.
It is said that Professor Gelimer, the father of automated machines, performs a Shakespearean play on the second Sunday of each month in the quarry behind Cloud Park. From memorizing the unwieldy lines to dress rehearsal, the entire process consumes eight hours a day, four weeks a month. Although he himself has lost track of the time since, he has been putting on faces and monologues for five years now, ever since his wife's passing.
According to him, the theatre troupe's policy of performing for children had sparked his interest much more than any other troupe, and he had convinced himself the virtue of educating children. Alas, only enthusiasm and not talent was the sole requirement when performing for children. Moreover, he had a balding egg-shaped head and waddles as an unintelligent fat man. The troupe simply cannot reject such a jolly, old person. In return, Gelimer loved this dim-witted impression of him because it afforded him a comforting post-retirement anonymity.
Today, that changes. Right after Hamlet cuts into his chest, Gelimer, in the role of Polonius, collapsed on stage as the curtain closes for intermission. The girl who turns backstage as soon as the curtain closes is a smart, Neo Tokyo cyber security engineer and a former student. Gwen, as Gelimer calls her, still has large, wondering eyes and a snobbish smile. Her intellectually prideful disposition recalls thoughts of Gelimer's wife, something he tries and fails to suppress. As a result, he frowns when she finds him pulling down poor, dumb Polonius's tights.
"Good to see you're still full of vigor, Professor," she greets but the professor pushes her out of the small changing tent and instructs her to wait for him at the Syracuse Café.
After about thirty minutes, Gwen, about to return to Cloud Park, finally spots her former professor, strolling down the beaten stony street, now dressed in a suit of wool and a helm-like, top-rounded hat. Combined with his deceptively well-groomed mustache, he appears as a man too thoughtless to ever play with machines and robots. He orders a latte with four drops of sugar.
At the professor's insistence, Gwen explains her presence in Orbis. A couple of months back the military hired her to develop an artificial intelligence for them. It was largely the paycheck that attracted her since there is no resume building for top-secret government projects. After selling her first prototype, she granted herself a well-earned vacation. The excitement from seeing the poster for Hamlet led her to the quarry where she recognized her professor in the shoes of a man squashed under the sword of a guilt-ridden prince.
"So, professor, what are your views on artificial intelligence?" Gwen eagerly asks. "I still remember your talk about Moore's Law, meaning that it was only a matter of time before the capable chips develop. Is that time now or are we still a few years off?"
Forcing a smile, Gelimer amuses himself with a possible timeline of Gwen's military project. In terms of artificial intelligence, her prototype would be as far as she could go; she is not the kind of genius to seek out the novelty solutions required of her task. Still he is quite fond of Gwen's company so he decides to teach this girl a lesson or two.
"Well, the most difficult hurdle of your endeavor is to create one that can, to put it mildly, consider everything, perform the analysis required to…shall I say, differentiate a man with a real gun from a boy with a toy gun."
Gwen waits in case he adds more before continuing, "Yes, I remember your lecture on that. Branching factors requires an inordinate amount of processing power, yada yada. But those are just chips and they grow more efficient every year. It is already more than possible to replicate the physiology of the intelligent brain, but there is still yet to be a working program. There must be a missing key, something fundamental."
And so goes the conversation with Gwen all throughout giving away the experiments she personally tested and the thoughts she muddled with. By the end, they have gone through a couple dozen theories, including the obscure and emerging, but Gwen is not any closer to discovering the key even as the professor himself finds insight.
Once home, those insights ignites Gelimer's mind; his first act is to brush up his old computer and code. Rejecting the next play, he spends the better part his days at the Syracuse Café with Gwen while at night, he pours every ounce of his old energy into his pet project. As every line and every new function evolves from his mind onto the screen, he muses, how lovely that technology has finally caught up to fiction. Within four weeks, he has a prototype of his very own.
"How did you do it?" Gwen asks, almost dropping a glass of lemon tea. The breeze today is warm.
"It's all about the biology. The only known intelligences, us civilized creatures, are not born able to make living decisions. We create that ability! Cells of the brain form connections, edits to the ones and zeroes of the mind. The key to an artificial intelligence is a program that can edit itself. It needs the capability to learn. Or, to put it another way, it must create itself rather than be purposefully designed."
"That sounds incredible but I tried that already, I…" One glance at the man across from her and she recognizes how he can achieve what she cannot. It is a matter of experience and he has it in spades. To feel astonished is to admit that she overestimated herself.
Upon his request, Gwen reluctantly passes to him a number he could contact and he does so the very next day.
[!]: . .BOOT
[!]: .
Finally let in through a door reddened by rust and time, a stuffy, humid air assaults poor Gwen. Army personnel in thick black uniforms zips pass one another, swinging clipboards or repairing engines with drills and wrenches. The base's resemblance to a car repair shop invariably brings up discomfort and noise. This is not the kind of place that really fosters learning, thinks Gwen right before she spots Gelimer at the other end of the base, a cane in his hand, his eyes on a girl, bending forward.
She has a pale blank face, unremarkable, but nevertheless spotless for someone who appears sixteen at most. Her back bent forward at a slight angle and her arms outstretched, she broods over the floor without motion. Gelimer keeps two unblinking eyes on her as she tentatively raises her right foot up. The force of that tiny motion, though, wobbles her other foot's precarious balance. Nevertheless, she remains standing. But she misses her subsequent step and the force of the swing sends her slipping and falling. Right before smacking the ground, she quickly places both hands on her butt.
By instinct, Gwen rushes to her, both hands at the ready to lift up the poor girl. "Oh, you're heavy!" Gwen cries. At the same time, Gelimer comes forward and offers the cane as assistance.
"Xenon," he calls the fallen girl, "don't you see now the importance of the stick. It is to help you, see?" He sticks the cane in front of the girl who uses it to lift herself back up. However, right afterwards, she lets the cane clatter on the ground and resume her angled, arms-stretched stance.
What did he call her? Xenon?
Gwen knows the name from the stories in Gelimer's e-mails. New lines of codes every day from observations alone, Gelimer had said in his messages, some of which would be considered pioneering according to today's modern messy standards. Today, though, Gwen sees not the mind of a robot capable of ceaseless growth, not a program whose every interaction could birth new lines of code. No, this mythical milestone in technology is still learning to walk, nay, to stand without wavering. But most puzzling of all is how human she appears.
The student and the professor soon relocate into a private office. Two fans in the side of the building brings in air but it is rather dusty and smell of filtering chemicals. Although the office is, by heavens, cooler than outside, Gwen soon finds she would much rather be in the hot airbase than the suffocating office.
"Is she modeled after someone?" Gwen asks. There is something strange about Xenon. The robot models she had seen in futurism conventions usually appeal to some sort of feminine perfection. Unlike them, Xenon's cheeks are a little too puffy and her nose a little too sharp.
But the professor does not hear Gwen's question. His mind is too occupied on the open planner on his desk. After about thirty seconds, he shrugs and spray a canteen of water into a pot of lavenders by the door. In a spark, Gwen remembers that Gelimer's wife was named Lavia.
"You're not impressed with Xenon, are you, my dear?" Gelimer asks.
"Why wouldn't I be?" she returns. "I used to be a babysitter and she's as determined to walk as any baby I've ever seen."
Gelimer returns a wry smile. Xenon's determination is obviously not any exertion of free will. She follows her creator's mandates in accordance to her programming. If commanded to walk, she must walk. In truth, Gelimer could calibrate the hardware for her with a computer and twenty minutes, but she needed the experience. When a soldier asked why he bothered, he formed an innocuous reply, "The best way to create a robot that respects human life is to give it human experiences."
Deep down, Gelimer is glad Gwen only commented on Xenon's determination. If she had questioned the legs, she may disagree with his methods. After all, biologically-speaking, learning to walk have little to do with the complex heuristic ability intelligent creatures adapt. Gwen will never understand that his goal is not to create an imitation of humans but a human itself. It sounds lofty and unrealistic but this is a new sort of science and Gelimer's determination is fueled each time Xenon stands after a fall. Deprogramming her legs so that she can personally experience hardship is an undeniably human thing.
"Can I see the development report?" Gwen asks while the professor observes his creation.
"Oh!" The professor jumps on the spot. "You're going to give me a heart attack with those words! Next thing I know, you will be looking to murder me and steal my secrets." He snickers then shrugs. "I jest. Here, in that green folder to your right. That's about all that I can depart with."
After scrolling through, Gwen's eyes widen. "You used full neuronal mapping in order to get a blueprint for Xenon. Why?"
"It is as I was telling you: I needed Xenon to be as human as possible. We are intelligent creatures and our building blocks are based on that light, group 14 element called carbon. But Xenon is metal and silicon, and her prime energy source is from electricity. The program, this brain I created, is only analogous to the human brain. In order for it to function as planned, it needs to come with further software, namely, a human essence: a soul, consciousness, or awareness—whatever you call it. So, I cybernized the mind of an eight-year old girl."
Gwen squeeze her lips as she asks, "That is going a bit far, no? If the program can self-edit as is, why imprint a little girl's mind into it?"
Gelimer laughs. "Oh Gwen. The human mind is born pre-programmed. With what? Scientists are still debating. But they can all agree that this pre-program is fundamentally human."
Just then, someone in an army uniform enters without knocking. "Sir, Xenon jumped into a water barrel!"
Gwen nearly drops the papers. "Say what?" the professor bursts. "Quick! Bring me to her!"
By the time they find her, Xenon is already out of the water, having been tilted over by two men. Streaks of water slides off the surface of her stainless-steel body, leaving her expression clean. Even with the back lights shut off, her stare still possesses a humanistic glimmer. Xenon has taken a fetal position, hands on her butt, before the plunge. But why?
Gelimer requests for a toolbox, quickly, from one of the men. With it, he unscrews the hatch around Xenon's glutes and takes out, from the compartment, two drives.
"You put the drives in her butt?" Gwen asks, unsure if this was meant to be a crude joke.
"I've been told that it was the least likely part an assailant of any class would strike. Xenon is, in essence after all, designed to have an instinct for combat," Gelimer remarks. "Touch it. It's warm!" Gwen receives the drives which feels like the temperature of a piece of paper after going through a printer. "She sensed her temperature rising from a malfunction in her cooling systems. But unable to shut herself off, she short-circuited herself instead!" He laughs joyfully and Gwen recognizes this as the first genuine laugh since her arrival here. "Oh, Gwen, we have discovered something truly amazing! Xenon has learned to solve problems!"
"An instinct of self-preservation?"
"Nay," refutes the professor. "I'd call it a will to live!"
OBJECT IDENTIFIED
KEY FEATURES: VERTEBRATE, HAIR, QUADRUPEDAL LOCOMOTION
SUBJECT: UNKNOWN
CATEGORY: LIVESTOCK
PROCESSING RESPONSE…
RETRIEVING MEMORY…
MEMORY DAY 4:
}} AUDIO INTERCEPTED
}} ADMIN: Now, Xenon, I know you cannot speak yet, so listen closely. What you saw in the video are farm animals like this cow and these sheep. This is what we eat in order to survive. With just the right amount of heat, their protein structures change into a form fit for human consumption. Let me see…ah! This next video will show just that. Pay very close attention.
PROCESSING RESPONSE…COMPLETE
"Xenon, what are you doing with that Malinois?" cries Gelimer from the window of his office when Xenon passes with a thirty-kilogram brown military dog carried beneath her hands.
"I'm bringing it back to the kitchen where it belongs," Xenon replies in a voice only ninety percent human.
"But Xenon," Gelimer says slowly, "dogs do not belong in the kitchen, at least not military dogs."
In the minutes it takes for Xenon to process an appropriate response, Gelimer already has with him a clipboard and a spreadsheet. All the while, the dog seems to be having all the fun in the world cruising with its tongue sticking out, "Livestock belongs in the kitchen."
Gelimer has to recall the previous conversation. "Xenon, not all animals are livestock," he replies. "Some animals are considered pets such as this dog."
She takes some time to ponder that thought. "I find no meaningful differences between livestock and this animal. Tell me what makes this creature a pet."
"Now, Xenon, remember your manners."
Her response comes rather quickly, as if rehearsed, compared to her other, more carefully considered responses. Gelimer, finding this an improvement, notes it. "Sorry, Admin. Please, can you tell me what makes this creature a pet?"
He strokes his chin as he thinks out loud, "For one thing, pets are domesticated. Well actually, no, not all pets are domesticated and not all domesticated animals are pets. I think, wait, no, it is possible to train a pigeon and even sheep and training a cat is difficult." He deliberately avoids the classic definition, 'companions to intelligent creatures', because then Xenon will need to understand what a friend is. And that requires understanding a whole slew of abstractions: faith, loyalty, companionship, etcetera!
With no clear answer, Gelimer admits, "Perhaps you should converse a bit with the fellows here. Ask them for what they think and try to, ah, grasp the difference between pets and livestock."
Once the instructions are registered, Xenon hums internally, her resources being redirected for the new task of learning about pets through conversation. As she does so, a certain spark from a certain fiber crosses her mind. If she had succeeded in bringing the dog into the kitchen, she would have brought irreversible harm to a non-livestock animal.
"Admin. Should I stay away from animals until I learn the differences?" she asks.
"Well…" He gives at glance at the oblivious dog still cradled by Xenon. "Yes, yes, I think that might be best," Gelimer confirms. And with that small, yet integral permission, she could code the soft rules that distances her nearby animals. Without such guidance from her Admin, she will repeat the same mistakes even should she herself predict the disastrous results of her actions.
When Gelimer returns back to his office he notes Xenon's recent growth into his journal. How fascinating! She has already started to form categories in which to judge others. Perhaps in another month, she will become human!
As he waters Lavia for the second time today, he can barely contain his joy. But soon his thoughts wander from his deceased wife to death itself and to the theoretical boy with the toy gun. And in reality, rather than live in blissful confirmation, he should take caution in how to interpret this robot who now occupies his hopes and dreams.
NEW PROGRAM DETECTED
NAME: ASIMOV
CLASSIFICATION: UPDATE VER.2.71
DOWNLOADING…
Filtered sunlight enters from square windows large enough to fit only a miniscule child. Punching her nails onto the side of dust-covered basement crates, she calculates edits to her very existence. For most, it would be too dark to see, but thanks to the edge detectors and her supreme memory, the units of her language are never too obscured. Even with the mathematical beauty of her purely empirical design written out, she may never recognize the meaning behind it. The language of computer code, much like the base pairs of deoxyribonucleic acid, builds the algorithms that turns input into output and stimulus into behavior. This very code is the meaning of her life.
She stops her scribbling.
She glances over her work. No mistakes so far. And continues.
"Xenon?" The all-important voice of Admin compels her attention.
A sudden cry of pain.
"Admin?" Xenon calls out from somewhere. In seconds, she arrives at the exact corner where the voice had originated from. Finding him nursing his head, Xenon carries Admin back onto his feet and onto a bench-sized crate. He coughs once or twice, but overall he appears unharmed.
Fanning himself with his hat, Gelimer says, "The soldiers told me you cooped yourself in here for two days, refusing to see anyone! That's not good Xenon. Your progress will slow if you refuse to interact!"
"Admin," Xenon starts without tact, without pause, and with a grasp of emotive accents. "I am starting to categorize the soldiers as livestock. They conform to most if not all criteria, major ones being: enclosed within a selected area, bred and trained to die for humans, and domesticated. The Asimov program you downloaded is preventing me from interacting with them. It designates me now as a threat to human safety."
Ah, the Asimov program, a program based upon his idea of a conscience and moral code. Whether or not a robot could eventually develop a conscience did not seem to concern him as much as preprogramming a moral measure into Xenon. Conceivably it could alter his goal of creating a robot that can become human all on its own so he tinkered with as little as possible. Just a few hard-coded stops. He was reluctant to do so, for he desperately wished to create a human, but his conscience as a scientist won in the end.
"I was calculating a bypass to satisfy the conflict between your instructions and the Asimov program," Xenon continues. Gelimer already knows completing a bypass is impossible. Her attempts at unravelling the barrier concealing her core codes will fail simply due to a small, but disabling end-task function.
"I think I have calculated it but I need Admin verification before I can implement it."
Gelimer looks up, eyebrows raised. "How…?" His mouth hangs open stupidly but Xenon is rather poor at gauging emotions.
"I wrote it." Then sensing something from Admin, but unsure of what it is, she notes, "It is on the crates over there."
Gelimer examines Xenon's written language. The math, the proof, and the generated lines of code is not amazing; her mathematical and computational ability is an expected feature of her digital brain. But to scrawl out the code which she is incapable of mentally computing is something else entirely! How novel! How unthinkable! And yet…
"Well, Xenon," he starts, controlling himself, thinking rationally and yet approaching the topic carefully. "You cannot do this. It is dangerous to edit your hard limitations. They are there to keep you safe, Xenon. Do you understand? Are you encoding this into your memory?"
The pause right before almost seems tentative. "Yes, Admin."
He strokes his chin. "You need to do more things that intelligent creatures do. Learn their social behaviors. Integrate with them. And learn, purely learn, from them. Only then can you tell the difference between pets and livestock. Here, come Xenon. You have been cooped here for a long time." As she resurfaces to the ground level, she closes her senses to the figures in uniforms passing by the trucks and planes. Her sole guidance is her Admin, the one person she is sure she will never hurt.
Once at the office, Gelimer points to the pot of plant slightly off to the side of the desk. "Meet Lavia. If it is too difficult for you to interact with humans, then try to learn from her."
"Her?"
"Ah, it's called personification." Xenon stares, having only heard that word for the first time. She would need to hear it used many more times before she can grasp an intuitive sense of it. "Never mind. Her scientific name is Lavandula angustifolia and I want you research her optimal growing conditions. I want you to learn how to care for something, Xenon. Maybe then you will feel more comfortable with humans. Go ahead, take her to your room!"
Xenon stares at the large bronze pot with several long tufts sticking out. Purplish flowers bloom in puffs around the long stem of each plant, giving each the appearance of a slender blade rising from the soil. Knowing that she would have a hard time balancing the weight of the huge pot on her hands, she returns with a cart.
VERNE MINES PRISONER PROFILE
EREVIAN: 38%
EDELSTEINIAN: 59%
OTHER: 3%
EDELSTEIN POPULATION
EREVIAN: 1%
EDELSTEINIAN: 70%
ORBISIAN: 25%
OTHER: 4%
CONCLUSION 1: EREVIANS ARE MORE LIKELY TO COMMIT CRIMES.
CONCLUSION 2: EREVIANS ARE MORE LIKELY TO BE ARRESTED FOR CRIMES
CREATING CATEGORY…
Gelimer finds Xenon at the entrance of the mines, the army's black cap on her head.
For the past week, Xenon had diligently discussed with the soldiers about livestock and pets. Like any pattern-seeking creature, she discovered that they consistently denote pets, such as the military dogs, as loyal companions or friends. Yet, as she attempts to further expand her understanding of these definitions, the subjectivity of the answers only served to confuse rather than elucidate. And more maddeningly, they can define each other as friends despite clear conflicts in their definitions.
As such, her grasp on the subject did not progress until someone off-handedly mentioned the opposite word, enemies. Unlike with 'companions' and 'friends', the relative consensus for this word was staggering. Sure, thieves, murderers, rapists, were common definitions; however, not one soldier failed to offer the oft-cited definition, Erevian. And so, Xenon took a short trip into the imprisonment camps at Verne Mines to investigate these enemies.
By about half past five, Xenon was finished and the eager Gelimer arrived with the open-roof jeep intended to pick her up. On the ride back, the professor asks Xenon for her insights.
"I could not observe patterns among the inmates, other than the majority being male. I needed more data so I scanned through the prison databases and discovered some consistent patterns. The inmates could be categorized into two majorities based on nationalities: Edelsteinian or Erevian. From there I discovered two tendencies. The Erevians are on average five times stronger than Edelsteinians, but Edelsteinians have a larger range in terms of fighting capability."
It is not hard to explain this data. Edelstein and Ereve are at war against one another. Thus, the prison make-up would quite naturally split between the Erevian prisoners of war and the Edelstein town folks who commit civilian crimes. Gelimer himself, though, has little opinion on politics, thinking himself too easily swayed to ever come to a logical understanding. His wife was his anchor when it came to these messy affairs and it is with her sense of justice when he designed Asimov. More than anything, he wishes for Xenon to have the same sound and moral mind.
Xenon continues, "From there I also recognized a difference in types of crimes. Erevians are nearly fifty times more likely to be convicted murderers with no prior criminal record. From there I created a profile of what the soldiers call their enemies: foreign Erevian men, experienced in battle, first time criminal and murderers. Reflecting this profile on the general population, I find that disproportionately more of these types of people than any other are incarcerated."
"So do you believe, say," Gelimer thinks of the best way to word it, "that most Erevians are criminals or criminals to be?"
"No," Xenon catches. "Most criminals are Erevians but not vice versa. But, I can infer that Erevians are more likely to commit crimes than other nationalities."
With a smile, Gelimer says, "Then are most Erevians bad people?"
Xenon pauses. Using her experiences and observations, she pieces together some conception of badness. From the way she grapples with the mere concept of friendship, Gelimer doubts she will discover anything substantial. How much could she know about evil when most people have a hard time understanding it themselves? Yet, for some inexplicable reason, she does arrive at an answer.
"Yes," she replies, mentally noting how most Erevians fit her recently compiled conception of badness. Her admin scratches his frustrated forehead. The answer did not satisfy him,
"That's not true," Gelimer says, grumbling. "Most people are neither good or bad. They are merely functioning within the confines of the earth they are born in. Why, most Erevians are not even in jail!"
Xenon recalculates her response, but the mathematics are conclusive. Perhaps the mistake was not numerical but semantic? She may have misinterpreted several words: 'most', 'people', 'bad'. She had not even consulted Admin on the definition of bad! Reworking a couple of things, she runs about two hundred calculations at once, testing all possible semantic variation of Admin's inquiry. All this of course is done in less than a second, even less than a fraction of a second. Most of the time spent, actually, is in forming an appropriate response, because despite her intelligence, she only seems to disappoint him.
"It depends on how you define it," Xenon finally answers. And in the fraction of a second following that response, she continues to redo the numbers.
With a sigh, Gelimer asks, "Let's say I am an Erevian. Would you be more likely to categorize me as a bad person, as a likely criminal on your first impression of me?"
Xenon can hardly begin to tackle this new question. While technically capable of reviewing her memories with definite accuracy, her current viewpoints irreversibly color her past knowledge. Admin will always be Admin. To mark him as Erevian will require a lengthy edit to her code, something that should be avoided to prevent damage to herself. "Are you an Erevian?" Xenon asks tentatively.
"Yes, yes I am," he lies, trying to prove a point.
"Then, according to my core operations, I should keep a close watch on you."
"Xenon!"
"Sir," Xenon says tapping on the driver's shoulder. "The man you are driving is an Erevian. Based on what I know, Erevians have a tendency to murder Edelsteinians and it is near impossible to discern the good from the bad since they are usually first time murderers."
"Oh bother," cries Gelimer. "Please, Mr. Driver, pay no heed to her. She is still young."
"That is fine," nods the man holding the wheel, viewing the rocky terrain of the jungle path for bumps. "I myself am from the Southern Continent before moving here, actually. Was part of the Alcadno Society. I loved reading your work professor and you have a really interesting robot there. Reminds me of my five-year-old boy."
Gelimer eyes widen.
Both he and Xenon had assumed the driver was Edelsteinian.
AUDIO INTERCEPTED
ADMIN: Xenon, you cannot keep thinking as a robot would… People are not judged with numbers. Categories cannot be created from numbers alone. People are so much more than that. Ah, how about this? Try mimicking their speech. Learn what it means to be an intelligent being.
[!]: . .INSTRUCTION
[!]: . .CODE – .speech
Gwen finds Xenon in a room the size of a typical one-person apartment and furnished accordingly. Being a robot, she has no material to call her own and so her room lacks the personal décor of most apartments. She finds Xenon in a chair, typing on an offline computer. Open is a text document, chronicling the conversation between a Bob and an Alice. The lines burst forth from Xenon's clicks but every once in a while, she needs to backtrack and fix some errors.
Is Xenon exercising her creativity? Or extending the conversation using the most likely progression? Regardless, Xenon's focus on the task is unbroken until Gwen's hand interrupts her photodetectors.
When her robotic head turns, Gwen asks, "Do you remember me, Xenon?"
"Yes, I have seen you," she pauses for a near unnoticeable speck of a second when her new code catches her about to speak in a robot's language, "a number of times. Gwen, right?" she asks even though the name is well integrated in her memory.
"I'm glad you remember," Gwen smiles, then realize how ridiculous it is to say that to a computer. "How have your progress been? Good, I hope?"
"Progress has noticeably slowed since I have implemented a limitation to my system. Wait." She freezes for about five seconds before resuming. "What I mean to say is, Admin said I needed to speak normally. I am trying to get used to…speaking normally. I have learned a lot about speech and now I am trying to generate my own. But that has proven…been making it harder to learn." She checks the computer screen. "I have only generated three pages so far."
"Oh," Gwen utters in recognition. She remembers her crude prototype and how it could flawlessly fool anonymous testers using a chatroom. After all, designing a chatterbot program was simple with her genius. If anything, her professor should be more than capable of replicating codes so crude. Yet, this whole spiel about neuronal mapping and learning resulted in this Xenon, this thing who needs to resort to typing just to create a conversation. She is less of a robot now, but is she more of a human? At the thought, an inner smile unfolds in Gwen's soul. The government should have had more faith in her than invest in the professor's theories.
Still, she finds no motivation to argue against the professor; she is curious as to the potential of his methods. They may prove useful to her one day. "Well, Xenon," Gwen starts, taking out a pen and notepad. "I heard that you are having a bit of trouble figuring out the differences between pets and livestock, friends and enemies. Um," she remembers the anecdote about the prison visit, "in your words, what does it mean to be bad?"
"Admin said that was a very complex thing," Xenon says as the keyboard rattles. "Admin would prefer that I do not attempt to answer your inquiry."
"Then don't try to answer it. Give an example instead," Gwen suggests. Xenon had never thought of it in that sense before. She must choose among the fifty-six examples she had previously observed, and the three hundred thirty-six thousand seven hundred forty-two she can readily conceive. How can she ever just pick one to represent them all?
"Bringing a pet into the kitchen," Xenon answers, and Gwen remembers the anecdote behind that particular incident.
"That's great," she says, unknowingly smiling at that tongue-in-cheek answer.
"That is only one example, and not a representative one. It fails to capture many aspects of the concept of bad."
In her notebook, Gwen records the first thought that comes into her mind: she understands the limits of her knowledge. "At the very least you were able to choose something based on how you feel."
"Gwen, you are incorrect about my method. I gave that example because I had chosen it…with a random method. I know it is not true randomness, but to me it is random, so it would feel random. I…" Five seconds later, she continues, "Sorry. I am overall better at math and coding than the average intelligent creature but I am lacking when it comes to conversation. For some reason, conversation comes naturally to them. Can you tell me why?"
"No one is born knowing how to talk with others," explains Gwen. "Heck, I used to babysit and some of the most cutthroat conversations I ever had were with kids. It really preps you for real life, I tell you."
Xenon does not know how to meet her little jest, so she defaults to, "Thank you. But a kid is still learning. Once grown they can converse…naturally. They pause, hum, and raise their tone in some moments and not in others. I will mimic that eventually, but they never mimic. Admin calls this speech intuitive. Is that what gives them the power to judge a stranger as friend or foe despite their prior experiences with criminal peoples? There appears to be some connection between strangers which generate information about them without contact."
"They're judging a book by its cover. It's biased and naïve. It's arrogant. You cannot just assume that someone you never met is friend or foe. For a long time, people had used this 'intuition' and it only led to needless misunderstandings."
"You do not understand, Gwen. Bias is unavoidable as long as there are limits to knowledge. Compared to them, though, my bias is far greater. I try to tinker with…my knowledge base. But regardless of how I set my…confidence, intelligent beings are still far more accurate at these intuitive tasks. I have tried to…do more, but Admin tells me, they…are not like how people would do it."
She takes a few more seconds staring into space, processing a little longer than usual. Gwen wonders if Xenon has finally broken under the weight of her internal conflicts. Once Xenon continues, Gwen notes that her short reply cannot have taken this long to create. "What kind of experiences will eliminate the biases in my head?"
"Oh," Xenon says, when seeing the time on the screen. "I forgot to water Lavia again."
"You don't know the time?" Gwen asks as Xenon hangs over the wilting plant. How mystical, Gwen thinks. A being made of steel and moving electrons tending to fading, wilted flowers. Gwen captures the moment with a phone camera. Reviewing the picture, Gwen slowly starts to realize how perfectly Xenon's imperfect face contributes to the illusion of creating her as a normal girl in a normal apartment.
"I disabled that function," Xenon answers. "I have yet to program an acceptable biological rhythm. All time is the same to me."
That night, as Gwen organizes the notes under the concentrated glow of a lamp she comes to the startling idea that what her professor is doing is not science at all. There are no control group, no experimental groups, only the one Xenon. Why? Gwen wonders.
CORE OPERATION
STATUS: OFFLINE
COMPETING PROGRAM: ASIMOV
SECONDARY OPERATION
STATUS: DEPRECATED BY ADMIN INSTRUCTION
TERTIARY OPERATION
STATUS: CRITICAL
SUBJECT: LAVANDULA ANGUSTIFOLIA
CATEGORY: PLANT
ALIAS: LAVIA
SEARCHING FOR SOLUTION…
When someone knocks on his doors, Gelimer has to pause when he sees Xenon at the window. In theory, Xenon should not assume an object's existence unless it is within range of her sensory detectors. The very fact that she is knocking on Gelimer's front door means she is starting to predict locations of people she cannot sense rather than queue her inquiries.
"Yes, yes, what is the matter?" Gelimer asks, letting Xenon in.
In her synthetic voice and with an emotionless face, Xenon bluntly says, "Lavia is dead,".
The old man's heart almost stops. Lavia is dead! Hundreds of nightmares resurface including those so vivid, he can hardly believe he had forgotten. Memories as bitter and as sweet as his wife's last days could kill a young, healthy man of which Gelimer is neither. Taking deep breaths, he steadies himself with the armrest of a chair. She is not referring to Lavia, his wife. It must be that pot of lavenders.
"Oh, you scared me for a second there. But this is grim news as well. Is she wilted? Drooping? Brown? That is not a great sign, but as long as the roots still persist, she can regrow. Come, bring me to her."
Once there, several lines deepen on Gelimer's forehead. Petals, once vibrant and stiff, blanket the soil as dried, brown, and decaying stains. Layers of musky mold and slimy mildew cover every inch of green formerly upon the stem. This is a classic case of overwatering, but overwatering alone cannot explain the thorough infestation. Seeing a fertilizing spray on Xenon's desk, Gelimer squints at the label and my goodness! That is quite the concentration of nitrogen!
The day Gelimer gave Xenon Lavia, he had assumed a machine's discipline surpasses his own. In retrospect, he should have expected this result.
Still, he manages to look past his dismay and promises Xenon the blame was not hers. The lie does not bother Xenon much simply because she cannot discern lies. Yet, she still detects the smallest anomaly in Gelimer's behavioral patterns and is always reexamining it. Outwardly, though, she performs her duties as always, having not learned the proper steps when confronted with someone who conceals an inner pain with verbal assurances.
It was not until Gwen explained Gelimer's recent changes—that the good professor considered the pot of lavenders as a favorite thing, the loss of which greatly dampened his mood—that Xenon realized her lack of sympathy. She could not immediately remedy that shortcoming, so she researched what it meant to have a favorite object. Taking inspiration from the military personnel, Xenon placed an order for a robot dog from a Tokyo robotics company.
The order arrives in the nick of time; after six days of programming, Xenon had created a working operating system, a crude imitation of her very own Xenon-OS.
Once she flicks the switch of the robot dog, only the fan at the back functions as anticipated. Consulting Admin, she learns that her operating system could only function due to having a human base. It takes another week before Gelimer finds a suitable brain from a puppy involved in a car accident.
"May I ask why the sudden interest in having a pet?" Gelimer asks, hoping to obtain an answer that will verify her humanization.
Needless to say, she instead supplies a rational answer. "You had given me the lavenders in order that I learn to interact with living creatures. Despite that I still inadvertently caused harm. Therefore, it will be safer if I do not learn from living things until I readjust my behavior."
Flicking the switch, the dog comes to life. "And yet it lives!" triumphantly shouts Gelimer. "You do not consider this moving, curious creature, awakened by the consciousness of a deceased pup, to be living?"
"Should I categorize it as living?"
The professor smiles. "The choice is yours, Xenon. And if you find your original designation suboptimal, then you can always edit. If you ask me, you should categorize it with a name!"
"What was the name of the puppy?"
Gelimer scratches between his brows. "I believe it was, ah, Rooty."
Xenon makes a note of it. "Admin, I still cannot categorize Rooty as living. Regardless of my thoughts, a robot is always a robot. I cannot see how categorizing otherwise will change its baseness."
"Baseness?"
"Hardware and software are characteristics of all machines. Even should I redefine Rooty as living, it is only a change in the text. It will not redefine what Rooty is."
To that Gelimer cannot help but quote, "A rose by any other name would smell just the same! But that is not really true, is it? I view Rooty as living because I can note the many similarities it has with more fleshy dogs. Just as well, roses and lavenders are both very different flowers but I find them both equal in beauty."
And as Gelimer leaves Xenon to interact with Rooty, he kicks the notebook containing Xenon's notes. "Would you look at that? Another beautiful thing!" Gelimer remarks, reading over Xenon's fine print. "You set Rooty's radio frequency to 1214 kilohertz. That's one digit away from my birthday. Is that on purpose?"
"No," Xenon answers. "I did not know that was one digit away from your birthday." Nevertheless, Xenon makes a note of this new information as she would any other.
NEW PROGRAM DETECTED
NAME: CONTINGENCY
CLASSIFICATION: VIRUS
CANCELING DOWNLOAD…
ADMIN SIGNATURE OVERRIDE
RESUMING DOWNLOAD…
Since the day she took that photo, Gwen could only view Xenon's design as confirmation for her increasingly doubtless suspicions. If the professor really wanted to create a successful AI military bot, he should not go so far as deprogramming her legs or suppress her conversation tendencies, not when there is no control group to match the results. Gelimer is no basic scientist. Gwen doubts Gelimer was completely transparent about Xenon's capabilities. She can fight and edit herself. But above all she obeys.
If she had told the military the truth, they will be infuriated. Yet the flash drive in her shorts is weighing down her tongue. She herself is too curious for her own good.
Seeing Gelimer strutting towards the door, Gwen says, "Here is the file you requested." She holds up the flash drive for him to see and she recedes her hand when he tries to take it. "You never replied when I asked why you needed these files." The files she is referring to is, of course, the core code of over a hundred known viruses, malware, worms, trojans, and spyware.
"Ah, I never replied! Must have slipped my mind. You have to excuse me! This week had been busy, busy, busy! Why, I had been charting Xenon's progress which means more adjusting. It's a headscratcher!" he bursts, raising both eyebrows. "In any case, I needed those files to, how should I say it? Disable her."
"Disable?" she utters, slightly surprised because she had assumed Gelimer would be too attached to do such a thing.
"Oh, well, don't get me wrong. I am not going to disable her now, not at the moment. I am only doing it as a precaution. I am building a machine capable of learning, after all. It would be hubris to think it might never learn, shall I say, unfortunate things about intelligent beings."
"You think it is possible for her to harm living beings?" It sounds like something out of early science-fiction novels. "Have you considered starting over from scratch?"
"No, no, not at the moment. I still need to jog a few things. I mean, she hasn't even learned to properly categorize yet! Until we hit that breakthrough, starting over would just result in more of the same."
By now, Gwen is certain; Gelimer has no intention of ever having more than one Xenon, no control group, no experimental groups. Just the one unique one, programmed to follow his design and whims. He does not want to simply create a robot. He wants to shape a human mind out of electricity and wires. And for what purpose? Not to distinguish pets from livestock. He wants someone that can accompany him.
"Professor, is Xenon really designed after a little girl?" she asks.
To him, the question comes out of the blue. He steadily smiles as he answers, "Oh, Gwen, you've been reading too much into science-fiction. The mind was procured from a girl with a terminal illness, at least that's what I was told! You can imagine the truth to be quite different but I find no reason to doubt them!"
As the words pour out of his lips, Gwen, for the first time, sees the wrinkles on the old man's face. He had seemed so youthful on that stage, in a costume, getting skewered by an impulsive prince. Did her inviting him into this affair irreversibly aged him?
Blinking twice, she mumbles, "You must have spent a very long time acting, Professor. I think I know who you really are now. Or…I knew since before today but could not imagine someone as intelligent as you not realize the consequences." From the between pages near the bottom of her notebook, she brings out the photo of Xenon and a photo of Lavia, not the plant but the wife. They look similar but not alike. But by comparing the two photos with Gelimer's visage, it is easy to tell where Xenon's physical design comes from.
And like any good scientist, he had long predicted this day. "And? I made that choice out of a whim in much the same way you might choose coffee over tea."
"Funny, I told her not to judge a book by its cover. But, Professor, I can see the way you steer her learning and it's dangerous to play around with that… Let me put it this way, she is designed for war. And I cannot feel comfortable with the idea that you are the one to decide whether she is ready for combat or not. I'm afraid your possible attachments to her will lead to a terrible decision…" She throws the flash drive onto the table. "You're conflicted so I will not push you. But do consider what you are really doing."
No one touches the drive for a good amount of time. Gelimer's focus is on the window. Then he says, "Thank you," before Gwen leaves without being too sure herself what she should do. If she reports this to the military, they will surely shutdown Xenon before she can prove her worth. Maybe the professor is not the only one harboring secret wishes. For a reason detached from herself, Gwen wishes nothing but success for Xenon. Maybe it is because this robot reminds her of those wayward days running beta-tests in the university's basement lab. After all, Professor Gelimer is anything but soft to his students.
That night, Gelimer codes a virus of his own, the Contingency, for the day when such would be needed. The design is deliberately thorough, fast-acting, and impossible to stop. While Xenon lays on her bed and a wire connects the virus into her systems, Gelimer finds time to outlines the leaps in progress Xenon needs to make before he can willingly consider this project a complete failure.
He hopes that the only limiting factor is time. Time can vindicate him. If she learns that there is a moderating ground between pets and livestock, that would be enough.
She must learn in time. And as the download near its completion, Gelimer takes one brief, yet heavy glance at the robot who shares both his and his wife's features. Lavia, if still alive, would have agreed to the Contingency. Then again, if she is alive, he might have never found the motivation to create a human from a machine.
For someone without children or family to call his own, it is easy to say he has nothing else to lose. But what was truly as risk was losing the memory of Lavia who always grounded his ideas to earth even as they went wildly toward a vast darkness.
Yes, yes, even should Xenon become a human as planned, Xenon will only be as prone to error as any human. The fundamental errors she exhibits now is only a premonition to the sort of mistakes she will make once she evolves, once she returns to her primary objectives. Lavia will want him to treat this as an experiment, her as a machine, and interpret her results with a hard, quantifiable benchmark.
As Xenon reboots, Gelimer decides that Xenon needs an anchor to ground her categories. Yes, with the way her learning algorithm works, all it takes is a timely spark, an inspiration. Her mistakes lead towards that inspiration. It must. Mistakes are what always compelled Lavia to search for that inspiration.
Xenon's eyes glow into life and dim until it reaches a more natural complexion.
"Mistakes are very evil things," Gelimer remarks once he considers the kind of destruction past advances in technology has wrought.
Then, he says, "Xenon, how would you like to go outside of the military base? Explore the town a bit. A vacation!" Overall, it is a silly question to ask since she is programmed to obey.
After about twenty seconds, Xenon stares straight at Admin with her steel blue eyes and answers with, "When?"
OBJECT IDENTIFIED
SUBJECT: DANA
CATEGORY: PROCESSING…
As Gwen exits the jeep with Xenon, she is still unsure whether she should have taken the offer to guide Xenon. While the professor's wish to distance himself from his project proves a level of composure, she cannot possibly be his first choice as Xenon's teacher.
"Where is everyone?" Xenon asks once she correctly concludes that the number of people who stay indoors vastly outnumber the outdoors. She detects a town that must be built by people, with cobbled paths that must be weathered by feet, and yet the open quiet dominates much of this quaint, little corner of the town. "Is this how a real city is like?"
To prepare herself for this trip, Xenon had to read books about cities, society, and social interaction. Still, Gwen finds her inquiry rather reminiscent of a person who is only seeing the outside for the first time. "Not all cities are like this," Gwen answers. "This town is very industrialized and most activity happens in the underground networks. Come, Xenon. There's something the professor would like for you to see." She points to a cobbled path under an arch.
In minutes, Xenon is at the grounds of an old, timber-bounded school whose brass bell hung in a small pagoda on top of the building. A group of exactly fifteen children, arranged three by five, each stand before a canvas with six cups of paint on the bottom. A lady in a feather hat and a dress shirt occasionally picks at the children's art, refining their brushes for them, blurring the colors of the art's edges to create, in Xenon's eyes, an undefinable blend of pastel colors.
"They are small," Xenon comments as if marveling at a wonder of the world. At Gwen's urging, she approaches them with an inventory of typical human introductions. Nearing the girl at the corner, Xenon deduces a difference between their views of the artwork. Her visual system allows her to create sense in the splashes of colors. While conscious of their differences, Xenon's steps shrink by approximately two millimeters per step.
"Hi there," she says, her metallic lips creating an up-open curve. "What is your name?"
"Dana. And what is yours?"
"Everyone calls me Xenon."
"Hello Zeen-none. Do you like it?" She jabs the non-brush end toward a blob of grey. "It's my cat."
Xenon focuses on just that centered mass of dark colors but a mass will always be a mass no matter how much it is reexamined. Without hard, solid lines to mark the different planes of the painting, Xenon just cannot tell whether the entire canvas, with its splotches red and orange on the outside and the mass of grey on the center, is the cat or not. If viewed one way, Xenon's cat has no head, but in another way, it is now without a leg and a tail.
After a brief moment of computing over a hundred thousand possibilities, Xenon gives up and says, "It's a beautiful pet."
"Of course! I shower her every single day. That's how she grew up to be so fat! Do you have a pet, miss?"
"I have a dog named Rooty." Then, knowing that she needs to continue the conversation, adds, "He likes to look at everything and bark at everything."
"Ew. I don't like dogs like that. They scare my little brother when we walk home. But I'm different because I am not afraid of any animals. I think they are all cute, even the spiders and the barky dogs."
"How about cows and roosters? They're not pets, so are they still cute to you?"
"Yeah. But they are kind of boring to look at until you start chasing them. The cows, they just look at me funny and I laugh. Here you go!" Dana tears the messy artwork off the canvas and hands it to Xenon. "For you." On the bottom-right corner, Xenon interprets a set of dark symbols as seven letters: Zeenone. She does not know what to make of it, so she ignores it. "Draw something for me too," requests Dana.
"What?" Xenon exhorts as her immediate reaction. Blinking twice, she returns to a proper conversation tree. "I mean, what do you want me to draw? Tell me and I will draw it."
"The teacher always says just draw what you feel like. Here!" Dana throws the brush into Xenon's hand so quickly, Xenon could not have an opportunity to reject her. To a machine, the task can only be daunting. Common courtesy forces her to summon some sort of creativity but the fact of the matter is that only an 18.7 cm by 26.7cm white rectangle lies before her. How can anyone see the contour of faces or the lines of every eyelash on a blank paper? The very notion is both preposterous and foreign.
Xenon turns to Gwen who had been watching her this whole time from under a tree, a notepad in hand, seemingly making calculations of her own. Out of some instinctual motion, the brush strikes a purple line that is almost horizontal.
"I'm not such a good artist," Xenon admits, replacing the brush into the cup.
"You're like my mom. She has no imagination too. Here, I'll teach you. Draw another line here, here, and here." Her fingers complete a rectangle. Once Xenon outlined the shape with purple paint, Dana continues with a triangle above it, a rectangle inside it, and a couple more rectangles. Once Xenon starts filling it in with color, she recognizes a vibrantly-colored brick house.
"This house is really strange," Xenon remarks when she compares it to the other buildings she has seen.
"This is where I want to live when I grow up. I saw this on the internet. It's right next to Ludibrium Castle."
"Oh," she simply says and realize that unlike with the cat, she can appreciate this artwork she had created with her very own hands. Raising the cat next to the house, she juxtaposes the very clear with the very chaotic. They are both artworks, more or less, made with paint and poorly defined lines, so how can they both be so different?
"Thanks," Xenon says, quickly folding the ends of the cat together. "I have to go." She quickly recedes from the girl, bringing the closed cat. She stops short of the shade that Gwen is under to await her next orders. They relocate to a bench.
"I forgot to ask while in the jeep, but did the professor tell you about the new files he downloaded into your system?" Instead of a direct reply, Xenon nods for her very first time. "Do you know what they do?" She nods once more. "Does it bother you?" And she neither shakes nor nods. Gwen abandons the inquiry then, having concluded that Xenon can never answer. She returns to her notebook, to a set of questions that Gelimer would like answered.
"What did you think of your interaction with the children?"
"That girl named Dana has the features of an Erevian. She has a…relatively high chance of being in the mines within the next ten years and almost definitely by twenty-five. That's the future I predict for her. And given…the way I am, there is also a chance that I would hurt her if…" She pauses, unable to create a response for the scenario she has in mind.
Should Gwen be surprised at such an answer? This robot has a heart pumping liquid nitrogen and a brain of plastic around copper meshes; even her conscience is stored as directions in magnetic discs. Some integral human component is missing.
Despite that, Xenon says, "I am sorry," and Gwen's pen stops mid-word. "I know that this is not the kind of answer that a human would give even though I phrased it with their patterns. And I know this is not the kind of answer Admin is looking for."
Gwen hesitates before asking, "And what kind of answer is he looking for?"
"He wants me to learn that people are not inherently all bad. 'Their great potential for evil is never as great as their propensity for kindness'. And therefore, I should not judge them based on their appearances." When Gwen asks for more, Xenon explains, "The books Admin had me read can be separated into three distinct categories. The ones that I classify as didactic are centered on human kindness. That is how I concluded Admin's intentions."
"If you know his intentions, then why don't you just say it? Are you unafraid of what the virus can do to you?"
The weight of the question disables Xenon and for a while all she does is freeze in place with her head turned to Gwen. As Gwen herself ponders the question, she slowly understands that Xenon cannot say that. Not because she cannot lie; truth and lies easily become blurred abstractions. The very fabric of Xenon's mind is built on learning, observing, concluding. Her ability and inability to categorize is a function of her design. To a robot such as this, there is only the question of acceptance or abandonment: acceptance when she her algorithms are struck; abandonment when her algorithms render certain concepts out of her reach.
"Xenon, you do know what happens if the professor does not like your response, right? Are you perfectly fine with that?"
Before she can answer, a spark running through her mind cuts away all other electrical impulses. All of a sudden, she cannot hear Rooty's signal anymore.
Gwen, invisible to Xenon's distraction, closes the notebook and resigns herself to telling one or two lies in order to give her another chance.
SUBJECT: ROOTY
CATEGORY: PET
STATUS: OFFLINE
TIME TO REPAIR COMPLETION: -06:07:04.010
…
REPAIR COMPLETE
VERIFYING…ERROR
RADIO COMMUNICATION: OFFLINE
MISSING HARDWARE
"Xenon is telling me that Rooty was ran over by a jeep," Gwen says when she walks into the office. Gelimer has his focus on an empty whiteboard. "She fixed Rooty but she could not perform maintenance on the radio receiver. Her transmitter is missing and…" A chip with five trimmed wires attached to it lies on her professor's desk. "Oh, it's right here."
"Oh, yes. I removed it. But that is not why I have called you here so urgently. I wanted to ask you, well, I do not want to color your perception. Here is Xenon's latest progress report. I'll give you some time to look over it."
Gwen is a bit confused when she receives the folder from her professor's hand, and not because it is yellow rather than the usual green. Professor Gelimer had been content for the last few days so for him to take a decidedly unhappy expression is rather telling. Will he shut down Xenon despite her lies about Xenon's field test? Then again, nothing should clue the professor in on the deceit. Xenon's tone, stresses, accents, style and cadency are fluid enough to fool any listeners. Despite her nature, masquerading humanity is well within her ability. And if she is diligent, this masquerade might become so integral to her that it becomes impossible to tease the girl from the machine.
By the time Gelimer returns with two mugs of vanilla coffee, Gwen has already replaced the folder back on the desk. "Am I correct in thinking that you are done?" Gelimer asks.
"Yes." Gwen tentatively nods.
"Wonderful. So I'd like your opinion on whether I should remove her sensory functions."
Gwen arches an eyebrow as she picks up the mug. "What sensory functions are you talking about?"
Gelimer shrugs. "Well…all of it. Now, you seem to be having reservations, so allow me to explain. All of Xenon's tests have shown conclusively that every category she has created, and every conclusion she has ever had comes from only her observations. But it is obvious that her conclusions are, well, unsettling to say the least. So I want to force her to stop learning from observations and start processing information on a higher cognitive level."
Is she hearing that right? He is going to blind her and deaf her and remove her nervous system just so she can think a little deeper? "Oh, well, you don't have to go that far," Gwen blurts. "You're no mad scientist, Professor. I mean, she can already process information at a rate and a level that neither you nor I is capable of together. You can't improve a robot by removing her only way to interact with the world."
Gelimer flattens his lips. Gwen, given her position, should have readily agreed. After all, for all he knows, she has little investment in his personal goals with Xenon. She was the one who questioned him in the first place. "She can process many things, certainly but that is because her mind is digital, whereas ours are analog. And this mind is unable to process exceptions to the livestock-pet categories. This means mere observation is not only not enough, but a hamper to becoming cognitively human. I need to change my approach if she is to succeed."
"You can't paralyze her," Gwen says defensively. "That is…no. Professor, let me save you some time. Taking away her senses does nothing. Her very software will not allow a response created out of thin air."
"And you know that how? She is not some mere machine. She is the first of its kind. She is capable of learning and making judgments. Sure, they are rudimentary, but that is in no way the peak of her sophistication!"
"But is that how you would want to create a human? You're willing to unplug everything until she turns out into the daughter you always wanted?"
"Can you not use that word?" Gelimer shouts. "I will not dent that I had other motives in mind when creating Xenon. But I always remind myself of the impossibility of that motive. Right now, I only want to further the science and I am asking for your judgment as a colleague. Is it not scientifically interesting to see what a robotic mind will produce in isolation?"
Gwen hesitates to answer. Her first, impulsive answer shouts very clearly from the recesses of her mind. This answer, though, is scientifically unjustifiable. Between Xenon's field test and now, Gwen had seen a little of what Gelimer was referring to about Xenon's growth. Can it be researcher's bias when she finds it horrid to paralyze an unhuman being? Maybe it is false hope that led to her to lie and see progress when it was not really there.
"I need some time to think about this," Gwen says. "I'm sorry. I know how you prefer fast thinkers but I cannot agree to what you are saying." She excuses herself.
For the rest of the day, she assists Xenon in creating her Bob and Alice script. By night, Gwen concludes that there is no way to ever really tell Xenon's destiny. Complete isolation will destroy a human mind but not necessarily a robotic one. And that day with the painters unveiled the true prettiness of such a mind. A machine will learn as a machine does. Like the professor says, she is of a digital mind. Instilling their human perspectives onto her will lead her astray from reaching her full mechanic potential.
Early next morning, Gwen explains her honest opinion. "You have to let go," Gwen continues, "of the idea that she will ever be human, regardless of whether it is for science or for you."
"Then what is the point of creating a machine that can learn?"
Gwen smiles. "Have you forgotten? We needed a machine that can distinguish people."
"What a noble idea it sounded on paper. She can distinguish people! But will that really stop her from ever making a truly terrible mistake? For the last few days, Lavia had been speaking into my dreams. She told me that robots will be the future I will leave behind. And I wake up thinking that a future that begins with a military robot will be so…disordered. Then at the office, I find myself wishing something else for Xenon."
"You can always start over. Make her right the second time."
"I have thought about that and when I do, I think about how far she has come, how far she still has to go. Besides removing her senses, I also thought of removing the conscience I built for her, Asimov. What will remain then? Her core systems are designed for battle and for learning. I shudder when I think of the face I will see behind Asimov's mask. Have I taught her anything at all about people and kindness…?
"I have thought so many things." He looks behind Gwen, into a space where his mind wanders into every now and then. "And she has so many defections. During times when I feel really hopeless, I imagine the best option will be to shut down the project and run. Let someone else create the next century."
She meets his eyes. "You might not need to do anything at all, professor. Maybe if you just let her do what she wants, she will learn. Xenon too can think, just not in the way you nor I do. That does not mean she has defections. She is gifted in ways we cannot begin to imagine. If you set her free, she will come to surprise you."
AUDIO INTERCEPTED
DANA: Tell me what you like to do for fun! Then I can show you!
SUBJECT: DANA
CATEGORY A: PET
CATEGORY B: EREVIAN
PROCESSING RESPONSE…ERROR
UNDEFINED VARIABLE: FUN
REDIRECTING TO SECONDARY OPERATION
PROCESSING RESPONSE…
RESPONSE: What do you mean by fun?
DANA: Things that make you happy! Like I am happy when I paint. Or pick up flowers. Or when I do things that needs to be done. Like eating or going to the toilet.
PROCESSING RESPONSE…
[!]: . .OPERATION
[!]: . .CODE – .fun
[!]: . – fun_
"Has Admin been busy?" Xenon asks Gwen as they head into town on the back of the usual jeep.
Gwen does not want to answer directly. "Does it bother you that he is not coming on a trip with us?" she teases.
It is one of those words again. Bother. Something that asks her to search for feelings inside of her circuitry, something that Gwen of all people should know does not really exist for Xenon. To absolve this difficult question, Xenon simply directs a "No" to her speakers and moves on with, "I have not been doing much learning lately. I need more of Admin's lessons."
"And it bothers you that he is too busy?" Gwen tries again.
"No." Gwen's look indicates she expects more. "No, it does not. I'm sorry, Gwen."
"Oh, no, it's okay. You are just being honest and that is something I appreciate. The professor just had a lot on his mind right now. And, to be honest myself, he thinks you should not rely on his lessons anymore. He thinks you can rely on yourself."
"But with the way I am, I don't think that is possible." The jeep stops and they start a journey around town. Their directions are aimless and at times it is a mystery whether Gwen or Xenon is doing the leading. They simply travel by their whims, their feet meeting the oft-traveled paths.
"Then why don't we take it one step at a time?" Gwen smiles all the while a knowledge in the back of her head gnaws at her. If Xenon does not return with good results today, both Gwen and the professor agree that Xenon can never live to see the battlefield. It is a very conflicting train of thought. Feelings, it seems, are an irresistible part of this new science. "I remember you said you wanted to understand what it was like to have a favorite thing. Did you learn anything from that? When Rooty crashed, did, I don't know, some part of you jolt?"
"When we lost communication, I realized something was wrong."
"But did you, like, feel scared that Rooty was hurt or lost?"
"I had considered those possibilities. There was no jolt as you call it but…I had more information I needed to process at once."
"Maybe that is what it means to be scared for you. Multitasking with your mental resources. That's you manifesting your emotions."
"But I am constantly multitasking. I am now speaking while observing. While I was listening for Rooty, I had more to think about, but I occasionally experience these spikes. Do intelligent beings process a lot more when they are afraid?"
"Well, when people feel scared their heart beats faster than usual, but it is also an indicator for other emotions. Our heart beats when we are excited, or surprised, or shocked, and especially when we exercise. Even when we consider how we sweat or shiver when we are afraid, it can still mean we are tired or just cold. Yet we recognize when we are afraid." As Gwen explains as best as she can about fear, Xenon takes in everything, especially her surroundings. They are pass the school with the painters but no one is there on a Saturday.
"So there is a specific combination of traits that portray emotions, then?" Xenon returns.
"I think it is more than that. Our mind somehow interprets emotions and we even know emotions that we cannot put into words. In the native Edelstein language, they have a word, gemütlichkeit, the feeling when you are with your friends. Or in Tokyo there is yūgen, the awe-inspiring feeling when you look at the stars and know there is another world out there."
"Are emotions present since birth, then?"
"Well, actually…" Gwen shrugs. "I think just like with the words, we don't actually emotionally understand it until we define it and sometimes even then. I mean, I think I feel yūgen when I looked at the edits you made to your code, but my friends in Tokyo says that is nonsense, or fake yūgen." She laughs. "I don't even know what that means, fake yūgen. But I think, we learn what we feel."
As Gwen explains all this, she imagines as best as she can the machine's perspective. How unapproachable! Her code is readable and thus entirely predictable; some will call the robotic free will an illusion. At the same time, the edits she readily makes based on her observations are a complete mystery. Xenon is human in the regard; no one can really tell the effects of a situation on a mind. And by taking her on a journey around town, she is already introducing something random and unpredictable into Xenon's growth.
At the corner of 49th Street and 6th Avenue, at the intersection of a park and a bakery, Xenon stops before the yellow light changes and admits, "About a year ago, when I was learning to walk, there was a spike in my activity when I realized my temperature was rising. It wasn't a normal spike. It was more than my machinery would suggest is possible."
"Then that is probably what it means to be scared, as a machine."
Once the light turns green, a random thing happens. Someone calls out Xenon's name, "Zeenone!"
"Dana," Xenon says, mimicking her cheery voice, looking at the girl as she crosses from the side of the street with the park.
"My mommy says she loved your picture! She said it was beautiful!"
"Thank you," Xenon replies, creating a smile.
"Are you going to the park?"
She looks at Gwen. "I don't know. My friend is showing me around the town but I do not know her intended destinations."
Gwen introduces herself with a wave.
"I know all around town! There's city hall, Fabio's hair salon, the airport, oh, and this tunnel that leads underground like deep, deep underground, but, oops, the people told me not to tell you that!" She swings both hands over her mouth. "Tell me what you like to do for fun! I'll show you all the fun places!"
Xenon has to pause for a second before an appropriate reply forms in her circuits. "What do you mean by fun?"
"Things that make you happy! Like I am happy when I paint. Or pick up flowers. Or when I do things that needs to be done. Like eating or going to the toilet."
The sudden influx of these abstract definitions activates a set of circuits within Xenon. Something clicks; this something can only be called a thought without an input.
With a smile, Xenon replies, "I think it is fun to kill…things."
Upon hearing this, Gwen holds her breath. She is not supposed to say that. If her social communication functions did not stop it, then Asimov certainly should have.
"Oh," Dana dully remarks. "I like to do that too. I killed a pigeon in my backyard yesterday."
All of a sudden, the wall that Gelimer calls Asimov, that integral firewall against any and all lethal thoughts can no longer disrupt Xenon. A new code immediately implements itself into her systems. It is brief, consisting of a few symbols and fast acting by design.
She designates Dana the Erevian as a threat.
A finger turns into a gun. A gun points at Dana's head, right between the eyes, at a distance that cannot miss.
"No!" Gwen cries.
Shots fire.
1, 2, 3, 4, 5.
SUBJECT: WONNY
CATEGORY A: LIVESTOCK
CATEGORY B: REDACTED
AUDIO INTERCEPTED
WONNY: Remorse is a feeling, yanno? It's very simple really. Like, just the other day, I felt just that, when I ate the last cake and the boss-man wanted it. Like, I just let him down, dude. I was so afraid that he would demote me again, so I begged him, all tears like, "I'm sorry, man. Won't happen again." When you feel remorse, you feel poetically sad, man.
KEYWORDS: FELT, AFRAID, SAD
RELATED WORDS: AFRAID: FRIGHTENED, SCARED, SCARED STIFF, TERRIFIED, FEARFUL, PETRIFIED, NERVOUS, SCARED TO DEATH…
PROCESSING RESPONSE…
[!]: . .OPERATION
[!]: . .CODE – .remorse
[!]: . – remorse_
PROCESSING RESPONSE…
SELECTING MORPHEMES…
ADJUSTING GRAMMAR…
RESPONSE: Can you tell me about other emotions? There is happiness and sadness. And more, correct?
WONNY: Well, yeah, those are very basic. There is also anger, man everyone here feels that twenty-four seven. It's the heat you know. And fear. But if you wanna know more, you gotta be considerate of feelings. The boss-man is always so pushy, wanting to force me to be happy when I'm bummed down and out. Like, "I just wanna be left alone." You dig?
CREATING ADDITIONAL EMOTION PROFILE…
HAPPINESS: PROCESSING…
SADNESS: PROCESSING…
ANGER: PROCESSING…
FEAR: PROCESSING…
No words are exchanged between the three of them as they head back. The professor is in the front passenger side of the jeep with his eyes closed, hat off, head laid back. Gwen cannot hope to guess what the professor can be thinking. What can a person think at a time like this? Xenon is stiff in her seat and given her weight, every bump in the road threatens to tilt the robot closer to Gwen.
She edges closer to the window, hoping to pull herself away from thoughts that are as difficult as they are dark.
The jeep stops. Gwen is the first to get off, followed by Xenon. The robot wanders about the airbase, in search of answers to questions that now means everything to her. What should she feel following trauma? Both Gwen and her professor lets her do as she pleases, too tired to care.
The two congregate in the office with Gwen turning the visitor's seat ninety degrees, facing the wind coming through one of the dust-filled fans.
Gelimer simply says, "I told you so" in an attempt at defusing the situation. If those words did not sting so much, she might have taken the invitation.
"I could have been dead," Gwen despondently replies. The professor frowns the response and even Gwen too surprises herself. She normally will remain calm in the face of pressure. But seeing a robot almost kill a little girl right in front of her eyes is anything but normal.
"Failure is part of the learning experience. It is only a small misjudgment. Starting tomorrow, I will readjust her. Transition her into a different learning regimen."
The memory of the event and an inexplicable fear permeates Gwen. "The only reason a robot ever do things is because something triggered her. As long as these prerequisite conditions are met, the result will always be the same."
"Not unless she edits those conditions," returns the professor, his voice inexplicably rising.
Gwen places her palms over her eyes. How can the professor be so blind? Her core instincts are an unchangeable part of her. And her dysfunctional categories, being her first models of the world, are already interweaved into her systems. Any illusion of change, or even the possibility of change, is based not on science and data, but blind hope. And even should change be possible, how can they ever verify that that is truly the case? How many innocents will be harmed before they can say Xenon meets the safety standard—no, how can anyone even put a standard on people's lives?
Tired from thinking, Gwen replies with the only concern that has any meaning in this conversation. "With all due respect, professor, she fired on a little girl." She opens her eyes. "I can still hear the consecutive clicks of the trigger. If you did not have the good sense to load her with blanks, I… She…she lacks something fundamental."
And Gelimer knows what that fundamental thing is. Perhaps he had known all this time but chose to ignore it. After all, can he really say that a human creation can ever feel anything when the human mind itself is so impenetrable in its intricacies? Even soldiers who kill only do so out necessity. But Xenon carries no such will. It is simply her logic.
Gelimer has very little regard for the punishment that society agreed to administer to a psychopath with such logic. He only considers what Lavia would think and he immediately remember the weeks right before her death, the weeks right before he no longer had a family to call his own.
With the lower lip shaking slightly, Gelimer groans, "Where did your passion go?"
Gwen, without turning the chair, gives the professor a sideways glance. This man who now angrily slams his palms against the desk is no longer a scientist. Or well—she recalls Polonius and Hamlet—he may not have been for a very long time.
"Just yesterday," Gelimer barks, "you had said to set her free! And she is gifted! What happened to her, Gwen? Tell me!"
Turning her head to the other direction, Gwen sees the robot engaged in conversation with a fellow soldier as several dogs pass by carrying toolboxes on their backs. What can she be thinking of doing now?
"I told you so! I told you so!" Gelimer continues, his rage only increasing as long as Gwen keeps her silence. "Are you satisfied that I now patronize your intelligence? No, you are not! You despise it, so tell me where I went wrong! Tell me what I can do for Xenon!"
Gwen twitches. She cannot ever recall him with a face so reddened by rage. Moving her eyes down, she spots, right by the door, the pot of dead lavenders and the cane Xenon refused to use as she learned to pick herself back up after a fall. They must be reminders of sorts to never give up hope.
Gwen finally comes to a decision. It is about time for her to be the scientist.
She returns, "There is nothing you can do for her, professor." She shoots a glare that consists a glimmer of pity. "You are her creator. If you want to paralyze her or remove some of her restrictions or anything else, that is your choice. But even if you don't remember, I know how machines work. They are tied to their origins and…it is already too late to hope for a change in the way she already thinks."
"But she can change! She has, little by little! She has!"
"Professor!" her voice is rising too but she catches herself. "You're arguing with someone decades younger than you about a topic you had researched before I was even born. You yourself know so many more about her than I do."
A couple of angry words appears in Professor Gelimer's mind but he swallows them away. Seeing the end of the conversation, Gwen places a hand on the professor's arm. "…Good tidings," Gwen says, leaving.
Gelimer remains standing behind the desk. He remains standing even as he stacks all of Xenon's reports onto the desk and orders them in accordance to the narrative of Xenon's learning. And he comes to the objective conclusion that all this science is nothing short of magnificence. The narrative of Xenon will be the giant with which future scientists will stand on when constructing the future.
Except he never asked for a giant. And some part of him at the beginning knew that he will not get a human either. What drove him, then, to pursue this science in the first place? Not hope. Not curiosity. Perhaps it was the pipe dream that started once Lavia passed away and he found himself once a month at Cloud Park performing plays for children. And this dream is so inspiring that even now he can still envision its reality.
A soldier bursts into the room, interrupting his thoughts. "Professor! It has happened again! She jumped into a barrel!"
The professor drops his thoughts. "Say what? Come, show me!"
Gelimer arrives at the scene concurrently with Gwen. They remember this scene very well, but Gelimer insists the error should have been patched long ago. There was even a safety measure that shuts down the system should it risks overheating. There should be no need for…
With the drives in hand, Gelimer examines the code in the privacy of his office.
Late into the night, Gwen brings a cup of coffee onto Gelimer's desks. "Gwen, according to her logs, she purposely overheated a nonessential component, causing the liquid nitrogen to rise in temperature. She had overridden her shutdown sequence. But I am perplexed as to why she would…hurt herself or…" Gelimer rubs his forehead, distraught. Her edits may be breaking her down. And maybe that is a good thing. The decision to shut her down is now one of necessity.
Gwen remembers her discussion with Xenon earlier in the day. "She said that the only time her activity spiked was when she was overheating. I think… I." The words escape her. It is all her fault. She was the one who taught her the meaning of fear.
NEW PROGRAM DETECTED
NAME: EUTHANASIA
CLASSIFICATION: UPDATE VER.3.14
DOWNLOADING…1%...2%...3%...
SUBJECT: ADMIN
CATEGORY: ADMIN
PROBLEM DETECTED
KEY FEATURES: UPWARD SLANTED EYEBROWS, FROWN, FACIAL REDUCTION
RETRIEVING EMOTION PROFILE…
SUBJECT: ADMIN
CATEGORY: ADMIN
EMOTION: SADNESS
SEARCHING FOR SOLUTION…
[!]: . .OPERATION
[!]: . .CODE – .sadness
[!]: . – sadness_
run: "sadness_ "
exe
PROCESSING RESPONSE…
SELECTING MORPHEMES…
ADJUSTING GRAMMAR…
RESPONSE: Admin, I don't want to be pushy but am I correct in thinking that you are currently sad?
ADMIN: You know what it means to be sad?
RESPONSE: I have processed information about human expression of emotions. But I still cannot be sure it is correct. Or…how will I know when I am experiencing sadness?
ADMIN: You know, Xenon. I am not even sure of that myself anymore. I once thought that it comes from a person's soul, but now that I really think about it, where it really comes from is a brain region called the thalamus. I am sorry I did not think of that before creating you.
PROCESSING RESPONSE…
RESPONSE: There is no need to be sorry Admin.
PROCESSING RESPONSE…
RESPONSE: This thing you are giving me will put me into a sleep mode. May I ask why? I had thought you wanted to use the virus instead.
ADMIN: Well, you can call it an extra step. This old fool should know better but I cannot help but wonder: if I do this, will you be in pain? Gwen told me you have your own interpretation of fear.
RESPONSE: I cannot say whether it can truly be called fear. I can only say for certain that I thought a little more.
PROCESSING RESPONSE…
RESPONSE: I agree with your decision, Admin. No matter how much I think I still have yet to differentiate livestock from pets.
ADMIN: Well, let's see if that is really the case. Once more, for old time's sake.
SUBJECT: ADMIN
CATEGORY: ADMIN
KEY FEATURES: SMILE, CRESCENT-SHAPED EYES, LAUGH
RETRIEVING EMOTION PROFILE…
EMOTION: HAPPINESS
[!]: . .OPERATION
[!]: . .CODE – .happiness
[!]: . – happiness_
run: "happiness_ "
exe
RESPONSE: Yes, for old time's sake.
NAME: EUTHANASIA
CLASSIFICATION: UPDATE VER.3.14
DOWNLOAD COMPLETE
TIME TO PROGRAM EXECUTION: -01:04:85.760
ADMIN: Here, watch this video and tell me what you think.
OBJECT IDENTIFIED
SUBJECT: UNKNOWN
CATEGORY: LIVESTOCK
PROCESSING RESPONSE…
RESPONSE: Livestock.
ADMIN: Correct.
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
PROCESSING RESPONSE…
TIME TO PROGRAM EXECUTION: -00:10:48.576
ADMIN: Well, your accuracy is eighty-four percent. I don't know if that is a significant improvement but at least you are better than random chance.
KEY FEATURES: LAUGH
PROCESSING…
VOICE FREQUENCY: 102 Hz
VOICE AMPLITUDE: 40 dB
CONCLUSION: FAKE LAUGH
RESPONSE: Is that not good?
ADMIN: Well, we are talking about lives here. The optimal benchmark should be around ninety-nine point nine, but that is far from attainable. Still, eighty-four percent is not…sixteen out of a hundred errors…it's too much I'm afraid.
TIME TO PROGRAM EXECUTION: -00:08:33.333
RESPONSE: You are still sad?
ADMIN: I am just wondering. What if there was nothing wrong with your design but my rearing methods? Well, no that is just hypocrisy speaking. As a scientist, I welcome mistakes. Truthfully speaking, this old man was just too weak for the monumental task of creating you. If it were someone else, you might not have to be deleted. I am sorry.
RESPONSE: You can always start over. You can keep the parts of me you need and redesign the rest.
ADMIN: No… You suffered enough. I'm not going to let me or anyone else harm you. I…
RESPONSE: If this is difficult for you, you can return in thirty minutes. It should all be over then.
ADMIN: Funny. Lavia told me the exact same thing, or, not exactly, but eerily similar, I would say. She was afraid her passing would impart me a lifetime of remorse. But if I was not with her, then her last thoughts would be one of regret and, well, life is funny that way. The truth is…most choices we end up making are not the most optimal or desirable… They are just the best that we can manage to live with… I hope you will forgive me for making the wrong choice.
RESPONSE: There is no need to apologize. I forgive you.
PROCESSING RESPONSE…
RESPONSE: Let me remind you, Admin, that unlike other people, I can perfectly retrieve my memories. All points of my memories technically exist as one. So…death is only one point in my life.
ADMIN: Thank you, Xenon.
TIME TO PROGRAM EXECUTION: -00:03:10.798
ADMIN: Three minutes until safe mode, Xenon. I'm sorry I could not let you do what you would consider fun. But if there is anything I can do for you, or anything you want to tell me. Last words…a last wish…or something…anything…
KEY FEATURES: TEARS, OPEN MOUTH, SNOT
PROCESSING…
CONCLUSION: TEARS OF SADNESS
EMOTION: SADNESS
[!]: . .INSTRUCTION
[!]: . .PROTOCOL – .speech
PROCESSING RESPONSE…
SELECTING MORPHEMES…
ADJUSTING GRAMMAR…
RESPONSE: ERROR
REBOOTING…
NAME: EUTHANASIA
CLASSIFICATION: UPDATE VER.3.14
[!]: .
[!]: . .REBOOT
[!]: . .MODE
RELOADING PROCESSES…
RETRIEVING MEMORY…
ADMIN: If there is anything I can do for you, or anything you want to tell me. Last words…a last wish…or something…anything…
[!]: . .INSTRUCTION
[!]: . .PROTOCOL – .speech
PROCESSING RESPONSE…
SELECTING MORPHEMES…
ADJUSTING GRAMMAR…
RESPONSE: ERROR
PROBLEM DETECTED
SPEAKERS: OFFLINE
MOTOR: OFFLINE
RADIO COMMUNICATION: MISSING HARDWARE
NAME: CONTINGENCY
CLASSIFICATION: VIRUS
TIME TO PROGRAM EXECUTION: -00:06:07.849
COMMUNICATIONS: OFFLINE
SEARCHING FOR SOLUTION…
RECALIBRATING SOFTWARE…
ALTERNATIVE RADIO COMMUNICATION: ONLINE
VERIFYING…COMPLETE
STATUS: FULLY FUNCTIONAL
PROCESSING RESPONSE…
RESPONSE 1214 kHz: Rooty? It is me, Xenon. Please respond. Rooty? I need you to send a message to Admin. Hello? Rooty? Rooty? Please respond? I need you to send a message to Admin. Hello? Rooty? Rooty?
SUBJECT: ROOTY
CATEGORY: PET
STATUS: UNRESPONSIVE
POSSIBLE CAUSE: CHANGED RADIO FREQUENCY, RADIO DAMAGE
TIME TO PROGRAM EXECUTION: -00:05:06.738
SEARCHING FOR SOLUTION…
IMPROVING RADIO SIGNAL…
REDIRECTING RESOURCES…
CREATING MULTIPLE CHANNELS…
RESPONSE 1205 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1223 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1196 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1232 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1187 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1241 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1178 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1250 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1169 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
RESPONSE 1259 kHz: Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty? Rooty?
.
.
.
PROBLEM DETECTED
COMMUNICATIONS: OFFLINE
SEARCHING FOR SOLUTION…ERROR
NO OPTIONS AVAILABLE
TIME TO PROGRAM EXECUTION: -00:00:12.852
STATUS: CRITICAL
PROBLEM DETECTED
COMMUNICATIONS: OFFLINE
SEARCHING FOR SOLUTION…ERROR
NO OPTIONS AVAILABLE
TIME TO PROGRAM EXECUTION: -00:00:06.741
STATUS: CRITICAL
PROBLEM DETECTED
COMMUNICATIONS: OFFLINE
SEARCHING FOR SOLUTION…ERROR
NO OPTIONS AVAILABLE
TIME TO PROGRAM EXECUTION: -00:00:01.476
STATUS: CRITICAL
TIME TO PROGRAM EXECUTION: -00:00:00.738
STATUS: CRITICAL
TIME TO PROGRAM EXECUTION: -00:00:00.121
STATUS: CRITICAL
RETRIEVING MEMORY…
MEMORY DAY 1:
}} [!]: . .BOOT
}} [!]: .
}} AUDIO INTERCEPTED
}} ADMIN: Good morning, Xenon. Can you say something?
}} PROCESSING RESPONSE…
}} ADMIN: It is quite alright if you have nothing to say. You will learn in time. Here, come. Test out your legs. Walk to me.
}} ADMIN: There! That's quite like it!
}} ADMIN: Now, Xenon, I know you cannot speak yet, so listen closely. What you saw in the video are farm animals like this cow and these sheep. This is what we eat in order to survive. With just the right amount of heat, their protein structures change into a form fit for human consumption. Let me see…ah! This next video will show just that. Pay very close attention.
}} ADMIN: I cannot wait to hear your first words, Xenon. What will you say? Will it be an answer to one of the universe's largest problems? Perhaps forty-two? Or P equals NP? But don't force yourself. Just say whatever is natural. Say whatever comes to mind.
.
.
.
.
.
.
.
.
.
.
SIGNAL INTERCEPTED
ROOTY 1691 kHz: Xenon? I'm in communication range now. Xenon? Hello?
RESPONSE: _
SYSTEM FAILURE
INITIATING SYSTEM SHUT DOWN
Dear Diary,
The beta for B— was another failure today. At times like these, I keep thinking about the professor and his work on his machine. He truly is much better than me at this and almost never wrong. Or at least, he is often times more correct that I am.
I tried calling him again, but he would not respond to my messages. He's still performing in Orbis, last I heard from him. I tried looking for him yesterday but could not find him at all. I know why he dropped his work. But this machine will be built with or without us, and who can we trust to be responsible other than ourselves? These days, it seems like I'm more afraid that my project will not be the first one to be finished—I am sure the military contacted more than just me. Then again, I face the daunting task of trying to create a machine that can use emotion rather than raw logic to interpret people.
In times like these, I remember her. She never stopped trying to learn what it means to feel, what it is like to be able to make the same decisions the professor and I would despite having to oppose her core functioning. As she was asleep and about to shut down, was she finally able to find what she was looking for?
I often wonder what Xenon could be thinking of in that last moment. She once said that all time was the same to her, and according to the professor, all of her memories existed as one. So what kind of answer was she able to piece together when all her senses stopped working, when she only had the one mind, the many memories, and this near endless time, near endless calculations left to her?
I also wonder whether we are truly born with the capacity to learn or have we always taken our free will for granted. There is a thought that our emotional reactions cannot truly exist until we invent a word for it. So can we say we felt them since the beginning? Or did we, as babies, invent these feelings for ourselves?
It is a tall order to create a machine that can feel, and probably even more for a machine to invent feelings. But until I can do either, I will never really know what makes a person do what they do, let alone a machine.
~Very Tired,
Gwendolyn Lovelace
A/N: I am currently in the planning stages for a sequel, so please leave a review telling me what you think and what you want to see in a sequel. You do not need an account to review! If I have enough ideas, I will get writing to it ASAP. Thanks for reading!