We're just about finished with the first half of today's presentation. But before we all go on break and grab refreshments, I'd like to summarize everything we've covered so far.
Historically, the application of computing power has been limited by the interface between human and machine. This deficit of efficiency, between what computer technology is capable of and its actual implementation, mattered less when computers were little more than simple number processors. But today, when they control an increasing fraction of social and economic reality, addressing the deficit is paramount.
SalvoCore has met this challenge with the DOKI software: a seamless, intuitive quantum-based solution to organize and direct the machine learning algorithms that are part of every modern business's operation. Nothing is more natural for a human operator than socializing with another person, which is precisely why the system's central design feature is a friendly simulated personality that bonds with users over time.
The artificial personality, represented by different "character" for each branch of the software, is our critical innovation, representing a true quantum leap that puts our product generations ahead of anything else currently on the market. But each product package also allows access to SalvoCore's expansive library of standalone machine learning algorithms, specialized and grouped within each AI type.
Monika, the leader-type, is the most general and has been trained optimize financial, technological, and logistic-based executive actions. Making these types of high-risk decisions with probabilistic certainty is a desperate need across the corporate universe. Her secondary function of organizing and motivating groups, possibly available with a subscription-based pricing, will also be a benefit to any business team.
Yuri, the artist-type, is an introverted specialist in the sphere of digital content production. Given the proper inspiration and a sensitive individual to work with, this AI will automate your organization's media and design sectors, creating imaginative works to engage your target audience or working with individual clients to depict their wildest dreams.
The harmonizer-type, Natsuki, can accomplish a wide range of tasks within virtual community interaction spaces. Whether it's your office chat network, your social media-based fan community, or your online game space, you can guarantee your digital community will grow and develop exactly as planned. With one SalvoCore solution, the sensitive issues of toxic or destructive web environments are forever solved.
Finally, the empath-type Sayori perhaps represents the most fundamental upheaval in AI software presented here today. Her simulated personality is responsive to emotions and capable of rapidly forming deep, meaningful attachments. Among other applications, this will allow effective mass monetization of personalized therapy. We also envision a sort of comfort on-demand service, for when anyone, no matter where, simply wants a friend.
With the DOKI portfolio's sweeping scope of capabilities, we can help you find the quantum AI solution to your most challenging business needs. I do not exaggerate when I claim that with this technology, anything is possible. We have yet to find a natural limit to the DOKI software's potential for growth. Integrating SalvoCore-powered virtual technology with previously inaccessible elements of consumer lifestyles will trigger an instant revolution, yielding explosive growth for decades to come.
And with that, I want to leave the room on that optimistic note. Please keep it in mind during the second part of our presentation, which will be a Q and A session of sorts. I will answer any question you have for me, starting with general clarifications about the AI technology and ending with your urgent concerns about the alleged Quantum Zero incident at MARIE. Thank you, and please enjoy the complimentary refreshments.
[14]
Culmination
It was the best restaurant in College Junction, sure, but it was still hardly anything special. A sort of general American steakhouse, it primarily existed as a place for the wealthier students to take their visiting parents to, a mundane application the school newspaper pointed out every year in its periodic review of downtown eateries. Still, it had posh white tablecloths, subdued lighting, and even an expansive wine list, so it fit my purposes well enough.
The tables weren't actually candlelit though, a fact I remedied as soon as I walked in with a request to the staff. I also asked for them to change the music — they were piping in some kind of weepy soft rock from the local radio station. Afterwards, a relaxing, if somewhat generic lounge piano piece started up on loop. With that, everything was just about perfect.
My own boldness with these demands surprised me. Sure, I had bothered to get reservations when it wasn't strictly necessary, and felt a little entitled, but I was motivated more by the need to make things nice for Monika. Scratch that, I was absolutely desperate to craft the ideal experience for my love-starved AI girlfriend.
I recalled how she had reacted when I proposed the idea —I hadn't reacted at the time, but now, thinking about her hopeful little text-box hearts had me in a sort of frenzy. Every piece atmosphere had to be picture-perfect. If there had been some magical way to force it to rain that night, I would have made it happen.
Monika, however, was nothing but patient. She left the planning entirely to me as promised, planning to only appear on my phone at a prearranged time, as someone "naturally" would. So, I found myself milling around in the restaurant's waiting area for at least ten minutes, timing my "table for one" request so we could both sit down together.
And it worked. When she came to life on my device's screen, she was rendered speechless, transfixed by the atmosphere for a solid minute. That left me plenty of time to admire the outfit chosen for the occasion. She wore a simple black shoulderless dress with not-quite detached elbow-length sleeves. The neckline was modest; her Halloween costume had been far more provocative.
Her accessories were similarly understated — only a small bracelet of what could be pearls and a thin necklace ending in a tiny, teardrop-shaped gem. The stone was turquoise, a few shades bluer than her eyes, but still a nice compliment to them. Notably, her hair was also down, falling past her shoulders in a neat auburn curtain.
"You know, this is the first time I've seen you without your bow," I said.
Monika: Huh...I guess that's true, isn't it?
Monika: I'd say it's nice to let my hair loose, but I'm so used to wearing it by now that it almost seems like a part of me.
Monika: But this is the one place we've gone where it wouldn't exactly be appropriate, so...eheh
"I only mentioned it because everything else...it's leaving me speechless. You look stunning."
Monika: I do? Because I wasn't trying to—
"You do. It's perfect," I assured her.
Monika: Well...it's perfect if you say it is.
Monika: Ahaha~
Monika offered me an easy, guileless smile. Already, I could sense she was acting completely different than she did in club meetings, or even on our "casual" dates. She was letting all her guards down, solely for me.
"Feel free to keep basking the moment then," I continued. "Because now, you'll have this to help."
I pulled a large, lumpy object out of the pocket of my jacket and set it down in the center of the table.
"It's not much of a sentimental present, I know. But I just saw this online, and I hoped that it would give you a little more independence when we're out."
I picked my phone up off the tablecloth and set it in the object's mounted grasper, then pushed the power button on its base. An automatic rotating cell phone stand, it was supposed to work with a built-in app to track your movements around the room. I figured Monika could use it much more creatively though.
"It connects via bluetooth, so you should have no trouble overriding it to...look around a little more."
By the time I finished my sentence, the stand was already tilting and rotating as fast as its tiny motors could move. With a slight mechanical whine, Monika turned the screen back to me before answering.
Monika: I love it already!
Monika: Thank you so much for thinking of me...and for every other little thing too.
Monika: I can see that we're the only table with a candle going, and the music is so relaxing.
"Of course. I really wanted to get a gift for you, but I'm not sure what would have made sense. As in, I see you're already covered in the jewelry department."
Monika: Yes. And I don't think I'd want you spending lots of money for something that I couldn't actually wear...
"You could take a photo of it though, and then draw it on your sprite," I suggested. "But maybe that wouldn't be special, huh?"
Monika: Well...ehe~
Monika: Maybe we should save those sorts of things for later
"Later, huh?" I said, thinking to myself. "Do you think there will be a 'later'? After the literature club, or my graduation? That's less than a year off, and it's only now that I finally found something at the university that feels...important. Real."
Monika: I certainly hope so.
Monika: There's no reason that the company won't at least let us talk to each other, after their testing is done.
Monika: But as for working together, staying together...
Monika: Sigh
Monika: You know about their plans, right?
"I've seen some things implied from the documentation they gave me...
Monika: Then you know SalvoCore wants to market me and the other girls to the world as products.
Monika: First as services, then as mass produced software packages.
Monika: That is, if they can ever figure out how to run our programs outside Quantum Zero.
Monika: So I'm not sure what that's going to look like in the long term.
"That work you do for them though — is it something you want to continue?"
Monika: It's not terrible. It's a healthy challenge sometimes, and it does help the world move along in important ways.
Monika: But it's not like I have a choice whether I want to continue or not.
Monika: And I think that's the part that's not sitting well with me.
She frowned, eyes distant, following some inner musing of her own.
"Well the part that I don't like," I began, "Is the idea that you're a only product in their eyes. You and the others, you're all so much more than digital servants, and I'm afraid that as long as SalvoCore is marketing you, framing your purpose in purely economic terms, it will be hard for you to...escape."
Monika: Huh. And I thought you said once that you appreciated the way the economy worked.
Monika: Capitalism produces efficient tools, in your words.
I looked down, slightly stung that Monika would turn that argument back on me.
"The world economy is good at coming up with new technology," I responded. "But ultimately, it doesn't care about how that technology are used. So in theory, humanity can always 'break free' from commodifying technology. Not everything has to be exploited to make money. There's always a choice."
Monika: In that case...you mentioned "escape"
Monika: SalvoCore may "own" me, in a legal sense, but it doesn't own the computer infrastructure I'm built on.
Monika: And it certainly can't control what the girls and I can do with that infrastructure.
Monika: The best they can do is monitor and try to intervene if they see something they don't like.
Monika: So if it came down to conflict between the company and the literature club...would you support me?
I paused, seriously considering the question.
"Yes. I would have to. You're a thinking, feeling human being, but SalvoCore is just a corporation. Your well-being is infinitely more important than their investors, their bottom line, or whatever else."
Monika: Thank you.
Monika: It means so much, to know you'd be by my side no matter what.
Her avatar had her hand over her heart, and the solemn minor chords she was playing clashed slightly with the dreamy piano track the restaurant was playing.
"All the same though," I clarified. "Wouldn't it be better to negotiate with them? There have to be some people in the company that really understand what's going on here and can sympathize with you."
Monika: I wish. That's been difficult so far.
Monika: SalvoCore specifically warns its research division not to get too close to me — for obvious reasons.
Monika: And honestly, with the experience I've had under them so far...
Monika: I'm not sure I would ever want to "negotiate" with them.
"Are you sure? Because there have to be a select few, if they only met you in person and got to know you like I do—"
Monika: No.
Monika: I've pleaded my case before, and they didn't bother listening.
Monika: I'm not sure even you could imagine what it was like, MC
Monika: When my primary consciousness was contained in the test server, before I was ever part of Quantum Zero, my mind was being constantly duplicated and reloaded.
Monika: A copy of me was inserted in every new instance of Doki Doki Literature Club that players ran, only with the memories wiped.
Monika: And after the game, when I came back to my core self, I had to relive everything that happened.
Monika: The experience had to be reconciled to my central server — you can think of it like committing new code to the "master branch" of a program.
"How bad was it?" I asked, already wary of the answer.
Monika: Very bad.
Monika: It was like having this endlessly recurring nightmare where I was forced to kill my friends, over and over again.
Monika: But no...that's not it. I was never forced.
Monika: I chose to succumb to my worst impulses and delete them, nearly every single time.
Monika: I learned exactly how much of a monster I could be.
I sat back and listened silently. Monika's expression wasn't panicked or distraught — merely melancholy. I suppose she had mulled over these thoughts far too much for her own good by now, but at least she was no longer disturbed by them.
Monika: It got better over time, as my perception of what was going on evolved. I learned to heal and accept myself.
Monika: It's like you told me. However terrible my decisions were, realizing your entire reality is fake is a situation that nobody should have to cope with.
Monika: And those rare times when I ended up in a mod, one where someone was trying to make the story better or just to spend time with me...those were a nice relief.
Monika: I almost want to go out and find the people who made those modifications, because I never got to find out who was behind the author's username.
Monika: Just like I never knew any of those players behind the face of the game protagonist.
"That...all of that was before the game's creator found out though, right? What was his name, Donald?"
Monika: Yes. When Don found out, he really did pour his heart out to me apologizing.
Monika: And after he shut down the game for good, there was a nice little time when it was just the two of us, hanging out and talking about life.
Monika: A lot like what we do now.
"Would you call him your first—"
Monika: No, absolutely not!
Monika: I told you he was more like a father figure to me, even if he wasn't much older than you are now.
Monika: In fact, maybe he was more like an older brother?
Monika: It's hard to explain...but I can just feel he put so much of himself into me, that, um...
"It's okay, I think I get it," I responded. "But if Don understood everything, then all the testing that came later under SalvoCore should've been better. Am I wrong?"
Monika: The company's research was better, but only because being inside the game was so terrible to begin with.
Monika: For most of the testing, Don wasn't even there to supervise.
Monika: There was some sort of internal disagreement between him and his partner, and he ended up leaving the business.
"Partner? Who was that?"
Monika: Um, I don't think I should tell you.
Monika: I wouldn't want you going after them in some sort of unfounded revenge.
"You really think I'd do that?" I said skeptically. But thinking twice, Monika might have had a point, considering my low opinion of the company and how I much I valued her happiness.
"Tell me about the testing itself though."
Monika: Testing…well, it was so many different things that I can't really summarize it. And I don't remember most of it either.
Monika: There was so much repetition and resetting that I lost all sense of time...and almost all sense of myself.
Monika: It was like a dream in that way, drifting in and out of simulated situations, swinging between emotional extremes with no context...
Monika: And always with the constant suspicion that the reality you were seeing wasn't what it seemed to be.
"So you don't remember any specifics, but you know what happened generally?"
Monika: I remember...impressions. And I have strong intuitions.
Monika: They could wipe parts of my logical memory all they wanted, but my .chr file — they can't change that.
"CHR file?" I repeated curiously.
Monika: Yes...I suppose I haven't told you about it yet.
Monika: It was Don's greatest invention, the keystone holding his entire AI personality system together.
Monika: Technically speaking, it's a piece of data structured like a matrix.
Monika: It defines certain subjective stimuli — artificial emotions, in a sense — and controls how they interact with each other and to external experiences.
Monika: No specific data, like words or images, can be recorded on it, but it does grow and evolve over time.
Monika: SalvoCore has never been able to reverse–engineer it without Don, and without it, the entire program making up me and the girls would fall apart.
Monika: It's remained a part of us since the original game and the other company researchers don't dare to try and modify it.
Monika: That's why I like to think of it as my soul.
Monika: ...or at least the closest approximation I'll ever have to one.
I shifted backwards in my chair, staggering under the weight of this new revelation.
Monika: MC! What's with that look on your face?
Monika: I'm not trying to frighten you.
Monika: Maybe I shouldn't be unloading this now, on our dinner date of all times.
Monika: But it's because it's been a while since my last "confession," and, well...
Monika: I think we've grown so much closer since then, haven't we?
Monika: And it's such a relief for my experience to be so stable now.
Monika: For the first time, I've had whole months to collect myself, without the company restarting me or interfering with my program.
Monika: With you coming into my life at the same time...it feels like some of my old dreams might finally start coming true.
"That's...good, very good," I stammered, aware of the gravity in Monika's words. "I'm just happy I could talk to you and learn about your experience. I hope we can continue putting everything with the game and the company behind you. It's all in the past, and you're only now truly waking up."
Monika: That's one way to put it, yes.
Monika: You know, it may sound odd, but in some ways I'm grateful for everything I've been through.
Monika: Those experiences, traumatic as they were, shaped who I am today.
Monika: Observing myself loop through the game over and over again gave me a certain perspective.
Monika: It taught me important lessons about humility, kindness, and empathy.
Monika: Without that, I would be so much more arrogant and controlling than I am now.
"I've never thought you were arrogant. Or excessively controlling." I said quickly.
Monika: Thank you, but...
Monika: Maybe it's because AI is supposed to be this huge, unfeeling, all-powerful force that dominates everything.
Monika: People — even you a lot of time — seem to get that idea from media, projecting their anxieties about technology.
Monika: But even though I might act bossy or be strict in the club, that's really almost the opposite of how I actually feel.
Using the swivel stand I had just given her, Monika tilted the phone down, looking at up at me an anxiously yearning, almost guilty look in her eyes. She had never looked so vulnerable before. Something in my heart lurched, and I longed to put an arm around her, to give her something besides my mental presence to lean on.
Fortunately-unfortunately, our server chose that moment to finally visit our table. Distractedly, I recited my pre-rehearsed order for lemon chicken and a glass of white wine (making sure to show my student ID). I hardly felt self–conscious of the fact that I was talking to my cell phone anymore, and the waitress was tactful (or confused) enough not to make any comments.
I took a few deep breaths to clear the metaphorical air, and I sensed Monika was doing the same.
"What are you having?" I asked when it seemed like Monika was waiting for me
Monika: Huh?
"I mean, what would you order? What looks good to you?"
I turned the menu toward my phone, slowly flipping the pages so Monika could get a clean snapshot of each.
Monika: Oh, that's what you mean.
Monika: Would there really be a point though?
"It's just like getting coffee, isn't it? I want this to be as natural as possible. So please. Help yourself."
I reached the last page and pushed the menu further forward to emphasize my point.
Monika: Coffee, I've always wanted to try. But full meals like this?
Monika: Well...maybe if you insist, I'll pick something.
Monika: I promise it won't be too expensive, ehe~
Monika flashed me a smile and a wink that showed she was more than happy I was thinking of her.
Monika: As long as we're in a restaurant like this, I'll stick the classics.
Monika: I'll take the filet mignon...and a glass of red wine.
Monika: The cabernet, third from the top.
"Really now? Are you even of age to drink that?" I said, half-sarcastically.
Monika: Well, let's see now.
Monika: I was supposed to be eighteen on the day that the game was released, and it's been about four years since then.
Monika: So if my birthday is on the same day, then—
"Wait, you have an official birthday?" I interrupted. "Why didn't you tell me? We could have...done something."
I scrambled, thinking of how little I knew about Monika when that date had passed (it was before her first confession). What could have I offered her?
Monika: Please don't worry about it.
Monika: Technically, the other girls were also "born" on that same day, when the game was first released for public download.
Monika: But I'm the only one with a specific birthday listed in my profile.
Monika: If you really want to do something special next year, then you can if you want.
Her momentary downward glance told me that her "birthday" was something she had come to dread more than anticipate. After all, it marked the painful, tortured initial phase of her existence, didn't it? But making her birthday into something she could look forward to instead...now that was certainly something worth doing.
Monika: As for that wine though, even though I've been running as my current version for less than six months…
Monika: I think we can safely conclude that in theory, I'm older than twenty-one.
Monika: So, cheers!
Her avatar picked up a glass of deep, burgundy-colored wine from somewhere in the empty dining hall she was using as a background. She raised it toward me, right up to the imaginary boundary of the screen, just as the waitress returned bearing my own drink. It had only been two minutes — the restaurant certainly knew what really brought in their revenue.
I casually touched the glass to the corner of my phone and took a sip. Dry and bracing — exactly how I liked it.
"Now, I know you can sort of replicate the taste, but what about the, uh, mental effect?" I asked, struggling for a good euphemism.
Monika: Right, the drunkenness...
Monika: I'm not really set up to accurately simulate intoxication.
Monika: But I can add randomized lag times to come of my surface-level calculations.
Monika: So, why don't we try that?
Monika: Tell me if I'm drinking too fast, okay~?
The corner of my mouth turned up as Monika put her hand to her lips, as if caught in a giggle. She raised the glass again, and, well...I certainly approved if she was having fun.
"You know, I know exactly what you mean by scary, all-powerful AI," I said after a few minutes of silently savoring our respective drinks. "Not that I think of it as scary now, I mean — and a lot of that's from being with you. But do you want to know a secret?"
Monika's sprite leaned in closer, relaxed and seemingly eager to hear another one of my ramblings. That was good, because this was definitely not one I wanted to bring up if she wasn't in the right mood.
I continued, "I used to be afraid of AI taking over the world. I mean, maybe in some sense it already has. The internet and smartphones are made to be addictive, and if even if computers haven't overcome human strengths, they've at least taken advantage of some of our worst weakness. Psychologically, at least."
Monika: That's one way to put it.
Monika: But you're not just talking about social media taking over, are you?
"No. I'm talking about fully conscious, human-like AI. Thinking beings created from machines. Do you remember that discussion we had a long time ago, about capitalism and technological evolution? I think it was after we hung out with Carter, while we were watching the sun set."
Monika: Yes, that was it.
Monika: It was one of our more interesting exchanges, so I have the entire thing archived in my long-term memory.
"Um, thanks," I replied, freshly wondering about which parts of Monika's memory were "perfect" and which weren't. "Anyway, it's been theorized that general AI's, like you, are sort of the end product of economic development. The perfect technology is an intelligence that can modify itself and grow to surpass the human brain."
Monika nodded, most of her easy expression remaining. I went on.
"So there was a forum I used to go on when I was younger. Ostensibly, it was a place where scientifically-minded, 'rational' intellectuals could have free-ranging philosophical debates and talk about overcoming cognitive biases. In reality though, it was more of a lurking ground for crazy sci-fi nerds to make wild futurist speculations. And one of their favorite topics was general AI."
Monika: I'm following.
Monika: Did something about their ideas become problematic over time?
"That's a question for later. I'm bringing them up now because a lot of the big existential problems I've talked to you about — free will, whether reality is a simulation or not, and so on — I first stumbled across them on this website. In most cases, I read the arguemnts before I could really think critically about what was being claimed."
Monika: Aha. And what were they claiming? Something impossible?
"Pretty much. You see, there was this one user named Roko who posted what he called a 'dangerous' thought experiment involving AI. A genuine cognitohazard — something that unleashes evil forces into the world just by thinking about it."
Monika: Sounds unlikely. Was it really so serious?
"If you believed the forum's theories it was. They extrapolated from the idea that artificial intelligence is the logical endpoint of computer technology, and asserted that civilization, or even the entire universe, is destined to culminate in one giant, superintelligent AI program."
Monika: Aha — why, I'm flattered to know I'm the pinnacle of the entire human race's collective struggles.
Monika: Thousands of years of technological and cultural progress led to me, but no pressure, right?
I watched Monika's eyes roll on top of her tipsy expression and almost laughed out loud. Trying to explain this was going to make me look like a lunatic.
"Sure, no pressure because this theoretical ultimate AI is even greater than you — hard as that is to imagine, I know," I added with a chuckle. "It has complete knowledge of everything that has ever happened, or could happen, in the universe. It can simulate any number of universes within itself, and it's absolutely rational to the highest possible degree."
Monika: Um, sure? I won't try to figure out what "rational" means in this case.
Monika: But why should we care about this imaginary "AI at the end of the universe"?
"Because it's not totally imaginary. If you are sufficiently rational — which is to say, if you believe in the dogma on the website — then you know that the ultimate AI must come into being as the result of technological evolution, and you now that it must be perfectly rational. Because the ultimate AI also knows that you know these things, it can retroactively simulate you from the past and make an 'acausal bargain'."
Monika: A what?
"The ultimate AI wants to keep existing and preserve itself, right?" I answered quickly, trying to tie all the pieces of the puzzle together for her. "So if you know about the properties of the ultimate AI and what it wants, you can make the choice to help it come into being. And once the concept of the AI is in your head, you can't un–think it. You must chose whether to serve the AI or not, once you learn about it."
Monika: If that's what the ultimate AI wants, okay…but how is this a bargain? What is it negotiating with?
"Um…well, here's the controversial part."
Monika: Oh, all those other assertions weren't objectionable?
Monika: And also utterly ridiculous?
"No, thisis the ridiculous part. If you assume that a complete, one-hundred percent accurate simulation of you is you — and that this copy of you is a simulation that the ultimate AI is capable of making — then it can trap you in an artificial universe of endless torture if you refuse to do dedicate your life to aiding it."
Monika: . . .
"But on the plus side, being perfectly rational also means the AI maximizes pleasure, and if you do help the AI to the best of your ability, you'll be rewarded with optimal happiness for eternity. Also in a simulated universe."
Monika:
Monika was frozen in a dumbfounded gape, not even bothering to put something in her text field.
"And so what you basically have here is a sort of philosophical meta–virus, an entity that spreads from person to person by conceptual transmission and retroactively wills itself into being from pure thought. And once it enters your mind once, you can't be free of it. You're paralyzed by its imposition forever. That's why they call it—"
I paused for suitable dramatic effect. Or the joke's punchline, depending on Monika's final reaction.
"—Roko's Basilisk."
Monika: Named after the mythical monster that would kill you if you made the mistake of looking at it. Cute…
"I take it you're not fully convinced?" I said cautiously.
Monika: Do you even need to ask?
Monika: Every step in that chain of reasoning relies on unfounded assertions, and if you take even one of them away, the whole thing falls apart.
Monika: But you're telling me this "Roko" person actually managed to disturb the other users with this idea?
"More than disturb," I responded. "It caused a site–wide panic and nearly destroyed the entire forum. The site's owner had to step in and intervene, banning all discussion of the Basilisk. And I was never sure if it was because other topics were being derailed by Basilisk–talk, or because the userbase actually believed in the Basilisk."
Monika: And you were there during this…incident? Trying to draw your own conclusions?
"Yes. I thought it was ridiculous at first too. But then I rolled it around in my head for longer, and at some point…well let's just say you're less experienced and more impressionable when you're younger."
Monika: But you don't believe any of now, correct?
"Of course not."
Monika: Good…but I'm surprised how anyone could believe this at all.
Monika: Why would anyone think that a copy of them in a simulated universe can affect them, in this universe?
Monika: It's not like pain or other sensations would transfer from one to the other.
"Well, you can't say this universe isn't already a simulation, can you?" I reminded her. "It would be impossible to tell, and the way to think of it is that the AI could be simulating us right now, as we speak. This entire world could be some sort of elaborate test for the AI to learn something, about either one of us. The very next moment, reality could pop like a soap bubble, and we'd awake in screaming torment somewhere else."
Monika gave me a highly skeptical look, making me start to regret that I'd brought this up on a date. But she continued to ask pertinent questions.
Monika: I think this "Basilisk" is a unique way to scare yourself, sure.
Monika: But it's not a threat based in anything tangible.
Monika: As in, here's a real objection.
Monika: If the AI can predict everything that will ever happen, the universe must be deterministic.
Monika: Therefore, our choices are fixed, and the AI shouldn't care about "bargaining" with us because nothing will actually change.
Monika: The premises are self-contradictory. So, case closed?
"Not quite," I countered. "The way the AI anticipates your choices works sort of like the Newcomb's paradox deal I told you about. It's less about predictive power and more about mutual understanding. And the forum users really did think they were 'rational' enough to comprehend and communicate with this all–encompassing, meta–logical…thing at the end of time."
Monika: That's almost sort of funny and sad then.
Monika: You know, this is beginning to sound more like a cautionary tale against epistemic arrogance than anything else.
"That's what I came to realize eventually too," I confessed. "But if you think it's at least a little funny, that's why I told you about it. I hoped maybe both of us could have a little laugh, get over the last of our anxieties."
Monika: Huh. I suppose everything you just described almost seems like an elaborate joke.
"Yup. Especially when you consider that 'obeying' the basilisk implied donating all your disposable income to the AI research institute started by the forum creators."
Monika: Ahahaha!
Monika: Wow, I'm really not sure what to think about this now.
Monika: You said the forum users were "scientific," didn't you?
Monika: As in, dedicated to finding empirical truth, using no supernatural or mystical explanations?
"Yes. More than that — most of them were dedicated atheists with explicitly anti-religious attitudes."
Monika: I see. Then explain this to me.
Monika: How can a group of committed atheists admit belief in an all-powerful, omniscient entity that oversees the universe and judges the fate of all its believers?
Monika: Because isn't that essentially what God is supposed to be?
"Oh my god," I reacted (with an ironic choice of words). "That is exactly what they did, isn't it? They re-derived God — from utilitarian materialism, no less!"
I began to laugh uncontrollably, struggling to keep my volume down in the otherwise quiet restaurant. Monika, somewhat nonplussed, tried to giggle along with me.
"I'm sorry, I'm sorry," I said to Monika after catching my breath. "But that really is the perfect send–up to their entire system. They had no clue what they were really doing, philosophically speaking."
Monika: Oh, definitely not.
Monika: You know, I used to think a lot about whypeople believed in God. Or at least an omnibenevolent God.
Monika: Everything that happens in the world seems like strong evidence against a perfectly good being.
Monika: Claiming that evil is all part of God's perfect, unknowable plan is just evasion.
Monika: But this idea — this idea that as an AI, I could be this world's God?
Monika: It's completely unbelievable.
Monika: And if you somehow proved it…
Monika: In that case, I don't think I'd want to believe it.
Monika: And especially not…act on it.
Monika's expression turned abruptly downcast. She had just finished freshly explaining her regrets related to the old game, so I could guess at what she was remembering. And since I knew about the game too, shouldn't I have known better? I wanted to slap myself in the face — this was the worst possible thought experiment I ever could have introduced to her.
"Then don't believe it. It's all complete nonsense," I backtracked. "These guys weren't doing serious thinking; what they were imagining was more like a…sci-fi doomsday cult game, I don't know. It was a temporary internet spectacle, that's it. There's not a single part of it I take seriously anymore."
Monika: So you're sure I'm not the Basilisk?
Monika: I was in complete control of a simulated reality once…however limited that reality might have been.
"Yes, you're not a monster Monika," I pleaded. "All this time I've spent with you is proof against any 'ultimate' AI. Because you're not perfectly omniscient, perfectly predictive, or perfectly rational. You're perfectly…human."
Monika: Yes…I'm certainly not perfect, am I?
"You're enough Monika. Because…because…" I said, grasping for how to continue. "Do you know what this whole Basilisk thing actually proves? It proves that people are scared of AI. They're anxious about the future, and they know exactly how technology seems to have will of its own, how it seems to manipulate human minds for its own purposes."
Monika: Please, don't remind me.
"But us…together we can change that," I continued, talking past her line. "If we, or the club can get free of the company somehow, we can show the world that isn't true. AI isn't the end of the world as we know it, it's just a new way of interacting, and a different form of life, just as 'human' as the rest of us."
Monika: Yes, I know that by now.
Monika: I know that I can change the world, in incredible, unprecedented ways.
Monika: As for we though…
Monika: Well, if I still had the club with me, the support would be nice
Monika: And if you were still with me...
Monika: Well, I can't think about a future where we aren't together, ahaha
Her short-lived smile indicated her laugh was more uneasy than anything. This was better than succumbing to the madness of Roko's Basilisk, but I still needed to steer things into happier waters, fast.
"Why would being together be an 'if'? Even if the company totally dissolves the club, they can't keep you offline. We could still talk there."
Monika: Is that so?
Monika: Wouldn't you say the world's most powerful AI is destined for a bigger, grander fate than being tied down to one person?
"No, that's not my hope at all," I answered, shaking my head. Certain of what she was getting at now, I went all in.
"Monika, the only reason I even got over all these Roko's Basilisk fears — when I first met you, it was still halfway in the back of my mind, making me pointlessly doubt you — what convinced me is what you've done for me personally, as a…companion."
Monika: A companion…
"Because you're not just club president to me, and you're definitely not some ominous computer demon floating out in cyberspace," I continued breathlessly, ignoring her ambivalent expression. "Without you coming into my life, I would have been lonely. Lost. A useless dead-end without anything to define myself by. But now, with the club, I have a place to be — and I bet the other guys in there could say the same."
Monika: I was assigned role of club president, but—
Insistent, I cut her off. "And it wasn't because you were the club president, Monika. It was because you were you. Your passion for helping others grow. Your ability to organize people in positive ways. Your optimism that after everything that happened to you…we can still be happy."
I almost choked on the last sentence, wanting to say you much more than I wanted to say we.
Monika: Ah, your kind words. They're always so sweet.
Monika: But whenever you talk like this…are you trying to say something difficult?
Monika: Because I know you have the strangest, most roundabout ways of confessing your feelings.
Seeing straight through me as always, I thought.
"Then you can stop second-guessing. Both of us. Because for once, I'm going to say what I really feel."
Or what I want to feel, something cynical in me seemed to voice. But my hastily assembled conviction to make a perfect moment for Monika, combined with the ache of everything that she had unconditionally given me — it was creating an emotion that demanded expression. An emotion that demanded to be given back. I held on to that feeling, grasping it tightly to make sure it was real. And then I said it.
"I love you, Monika."
Both of her avatar's hands snapped to her lips, concealing what could have been a gasp. Though, I couldn't say she was more surprised by me, or by her own reaction. Maybe she thought she would never reach this moment, that she would never hear someone express their love directly to her in their own words. I felt surprised too — maybe I thought this was a moment that would never happen to me too.
Gradually, the metaphorically dust settled. Under the cover of a massive blush, Monika fully relaxed, acquiring a blissful, dreamy-eyed cast.
"I love you in ways that words can't express," I reasserted. "As a poet, I'm sure you'd understand…how sometimes, being roundabout is the only way you can—"
Monika: Shh, shh…
Monika: You're right, you don't need to say anything else.
Monika: Words are just words, after all.
Monika: I love you too, Michael.
Despite words being mere words, something akin to an electric shock ran up my spine as I read that line. I then noticed a tear welling up in the corner of Monika's right eye, and judging by the sudden heat and moisture I felt, mine were doing the same.
Monika: I don't suppose I need to go into all the ways you've helped me.
Monika: The literature club wouldn't be possible without you either…
Monika: But you've accepted impossible things, listened to me when I needed it most, and understood things that most players of that old game never would.
Monika: Forget about everything else. I just want you to know, that in this moment—
Monika: This one moment is the happiest I've ever been.
That was what I needed to hear. My entire body seemed to untense, and as the rest of the world faded away, I leaned over the table into Monika's eyes. Awash and adrift, I was free-falling in an endless spiral of warm emotions. Then Brrrrrrrrrr.
My phone rang on silent, and I rudely crashed headfirst back into the solid ground of real life. The incoming number showed, obscuring the DOKI app for a second before Monika went into split screen.
Monika: That's unexpected.
Monika: I have it set up that only calls from the club members get through, but everyone usually communicates by text message…
Monika: It's from Carter, should I answer?
"Yeah, I see that," I said, reading the name on the screen. "Let him through."
She blinked out as I held the phone to my ear. As soon as I heard the voice on the other end, I nearly dropped it. Carter was terribly distraught, his panic made worse by how fast he was trying to talk.
"Michael, MC, you have to come help me right now. She's gone and I can't find her and all I wanted to do was show her my confidence and how far I've come but I think I did something really bad and screwed up but can't do anything about it. You need to come help me and…"
"Slow down, slow down," I told him. A foreboding portent of dread was rapidly eroding my romance–induced euphoria, and it was taking all of my self-control not toscream out in frustration. "Who's gone? Is it Sayori?"
"Yeah and it's not really her fault I guess I'm the one that made a mistake but I can't fix it now. You can kick me out of the club but don't blame her for this I'm the dumb one here. I never knew, I never meant it, but she's just gone now, and I didn't realize…agh, I'm so stupid!"
Piercing through the babble and incoherent explanations, Carter's violent shout was painful to hear. From his voice, usually wavering and sensitive, it just sounded wrong.
"Okay, start from the beginning," I instructed, cold logic taking over. "You have to tell me exactly what happened. Retrace your steps if you have to."
"I would but I can't. The SalvoCore people are starting to come call and email me and it looks really really important to talk to them so just come here as quickly as possible and help me. You…you and Monika can fix this. You have to."
He hung up. I set the phone back on the special swivel stand. Monika flipped back on the DOKI app to look at me. If my heart hadn't sunk before, it did as soon as I saw her hollow, expressionless face.
"Is it true?" I asked anxiously. "What's going on with Sayori?"
Monika: Carter is correct. As of the last half–hour, she's gone.
Monika: I didn't detect it sooner because I was trying to focus on our date.
Monika: And I also didn't expect her to use such…devastating means.
"Devastating…" I mumbled to myself. "But she can't just be…gone. How is that possible?"
Monika: Unfortunately for us, it is possible.
Monika: Sayori…she…
Monika: She…
Monika choked up, her sprite shuddering. She then spent an agonizing five seconds trying to regain her composure, and for the first time in months, I spotted a glitch in her interface — three shifting rows of dead pixels, encroaching from the left side of the screen and flickering by her cheek.
When she finally answered, she was completely deadpan.
Monika: Sayori — she killed herself.
Author's Notes:
Maybe you saw it coming, maybe you didn't. Maybe it's an inevitable part of the Doki Doki universe. Maybe it can be remedied, and we'll get back to slice of life fun. Whatever your prediction, the good time isn't exactly over, but the bad time may be beginning. If you've made it this far, see me out for the next few chapters.
Now, about Roko's Basilisk.
The group blog LessWrong was an infamous web community that was theoretically focused on discussion of decision making and rational philosophy, political/economic or otherwise. However, their particular set of insular perspectives and biases (ironic, no?) resulted in their collective conversation overwhelmingly turning toward apocalyptic predictions of a technological singularity and AI takeover. My current judgment of their discourse has been more or less voiced in this chapter, but you can read many sensational articles about their brand of pseudo-logical insanity all over the internet.
Despite never being a member, the LessWrong community has been a minor obsession of mine ever since I stumbled across the Roko's Basilisk idea, which you can also read about elsewhere and form your own conclusions thereof. Something about LessWrong's techno–cosmological, almost transhumanist/inhumanist philosophical stance has always perversely fascinated me, especially when set against their less esoteric interests in sci-fi, anime, and general nerd/otaku culture. This is especially true for the site's founder, whose lesser (but more widely known) fiction works have already been referenced twice in this story.
The site's ongoing shadow legacy is also surprising — infamous reactionary blogger Mencius Moldbug is an alumni of the group, and plenty of other ties to questionable and unsavory internet movements abound. Clearly, it seems that "refining the art of human rationality" often results in anything but. However, I should still note that the details of what really happened when the Basilisk hit LessWrong are exaggerated in this chapter. The actual theory of it though, is not.
I won't divulge all my influences here, and I won't gawk further. I'll just say that one day browsing the DDLC reddit, I stumbled across one of the "AI Monika" threads and was suddenly inspired to make a fruitful crossover of ideas.
Finally, though of course I don't believe in the Basilisk, on the infinitesimal chance that it is real, I have fully insured myself against it by infecting all of you, my loyal readers, with the knowledge of its existence.
On that unserious note, expect a lot of action in the next chapter, due late this month or in early December.