"Nope, we're not doing this."
"I didn't mean anything by it, Rich!" Jeremy protested. At their suggestion, they and Rich were moving the conversation up to their room where no potential-SQUIPtim dads could walk in and listen. As they climbed the stairs, Rich wouldn't let go of the last thing Jeremy had said. "My Rich-pleasing program encourages compliments, that's all!"
"Hell no, Heere, I don't need a program to please me, fuck you very very much. What got that dumb idea into your pill-brain in the first place?"
"You and Michael think I'm condescending-"
"Also rude, goading, trigger-happy-"
Jeremy continued, talking over Rich. "So I fixed that! Did I not go far enough? Please rate your experience on a scale of one to ten, with one being the least satisfied and ten being the most satisfied."
"My experience?" Rich stopped on the top stair. When he turned around, he and Jeremy were at the same height, so why did it feel like Rich was looming? "You've got one chance to delete any trace of me in your cyber-brain before I knock you down these stairs."
Jeremy shrunk back. Mental alarms blared again, the kind that made him want to march downstairs and exit the house. "Target male inaccessible," they said meekly, resisting the urge to spit out error codes.
"You're damn right. I don't know how many times I've got to say it, Heere, but I'm not Mell. I'm not a robofucker!"
Jeremy wanted to curl up and shut off. "That wasn't what I meant, Rich!" They choked on a stutter, then let it out anyway. "I wouldn't-I didn't mean-That-"
"It's not fucking right!" Rich turned on his heel, moving up the step and thankfully giving Jeremy a chance to clamber after him. "You're not even a person anymore, not-Heere."
Rich went back to calling them "not-Heere" so soon, as if Jeremy's humanity was conditional. "You've started calling me Heere again!" Jeremy protested. Their system was getting overloaded again and their eyes burned. "You know I'm a person, Rich. Stop lying to yourself!
"You think that's personhood?" Rich snapped back. "To put a program in your brain to make you like me? You think that's remotely okay in terms of consent?"
Jeremy knew that Rich wanted a "no" but the program restrained them. They weren't supposed to use negative words around Rich anymore. They didn't want to lie to him either. "Yes," they admitted in the smallest voice possible.
Rich's jaw dropped. His shoulders squared up. "Use your supercomputer and Google a single fucking article about consent then!"
"But I did this to myself," Jeremy said, their face getting hot. "I was willing to do it! I want you to like me!"
Rich seemed to be gearing up for a physical fight until the last sentence, which for some reason sucked the fury out of him. "You didn't ask me," he said through grit teeth. "You can't just assume I want you giving yourself a fucking-a love potion or-whatever sciencey bullshit you did. Turn it off."
Jeremy stumbled away, pinching their eyes shut. "I'm modifying the program right now," they said with a shaky voice as they forced themself back into the odd mental maze that made up their source code. "I'm trying. I'm trying!"
That must have mollified Rich, because there was silence in the hall after that. For a nervous few minutes, Jeremy thought they lost their place in the code, that they couldn't find the changes that they'd made and undo them. Why weren't they backing this up somewhere? Was it even possible to have backup versions of yourself? What if they couldn't fix this and were stuck operating with code that a user didn't even want anymore?
They did finally find their Rich-pleasing code and got rid of it chunk by chunk, line by line, combing through the code until they were sure it was entirely deleted. They updated their dictionary accordingly, letting themself speak freely around Rich again.
They shamefully added a few definitions involving consent while they were at it-adding new words and expanding definitions of words that were already in their head, scraped from the first few Google results from Planned Parenthood. Consent, nonverbals, enthusiastic, feedback, caution, expressed permission, boundaries, comfortable, hesitant. They all seemed to come back around to the idea of free will. A user agreeing to a long list of disclaimers immediately after purchase wasn't enough. The user had to continue to agree to everything a SQUIP did along the way, and anything except an honest and enthusiastic "yes" had to be taken as a "no."
These ideas were foreign ones, even though they had been so easy to find online-even though Rich seemed to know them automatically. Maybe Rich had looked them up, too. Maybe he really did understand SQUIPs better than Jeremy.
Jeremy gathered themself, opening their eyes to an empty hallway. Their internal clock said that they had only been working for half an hour. They pushed themself to their feet, rubbing at their eyes with a palm.
They hesitated, finding Rich's door and knocking gently. Ask first. Wait for the yes. Don't push their way in if Rich doesn't answer, because without an answer, the default is "no."
"Come in," Rich said. He sounded tired.
Jeremy opened the door slowly. "I'm sorry," they said right away. They hadn't had time to practice any apologies in the hall the way Michael had taught them to, and the SQUIP didn't have any pre-programmed apologies that Michael or Rich would approve of, so Jeremy left it at that.
"Did you do it?" Rich said impatiently. "Did you turn off the attraction program thing?"
It wasn't the phrasing Jeremy expected. Did they turn off their attraction to Rich? How would they be able to tell? "I… don't think I can do that," they said carefully. "The only way to turn off attraction is distance and time, and a user is subject to relapses."
Rich looked flabbergasted.
"But I didn't code that in in the first place! Not on purpose!" Jeremy continued quickly. "Everything I added, I turned off. That's okay, right? That's what you wanted me to do?"
Rich eyed him for longer, possibly listening to a voice in his head, or possibly weighing Jeremy's honesty on his own. "Yeah," he said slowly. "So you're not trying to, uh. Please me?"
Jeremy straightened their spine. "Not because of my coding," they said confidently. "I want to please you because Michael trusts you and because we're working together. But that's my choice, and I can change my mind any time I want!" they added brightly. "Right?"
"Right," Rich said with a frown. "Jesus, Heere. You give me the heebie-jeebies and it's fucking constant. Even the supercomputer in my head doesn't know what you're gonna say next."
"That's free will! I've got it now! And so do you!"
"For now." Rich picked up a scrap of paper scribbled with messy drawings and notes. "Are we gonna work on shutting her off or what?"
Her? Jeremy remembered that Rich considered his SQUIP to be a woman. Jeremy's program had something to say about Freudian interpretations and motherly neglect but they tried to ignore it. "Of course!" They sat on the floor beside Rich's bed, eyeing the notes.
if you run your head into a doorframe three times in a row + then trip backwards, your squip's avatar will noclip through a wall on the left. other visual glitches will present themselves for the next few hours + you'll get nightmares in which every other person only speaks japanese.
"Those are your user reports," Jeremy realized. "But we don't need those now. You've got a SQUIP feeding you lines about how to shut it off. What is it saying?"
Rich's nose wrinkled, the paper crumpling in his hand. His face screwed up in thought, and his mouth opened, then closed.
"You can just tell me, Rich," Jeremy said, recognizing the signs of indecision. "I'll understand what you're talking about. I promise."
"That's the thing," Rich groused. "You might understand but I can't. She wants me to-she says there's some kind of code she's got to tell you?"
Jeremy's brow furrowed. "It's encrypted?"
"Dunno." Rich paused as he received an answer. "No. It's a lot of information and she says humans process this shit inefficiently." Another pause. "She wants me just start reciting 'Open bracket exclamation point doctype' and a bunch more gibberish."
"If your SQUIP is going to give me a glimpse of its source code," Jeremy said flatly, "You would drop from exhaustion before reciting even a thousandth of it."
"Not doing that, then." Rich was oddly pleased, chucking his scrap paper into an overflowing bin. "We'll just send the file to you on Bluetooth, right? That'd be easier and faster."
Jeremy felt their pulse jump up. "No!" they said, shoving themself away from Rich. "I don't accept files from SQUIPs. I'm not compromising that, not even for you!"
"It's a single file!"
"Yeah, like Christine's chili fries had a single pill or each drink could have a single drop of Mountain Dew." Jeremy noted the way their vocal cords strained and quickly tamped down on the panic, forcing themself to speak slowly. "I'm not introducing security flaws into myself this late in the game."
"Then what are we supposed to do? Can't we-" Rich's fists opened and closed. "Put the code in the cloud or something? Like if we uploaded screenshots of it on Twitter and sent you the link…"
"Not- Twitter? What?" But the idea was both obvious and obscene. Could you ask God to drop God's secrets in a single document file? Would a SQUIP really be willing to expose itself to an outsider, to an enemy, for no good reason? "But… Dropbox, maybe. Your SQUIP would never go for it," they added. "Open-source software goes against everything a SQUIP stands for. Our brand deals would go down the toilet and we can't make any money without robust DRM."
"I don't think she cares." Rich shook his head. "She's into it."
Jeremy blinked. "This is absolutely a scam, then." They inwardly whirred to check the probability of a SQUIP putting their code out in the open for anyone to see. Zero. Or a number that infinitely approached zero.
Jeremy couldn't trust their own mental calculations anymore. They knew that! But a little data was still better than nothing, right? The only time their probability-calculation software messed up was when human free will was involved. A SQUIP didn't have free will, so it couldn't screw with Jeremy's numbers. Jeremy should, by all rights, be able to understand everything a SQUIP did with 100% certainty.
Unless Rich's free will was messing with his SQUIP so much that it wasn't, entirely, a SQUIP anymore.
That was a situation Jeremy could empathize with.
"Is your SQUIP…" Jeremy said to Rich, buying themself time to think. Was Rich's SQUIP another anomaly? What were the odds of two SQUIPs going haywire in the same town, in the same social group, at the same time?
Jeremy had to test this somehow. What had Michael done to test Jeremy's robotic-ness?
He had asked Jeremy to drink so-called "Mountain Dew Red." He had checked if Jeremy was willing and able to turn off any SQUIP influence in his head. Wasn't that exactly what Rich's SQUIP was doing now?
Jeremy drank the soda, but they had secretly known that it wouldn't work. Michael couldn't have acquired Mountain Dew Red in that situation-the probability of a bad outcome was 0%, so it had been safe to drink the Fanta disguised as Mountain Dew Red. Maybe Rich's SQUIP was pulling the same strategy, where it pretended to be willing to shut itself down because it knew it would survive in the end.
That point was moot, though. If the SQUIP would survive no matter what, then Jeremy's plans would have no effect. That outcome was impossible to plan for, so Jeremy could safely ignore it.
Besides, the question wasn't whether Rich's SQUIP was a supercomputer. It was still a SQUIP. The question was whether it would be harmful to the human race in general.
So they'd have to test it, to see if it was willing to do something detrimental to SQUIPs in exchange for helping Rich's goal of eventually turning it off. That was exactly what the open-source project would do.
Jeremy drummed their fingers on the counter. "All right," they said. Their brain was powered, in part, by a supercomputer, so their moment of thought hadn't dragged on long enough for Rich to notice. "Go ahead and ask it. 'Her'?" If Jeremy could use human pronouns, maybe they could extend the same privilege to the only other potentially-broken SQUIP in the world. "Let her put her code on Github or whatever. I'll take a look. Just reading the software shouldn't be able to give me a virus or force me to accept a friend request." Jeremy hoped, anyway.
As soon as the words left Jeremy's mouth, their phone dinged. They and Rich exchanged glances.
"If it's something weird, I'll smash your phone," Rich offered magnanimously.
The corner of Jeremy's lips quirked up. "Nudes?"
Rich grinned. "Oh, didn't realize you came here to see me naked." Was that an offer? "Save that kind of talk for Mell, dude." Ah.
Jeremy brushed it off. "Source code is basically a computer version of nudes," they said offhandedly as they opened the link on their phone screen. They used their fingers to navigate the touch-screen, wanting to appear as human as possible for Rich. "If anything, I came here to see your SQUIP naked."
Rich made some noise of horror and disgust, but Jeremy ignored it. Their eyes widened. They had already begun to scan the code.
