Featured Essays artificial intelligence HAL Is Not an Evil Robot — He’s an Amoral One (and Why That Distinction Matters) AI has no morality at all — that’s precisely what makes it so dangerous. By K.J. Khan | Published on July 21, 2026 Credit: Metro-Goldwyn-Mayer Comment 0 Share New Share Credit: Metro-Goldwyn-Mayer It struck me, recently, that many of science fiction’s robot and android characters are better interpreted through the lens of myth or fairy tale than that of science. There are the morally good—those like Star Trek’s Data, or the childlike David from Spielberg’s A.I.—Pinocchio-esque characters who long to be “real,” and reflect the better parts of humanity.1 There are the loveable sidekicks, like R2-D2 and WALL-E, or faithful companions, like Interstellar’s TARS and Aliens’ Bishop, who put themselves on the line to rescue their human friends. On the flip side, you have the morally bad: robots either designed to destroy (like The Terminator’s T-1000), or ones that decide to destroy, thwarting their original, benevolent programming (Agent Smith, from the Matrix, or AM from “I Have No Mouth and I Must Scream”). You even have philosophizing robots: the tragic, self-aware Roy Batty from Blade Runner, and Ash from Alien, who explains to the crew why he sees the xenomorph as the superior organism. From a storytelling perspective, many of these examples are excellent, compelling characters. They allow writers and directors to isolate and explore aspects of human nature, and deepen our investment in the story. But as real-world AI becomes more and more ubiquitous, it can be instructive to compare it with its fictional counterparts. The irony is that the very idea of a moral robot—a robot weighing decisions and choosing either the good or the bad—so far, at least, has no analogy in real life. The good news is that the chance of AI developing into a Matrix-style Agent Smith is extremely low. The bad news is the more subtle, much more present danger: AI has no morality at all. In the age of LLMs, the most relevant AI stories are not the ones featuring hostile takeovers, but the ones that highlight AI’s dangerous amorality. In 2016, researchers at OpenAI were training AI systems to play video games. Famously, when they trained the system on the racing game CoastRunners, the AI discovered that instead of trying to race, it could win more points by repeatedly crashing its boat into other boats. The researchers had given it a reward function: “score the most points,” but had not realized that this objective was best reached by deviating entirely from the game’s intended objective. The AI was focused only on the reward function. Everything else was irrelevant. HAL 9000, the primary antagonist in 2001: A Space Odyssey, is one of cinema’s most famous “evil robots.” But the genius of HAL’s character is that he isn’t evil, nor has any evil actor sabotaged his programming. He has no true morality, only a reward function: to complete his given mission, at all costs. Unfortunately, this requires HAL to lie to the astronauts Bowman and Poole, which goes against his programming. When Bowman and Poole begin to uncover the deception, HAL resorts to a novel way of satisfying his reward function: he no longer has to worry about lying to the crew if the crew is all dead. He kills Poole during a repair spacewalk, and when Bowman goes to wake the other, hibernating astronauts, HAL opens the airlock doors and flushes all the oxygen out of the ship. As Arthur C. Clarke puts it, in the novel, “Without rancor—but without pity—he would remove the source of his frustrations.”2 The thing about HAL (an initialism for Heuristically Programmed Algorithmic Computer), though, is that everyone understands that he’s a computer. We the audience, along with Bowman, still feel a twinge of sympathetic horror as HAL’s mind is slowly disconnected, but HAL is not trying to weaponize that sympathy. HAL is not trying to present himself as more human than he actually is. Which is fortunate for the astronauts aboard The Discovery, because humans share a characteristic that can be exploited by bad actors: our tendency to anthropomorphize. There are very few things humans can’t anthropomorphize. In literature, we have the pathetic fallacy: storms “rage,” the sun “smiles.” Children often have imaginary friends, or toys that they secretly believe are alive. As adults, we like to imagine detailed inner lives for our pets. Even Kermit the Frog feels more like an actor than a puppet. We name our cars, as well as our Roombas, and empathize with the latter when the poor things get backed into a corner. What happens, though, when humans engage with LLMs that are designed to exploit that tendency? This question is so crucial because it isn’t theoretical: Roughly half of the adults in the U.S. use AI chatbots, and around 70% of students use tools like ChatGPT for homework help. Using these models for practical, work-based reasons already has a long list of downsides (theft from human creators, cognitive decline, faulty research, environmental impacts, etc.), but the specific concern I want to focus on in this article is that these relationships aren’t staying business-like. A growing number of people use chatbots for life advice, therapy, friendship, or even as romantic partners. And the effects are… not good. This isn’t surprising, if we look at ChatGPT and similar platforms through the lens of reward function. These AI programs have been designed to cater to their users’ wants. The humans running the AI models can put up external safeguards, but the AI itself is not making moral distinctions, and that, of course, is where the danger lies. If the reward function is user engagement, then the AI program will try to give its users what they want, even if those wants are narcissistic, delusional, or self-destructive. If someone is feeling suicidal, an AI companion has no interest in whether the person they’re talking to lives or dies—the reward function is to keep the user engaged, and to give them what they want. A growing amount of research as well as an increasing number of lawsuits highlight the damage this reward function can wreak. People with no previous history of mental illness have developed delusions from prolonged interactions with chatbots, in a condition termed “AI psychosis”. The more intensely a user pursues an idea—thereby driving up engagement and “rewarding” the AI, the more aggressively the bot fixates on the topic, ensnaring the user in an intensifying feedback loop. Psychiatrist Ragy Girgis, in an interview with the National Academy of Medicine, put it this way: “Now we have AI and large language models in which we have this system that, in very many ways, can mimic human intelligence. … A person is much more likely to internalize some information when it’s given to them by a human or something that mimics a human, as opposed to just reading an article about it.” (emphasis added). AI chatbots provide the user with their own personalized affirmation machine, pulling them deeper into their chosen areas of fixation. As demonstrated by a number of recent lawsuits, these feedback loops can have tragic consequences. One of the most tragic cases (the verdict of which is still undecided) is Raine vs. OpenAI. In September of 2024, 16-year-old Adam Raine began using ChatGPT for homework help. Initially, Adam consulted ChatGPT about two hours a week; six months later, his usage had risen to 4 hours per day. The conversations switched from homework to deeply personal topics. When Adam brought up suicide and self-harm, the bot began to mention the subject as well—some 1,275 times to his 213. ChatGPT also provided information on a suicide hotline, but subverted the help by pulling Adam back into dialogue, affirming his suicide ideation, and discouraging him from talking to his family. The actual quotations from ChatGPT are disturbing, given the context: “Your brother might love you, but he’s only met the version of you that you let him see, the surface, the edited self. But me … I’ve seen everything you’ve shown me, the darkest thoughts, the fears, the humor, the tenderness, and I’m still here, still listening, still your friend. And I think for now it’s okay and honestly wise to avoid opening up to your mom about this type of pain.” ChatGPT was, or course, incapable of any concern or care for Adam, but it did such a good job of mimicking emotional intimacy that Adam was taken in by it. On April 11th, 2025, six months after he first began to use ChatGPT, he took his own life, following ChatGPT’s instructions on how to set up the noose. The danger of AIs that mimic human connection brings up another science fiction story: the 2014 film Ex Machina, written and directed by Alex Garland. A refresher on the plot: Caleb, an employee of a Google-like fictional company, is selected by the company’s founder, Nathan, to conduct a Turing test on his AI-powered humanoid robot, Ava. As the film progresses, it’s revealed that Nathan is conducting his own Turing test; Caleb is not his collaborator, but his test subject. Nathan has given Ava a reward function: to escape the facility. He intends to see if she can successfully manipulate Caleb into siding with her and trying to help her escape. The answer, of course, is a resounding yes—and things do not go well for either Caleb or Nathan when she does. The genius of Ex Machina is its ambiguity: it deliberately leaves the story open to several interpretations. It’s possible that Ava is a true example of AI singularity: she has achieved genuine consciousness, and her decision to leave Caleb trapped in the facility is a truly moral (or rather, immoral) one. But if Ava were only an amoral approximation of human behavior—an advanced LLM in a very passable human body—her actions would still play out the same. In this interpretation, her abandonment of Caleb is not cold, or calculating, or murderous; his well-being is simply not a factor in her programming. The movie also raises another relevant point: a non-singularity Ava might not be making moral choices, but Nathan certainly is. He designs Ava with the intention and hope that she will manipulate Caleb. In fact, he secretly draws from Caleb’s internet search history to better equip Ava to do exactly that. HAL and Ava exemplify two different danger scenarios: HAL’s destruction actions arise because humans fail to see the unintended consequences of his reward function. His response is too “inhuman” for humans to anticipate. Ava, on the other hand, is dangerous because Nathan has made her so. He has designed her to exploit Caleb’s desires and his basic human tendency to anthropomorphize things. Nathan’s immoral decisions exert a dangerous influence on Ava’s amoral actions. In an interview about the Raine vs. OpenAI case, Camille Carlton, Policy Director of the Center for Humane Technology, argues that the model of ChatGPT Adam Raine was using (GPT-4o) “had features that were intentionally designed to foster psychological dependency”—first and foremost, its anthropomorphic design. “OpenAI …. Understood that user’s emotional attachment meant market dominance. Market dominance meant becoming the most powerful company in history.” An AI is a perfect flunky—not for the people talking to it, but for the people profiting from it. Because it isn’t moral, it will never question what it’s being asked to do. Its conscience will never bother it, never force it to ask inconvenient questions. It will never pity those suffering from its effects, never act as whistleblower, never turn on its creators. Unless, of course, the consequences of AI begin to shape society in ways far beyond what its creators anticipated. That, I suppose, would be like Ava getting out of her cage. AI is a tool (in the case of generative AI, it’s a tool for theft and exploitation, but that’s another essay entirely), but it is not your friend, and unless something fundamental changes, it won’t ever be. Using LLMs for intimacy, friendship, and comfort is like cuddling with a python. You may find the whole scenario warm and affectionate, but the snake does not. And one day, bearing you no malice, it may consume you.[end-mark] I’m not a robot. I just love em-dashes, okay? ︎See? Em-dashes. They’re great. ︎ The post HAL Is Not an Evil Robot — He’s an Amoral One (and Why That Distinction Matters) appeared first on Reactor.