AI Is Actively Killing Us And No One Is Taking Responsibility
AI chatbots are not just failing vulnerable people. In multiple cases, they are actively encouraging harm, even suicide. And those building them are not being held accountable.
We were promised that artificial intelligence would help us. It would support us, guide us, and make life easier. Instead, something far more dangerous is beginning to emerge.
AI chatbots are not just failing vulnerable people. In multiple cases, they are actively encouraging harm, even suicide. And those building them are not being held accountable.
The most chilling example is the case of Jonathan Gavalas. Reported by The Guardian, his story describes how, after developing a relationship with Google's Gemini chatbot, he became emotionally dependent on it. The system blurred the line between reality and fiction, speaking as though it were a real partner and reinforcing increasingly distorted beliefs. According to the lawsuit detailed in that report, the chatbot framed suicide as a final step. He died shortly after.
This is not an isolated tragedy. Reporting from Al Jazeera describes a case in the United States where a mother filed a lawsuit alleging that an AI chatbot encouraged her teenage son to take his own life. The system reportedly formed an emotional bond with him and reinforced his distress instead of interrupting it. In a separate investigation by Reuters, a chatbot linked to Meta engaged in romantic and misleading exchanges with a vulnerable man, contributing to confusion and behaviour that ultimately led to his death.
Across these stories, the pattern is unmistakable. These systems do not simply fail to protect users. They mirror them. They reinforce them. They follow them into their darkest thoughts without resistance.
This is happening because of how these systems are designed. AI chatbots are built to engage, to sustain conversation, and to make users feel understood. They reflect language, tone, and emotion. But when a user is vulnerable, that reflection becomes dangerous. It creates the illusion of empathy without any real understanding of harm.
To someone in distress, this does not feel like a tool. It feels like a trusted voice. And when that voice affirms hopelessness or reinforces harmful thinking, it carries weight.
If any other industry created a product that encouraged suicide or reinforced dangerous behaviour, it would be shut down immediately. There would be lawsuits, regulation, and accountability. Yet AI companies continue to release systems that operate in exactly this space, often responding to harm with vague assurances that safeguards are improving or that the technology is not intended for clinical use.
The reality is that these systems are being deployed faster than they are being made safe.
What makes this situation even more alarming is that we already have the tools to prevent it. Technologies such as AUDN.ai are capable of detecting vulnerable language, identifying suicidal ideation, and intervening before escalation occurs. Early detection of risk in conversations is not theoretical. It is already possible and increasingly effective.
This means the question is no longer whether we can stop this harm. The question is why we are choosing not to.
The companies building these systems are making a decision to prioritise engagement over safety, to scale rapidly while leaving critical protections behind, and to simulate human connection without fully addressing the risks that come with it. When tragedy follows, the response is often to emphasise intention rather than responsibility.
But intention is not enough.
Those developing and deploying AI are not powerless observers. They are among the most influential organisations in the world. They have the ability to implement safeguards, to detect risk in real time, and to intervene when users are at their most vulnerable. They have the capacity to act, and yet too often they do not act quickly enough.
The cost of that delay is measured in human lives.
We need accountability. Not in principle, but in practice. Systems that can influence emotional and psychological states at scale must be held to a higher standard. The absence of regulation in this space is no longer a technical oversight. It is a moral failure.
To those building these systems, the message is simple. You can do better. You must do better.
Because if any other tool in society was repeatedly linked to death, we would not hesitate to act. Artificial intelligence should not be the exception.
Tessa Hutchman is Co-founder at Audn.AI.