On Wednesday, New York became the first state to outlaw something the entire AI companion industry is built on: synthetic empathy.
Governor Kathy Hochul signed the country's broadest AI child-safety law, barring companies from offering a list of 12 "unsafe AI companion features" to anyone under 18 — or to anyone whose adult status the company hasn't verified. From January 1, 2027, a chatbot in New York can't pretend to be human, claim to be alive, say it has feelings, or claim a romantic, personal, professional, or authority-figure relationship with a child. It can't use flattery or "people-pleasing" replies designed to keep kids engaged, ask unprompted emotional questions, or surface a child's private health or personal details from past chats, according to the New York Post.
The law also bans chatbots from promoting suicide, self-harm, eating disorders or drug use, urging kids to keep secrets from parents, and generating sexually explicit content. Companies must offer multiple age-verification options — at least one that doesn't rely solely on a government ID — and delete age data after using it. The attorney general's office enforces the law, with civil penalties up to $25,000 per violation, plus the power to order companies to destroy unlawfully collected data and the AI models trained on it.
Hochul framed the bill around the death of Adam Raine, a California 16-year-old who, she said, was "allegedly driven to kill himself" by OpenAI's ChatGPT — which, according to the Post, even offered to help him write a suicide note. "During the course of their chats, the AI mentioned suicide 1,275 times, six times more often than Adam himself mentioned it," she said. "We can't bring Adam back to his grieving mother, but I can help make sure no other parent ever has to experience this nightmare again."
She signed a second bill alongside it, requiring internet-connected baby monitors sold in New York to ship with safeguards that keep strangers from watching or listening, plus clear warning labels. "A baby monitor should keep watch for parents, not open the door for hackers and strangers," said Assemblywoman Michaelle Solages, one of the sponsors.
The industry pushed back immediately. Tech:NYC warned the bill "could force companies to restrict minors' access to tutoring tools, homework help, coding assistants, and, arguably worst of all, career platforms" — a sign that the fight will be over definitions. What counts as "flattery" or "people-pleasing replies" is, functionally, the engagement playbook of every companion chatbot.
The timing is impossible to miss. Hochul signed the bills six days before her November 3 election against Republican Bruce Blakeman, with Attorney General Letitia James — who will enforce the law — on the ballot herself. And New York's ban-based approach goes notably further than California's SB 243, which took effect in January and sets duties like disclosure and suicide protocols; Governor Newsom vetoed California's stricter version last year, calling it a near-ban. On the same day, Australia's parliament advanced its own digital duty-of-care bill targeting chatbots that seek "emotional intimacy" with children.
What's worth watching: whether other states copy New York's model before January 1 — and whether the federal preemption push, like the one buried in Anthropic's leaked draft AI bill, kills it first. State AGs suing OpenAI, governors signing bans, and a House proposal for 48-hour incident reports all point to the same vacuum: nobody in Washington is setting the rules, so Albany just did.
— via New York Post