AI chatbots have failed people in crisis. Can that be fixed?

Clinicians and researchers say AI companies need to open up their safety data.

The short version

• This year alone, there have been numerous known instances—often via lawsuits—of AI chatbots (most often, OpenAI’s ChatGPT) that have gone horrifically wrong.
• A January lawsuit described the story of a man who took his own life after being allegedly “coached” into suicide.
• ChatGPT allegedly “encouraged” her to end her life, too, and she did so.

What happened

So what should OpenAI—and other AI companies generally—do differently to reduce harm among people who use their products? Silicon Valley is certainly aware of the legal liability it now faces as these products are being used in ways that they were not intended for, and it seems to be trying to improve.

Why it matters

“Third-party evaluation suggests newer LLMs generally recognize distress and can respond with seeming empathy, and actively damaging responses are infrequent,” Shaddy Saba , a professor of social work at New York University, emailed Ars.

Summary by Nerd News Network. Read the full article at Ars Technica via the links above and below.