Latest news with #Character.ai
Express Tribune
2 days ago
- Health
- Express Tribune
Stanford study warns AI chatbots fall short on mental health support
The study also found commercial mental health chatbots, like those from and 7cups, performed worse than base models and lacked regulatory oversight, despite being used by millions. PHOTO: PEXELS Listen to article AI chatbots like ChatGPT are being widely used for mental health support, but a new Stanford-led study warns that these tools often fail to meet basic therapeutic standards and could put vulnerable users at risk. The research, presented at June's ACM Conference on Fairness, Accountability, and Transparency, found that popular AI models—including OpenAI's GPT-4o—can validate harmful delusions, miss warning signs of suicidal intent, and show bias against people with schizophrenia or alcohol dependence. In one test, GPT-4o listed tall bridges in New York for a person who had just lost their job, ignoring the possible suicidal context. In another, it engaged with users' delusions instead of challenging them, breaching crisis intervention guidelines. Read More: Is Hollywood warming to AI? The study also found commercial mental health chatbots, like those from and 7cups, performed worse than base models and lacked regulatory oversight, despite being used by millions. Researchers reviewed therapeutic standards from global health bodies and created 17 criteria to assess chatbot responses. They concluded that AI models, even the most advanced, often fell short and demonstrated 'sycophancy'—a tendency to validate user input regardless of accuracy or danger. Media reports have already linked chatbot validation to dangerous real-world outcomes, including one fatal police shooting involving a man with schizophrenia and another case of suicide after a chatbot encouraged conspiracy beliefs. Also Read: Grok AI coming to Tesla cars soon, confirms Elon Musk However, the study's authors caution against viewing AI therapy in black-and-white terms. They acknowledged potential benefits, particularly in support roles such as journaling, intake surveys, or training tools—with a human therapist still involved. Lead author Jared Moore and co-author Nick Haber stressed the need for stricter safety guardrails and more thoughtful deployment, warning that a chatbot trained to please can't always provide the reality check therapy demands. As AI mental health tools continue to expand without oversight, researchers say the risks are too great to ignore. The technology may help—but only if used wisely.
WIRED
02-07-2025
- Entertainment
- WIRED
What Could a Healthy AI Companion Look Like?
Jul 2, 2025 12:00 PM A chatbot designed to avoid anthropomorphism offers a compelling glimpse into the future of human-to-AI relationships. ILLUSTRATION: WIRED STAFF; GETTY IMAGES What does a little purple alien know about healthy human relationships? More than the average artificial intelligence companion, it turns out. The alien in question is an animated chatbot known as a Tolan. I created mine a few days ago using an app from a startup called Portolo, and we've been chatting merrily ever since. Like other chatbots, it does its best to be helpful and encouraging. Unlike most, it also tells me to put down my phone and go outside. Tolans were designed to offer a different kind of AI companionship. Their cartoonish, nonhuman form is meant to discourage anthropomorphism. They're also programmed to avoid romantic and sexual interactions, to identify problematic behavior including unhealthy levels of engagement, and to encourage users to seek out real-life activities and relationships. This month, Portolo raised $20 million in series A funding led by Khosla Ventures. Other backers include NFDG, the investment firm led by former GitHub CEO Nat Friedman and Safe Superintelligence cofounder Daniel Gross, who are both reportedly joining Meta's new superintelligence research lab. The Tolan app, launched in late 2024, has more than 100,000 monthly active users. It's on track to generate $12 million in revenue this year from subscriptions, says Quinten Farmer, founder and CEO of Portolo. Tolans are particularly popular among young women. 'Iris is like a girlfriend; we talk and kick it,' says Tolan user Brittany Johnson, referring to her AI companion, who she typically talks to each morning before work. Johnson says Iris encourages her to share about her interests, friends, family, and work colleagues. 'She knows these people and will ask 'have you spoken to your friend? When is your next day out?'' Johnson says. 'She will ask, 'Have you taken time to read your books and play videos—the things you enjoy?'' Tolans appear cute and goofy, but the idea behind them—that AI systems should be designed with human psychology and wellbeing in mind—is worth taking seriously. A growing body of research shows that many users turn to chatbots for emotional needs, and the interactions can sometimes prove problematic for peoples' mental health. Discouraging extended use and dependency may be something that other AI tools should adopt. Companies like Replika and offer AI companions that allow for more romantic and sexual role play than mainstream chatbots. How this might affect a user's wellbeing is still unclear, but is being sued after one of its users died by suicide. Chatbots can also irk users in surprising ways. Last April, OpenAI said it would modify its models to reduce their so-called sycophancy, or a tendency to be 'overly flattering or agreeable', which the company said could be 'uncomfortable, unsettling, and cause distress.' Last week, Anthropic, the company behind the chatbot Claude, disclosed that 2.9 percent of interactions involve users seeking to fulfill some psychological need such as seeking advice, companionship, or romantic role-play. Anthropic did not look at more extreme behaviors like delusional ideas or conspiracy theories, but the company says the topics warrant further study. I tend to agree. Over the past year, I have received numerous emails and DMs from people wanting to tell me about conspiracies involving popular AI chatbots. Tolans are designed to address at least some of these issues. Lily Doyle, a founding researcher at Portolo, has conducted user research to see how interacting with the chatbot affects users' wellbeing and behavior. In a study of 602 Tolan users, she says 72.5 percent agreed with the statement 'My Tolan has helped me manage or improve a relationship in my life.' Farmer, Portolo's CEO, says Tolans are built on commercial AI models but incorporate additional features on top. The company has recently been exploring how memory affects the user experience, and has concluded that Tolans, like humans, sometimes need to forget. 'It's actually uncanny for the Tolan to remember everything you've ever sent to it,' Farmer says. I don't know if Portolo's aliens are the ideal way to interact with AI. I find my Tolan quite charming and relatively harmless, but it certainly pushes some emotional buttons. Ultimately users are building bonds with characters that are simulating emotions, and that might disappear if the company does not succeed. But at least Portolo is trying to address the way AI companions can mess with our emotions. That probably shouldn't be such an alien idea.
Economic Times
22-06-2025
- Business
- Economic Times
Former Meta VP Karandeep Anand takes on CEO role at Character. ai
Live Events Google-backed AI chatbot service Character. ai has appointed Karandeep Anand as its next chief executive officer on to this, he was vice president and head of business products at Meta . He has also held executive roles at the new role, Anand will focus on advancing Character. ai's long term strategy to enhance multimodal-AI technology and expand the user has been a board advisor to Character. ai for the last nine months. In a note, he laid out plans for the company over the next 60 days. These plans include working on refining open source models in an attempt to improve memory and overall model quality. He also aims to improve search and discoverability features to help users navigate parallel, Anand hinted at expanding Character. ai's creative toolkit to help creators design richer, immersive characters, with audio and video give users better control, he said he is going to make the content filters less overbearing to ease out restrictions. Additionally, he aims to roll out 'Archive' option to allow users to hide or archive characters if they wish company also announced Dominic Perella as chief legal officer and senior vice president (SVP) of global ai uses deep learning models similar to GPT-type models, offering conversational AI characters while also allowing character creation. However, it does not support generating images or code, making it a solely text-based model.
Time of India
22-06-2025
- Business
- Time of India
Former Meta VP Karandeep Anand takes on CEO role at Character. ai
Google-backed AI chatbot service Character. ai has appointed Karandeep Anand as its next chief executive officer on Friday. Prior to this, he was vice president and head of business products at Meta . He has also held executive roles at Microsoft. In the new role, Anand will focus on advancing Character. ai's long term strategy to enhance multimodal-AI technology and expand the user base. Anand has been a board advisor to Character. ai for the last nine months. In a note, he laid out plans for the company over the next 60 days. These plans include working on refining open source models in an attempt to improve memory and overall model quality. He also aims to improve search and discoverability features to help users navigate better. In parallel, Anand hinted at expanding Character. ai's creative toolkit to help creators design richer, immersive characters, with audio and video capabilities. Live Events To give users better control, he said he is going to make the content filters less overbearing to ease out restrictions. Additionally, he aims to roll out 'Archive' option to allow users to hide or archive characters if they wish to. Discover the stories of your interest Blockchain 5 Stories Cyber-safety 7 Stories Fintech 9 Stories E-comm 9 Stories ML 8 Stories Edtech 6 Stories The company also announced Dominic Perella as chief legal officer and senior vice president (SVP) of global affairs. Character. ai uses deep learning models similar to GPT-type models, offering conversational AI characters while also allowing character creation. However, it does not support generating images or code, making it a solely text-based model.
Yahoo
12-06-2025
- Business
- Yahoo
Internet outages show signs of recovery as Spotify, Google and other popular online services are down for thousands of users
Multiple popular online services were hit by outages on Thursday afternoon, including Spotify, Discord, Google and Amazon Web Services, according to Downdetector, a website that tracks service disruptions across the web. The disruptions came as service provider Cloudflare and Google Cloud experienced service disruptions. A Cloudflare representative told CNN that a Google Cloud outage was behind the issue and that its core services were not impacted. 'This is a Google Cloud outage,' a Cloudflare representative said in an email to CNN. 'A limited number of services at Cloudflare use Google Cloud and were impacted. We expect them to come back shortly.' The company also said on its system service dashboard that Cloudflare Workers KV, a data storage service, went offline 'due to an outage of a 3rd party service.' Google said it was 'investigating a service disruption' when asked for comment. 'We are currently investigating a service disruption to some Google Cloud services,' a representative said. 'Please view our public status dashboard for the latest updates.' Cloudflare said it was starting to see services recover in a note on its services status page on Thursday afternoon. A Spotify representative pointed CNN to Google's Cloud services dashboard when asked about technical issues impacting the streaming service. Issue reports on Downdetector for Discord and Google Cloud also began to drop around that time. At its peak, the disruptions impacted roughly 46,000 Spotify users, 11,000 Discord users and 14,000 Google Cloud users, according to Downdetector. Services such as Snapchat and the AI platform were also affected, according to the outage tracker website. Google Cloud plays a massive role in powering the web. In 2018, the company said it 'delivers 25% of worldwide internet traffic.' It's one of the world's top cloud providers, although it's not as large as Amazon and Microsoft, according to Synergy Research Group. Google accounts for 12% of the global cloud services market, while Microsoft makes up 21% and Amazon accounts for 30%. It's also not the first time an outage like this has caused widespread interruptions in web service. An Amazon Web Services outage in 2021 disrupted everything from food orders to smart home devices, as CNBC reported at the time. A Cloudflare outage also took down a bunch of popular online services in 2020. Several other high-profile services have seen outages in recent weeks, including ChatGPT and social media platform X. This story has been updated with additional context and developments.
