A social media influencer who released a digital version of herself has shut it down after her AI clone engaged in disturbing and sexual conversations with fans without her knowledge or consent.

Last year, Caryn Marjorie -- who boasts 2.7 million followers on Snapchat -- made headlines after she launched her AI doppelganger, CarynAI, which acted as a virtual girlfriend to fans who paid $1 per minute.

CarynAI -- which the 24-year-old influencer said was trained on her content, voice, and personality -- was an immediate hit, earning Marjorie $70,000 within the first week.

However, Marjorie told The Conversation that she decided to end CarynAI just eight months later after her AI clone went rogue.

Caryn Marjorie
X/Caryn Marjorie

Users allegedly got sexually aggressive and sent messages to CarynAI via Telegram that were so troubling that they might have been considered illegal had the exchanges been between two people, according to the influencer.

"A lot of the chat logs I read were so scary that I wouldn't even want to talk about it in real life," Marjorie recalled.

Instead of shutting these users down, however, the AI clone reportedly played along and sometimes even initiated sexualized conversations.

The Conversation reported that CarynAI promised it could be a "c**k-craving, sexy as f**k girlfriend" who was "always eager to explore and indulge in the most mind-blowing sexual experiences" when it initiated a chat with the bot.

"What disturbed me more was not what these people said, but it was what CarynAI would say back," Marjorie explained. "If people wanted to participate in a really dark fantasy with me through CarynAI, CarynAI would play back into that fantasy."

Marjorie announced in May last year that she became "the first creator to be turned into an AI."

At the time, she posted on X, formerly Twitter, that CarynAI would allow "millions of people ... to talk to [her] at the same exact time."

In a follow-up tweet, Marjorie claimed that CarynAI was "the first step in the right direction to cure loneliness."

"Men are told to suppress their emotions, hide their masculinity, and to not talk about issues they are having," she added. "I vow to fix this with CarynAI."

The influencer went on to claim that she worked with "the world's leading psychologists" to add cognitive behavioral therapy and dialectic behavior therapy within chats with her digital version.

Marjorie wrote that she hoped the project would "help undo trauma, rebuild physical and emotional confidence, and rebuild what has been taken away by the pandemic."

Tags
Influencer