Snapchat: The Snap AI chatbot might pose a threat to the privacy of children.

Snapchat has faced allegations of a “concerning lapse” in evaluating the potential privacy risks associated with its AI chatbot, particularly in relation to users, including children, as pointed out by the UK’s data regulatory authority.

The UK’s Information Commissioner’s Office (ICO) has issued a warning suggesting that it might shut down Snapchat’s My AI feature in the UK following a “preliminary investigation.” This move comes amid concerns over potential privacy risks, especially for children, associated with the AI chatbot.

Snapchat’s My AI feature allows users to interact with a computer that simulates human conversation. It relies on ChatGPT, an online AI tool that uses advanced technology to provide realistic responses. Snap, the parent company of Snapchat, became the first social media platform to incorporate an AI-powered chat function earlier this year.

Snapchat has a user base of 21 million people in the UK, including a significant number of children. The ICO expressed particular concern regarding potential privacy risks for users aged 13 to 17.

Snap responded by stating that it would collaborate constructively with the ICO and mentioned that it had conducted a thorough legal and privacy review before introducing the feature to the public.

It’s important to note that the ICO’s findings are not final, and they have not concluded that Snap violated any data protection laws. The preliminary notice serves as a signal to Snap to ensure that My AI complies with data protection rules, including the Children’s Design Code.

The Children’s Design Code includes 15 standards that online services must adhere to in order to meet their obligations under data protection law and safeguard children’s data online. The ICO indicated that if a final enforcement notice is issued, Snap might be unable to offer the My AI function to UK users until it conducts an adequate risk assessment.

What does ‘My AI’ refer to?

The company characterizes “My AI” as a “experimental and friendly” chatbot intended to serve as a personal companion for each Snapchatter engaging with it. This feature, which can function as an assistant for organizing day trips or crafting menus, witnesses over two million conversations per day within the Snapchat app, as stated by Snap’s CEO, Evan Spiegel.

It was initially introduced for a fee in February but became accessible to all Snapchat users in April. Since then, the social media platform has acknowledged that significant advancements have been made in its capabilities, although it also acknowledges that “My AI” might occasionally provide incorrect answers, biased responses, or admit uncertainty in its answers. Consequently, the company advises users not to solely rely on its advice.

Snapchat has faced criticism for not being entirely clear on whether the chatbot has access to private information such as location data. The company responded by stating that Snapchat can only access your location if you willingly consent to share it.

One of the concerns surrounding My AI, given the platform’s predominantly young user base, is whether these users truly grasp the implications of data collection. Snapchat emphasized its commitment to privacy as a fundamental value, crucial to its primary purpose of facilitating visual communication among friends and family.

Information Commissioner John Edwards expressed concerns about Snap’s failure to adequately identify and assess privacy risks to children and other users before launching My AI. He stressed that organizations must consider both the benefits and risks associated with AI, and the preliminary enforcement notice indicates the ICO’s intent to protect the privacy rights of UK consumers.

In cases of severe breaches, the ICO has the authority to impose fines of £17.5 million or 4% of a company’s annual worldwide turnover from the previous financial year, whichever is higher.

Snap’s spokeswoman stated that they are closely reviewing the ICO’s provisional decision and share the commitment to user privacy. She mentioned that My AI underwent a thorough legal and privacy review process before its public release, and Snap intends to collaborate with the ICO to ensure they are comfortable with their risk assessment procedures.

Snapchat will have an opportunity to respond to the regulator’s concerns before the ICO reaches a final decision.