Snapchat’s My AI feature is a chatbot that can converse with users and give recommendations

The bot is powered by OpenAI’s ChatGPT, a generative AI model that can create realistic text and images

[{"selector":"#anim-e22e3bc5-9ae6-4439-a22b-ecbb636226f0","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-4355b15a-ac81-45ce-aec8-5f49b9a18122","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-10c9cfec-a6aa-4220-86d3-e8407359a343 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

On Aug. 15, the bot posted a story of its own, showing a flat image with two tones

[{"selector":"#anim-f37e3c08-4802-481b-af21-1f18d9a33b24 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-93ee6dd0-5a27-4287-8476-7d8af5ba9c20","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-186815b5-5cb6-4419-8416-32a9bbc7fafc","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Many users thought the image was a photo of their home ceilings and questioned the bot about their privacy

[{"selector":"#anim-92a458f2-6e5d-4f1d-a289-7ad663feebb2 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-e9530f94-fc63-46f9-a3ad-d1b9ec4394ea","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-9c95437a-c5f4-48fc-81be-32cde74d9e46","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

The bot only replied with “Sorry, I encountered a technical issue” and stopped responding

[{"selector":"#anim-f05b85de-5d7d-4e9b-bb01-847d62f9bee5 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-9ad5dc83-ff9f-43f7-9ddf-c32fd4c870ce","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-3069e6d2-15a1-4e13-897b-3a25a4bc2dfb","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Snap officials said the incident was a glitch and not an example of the bot working on its own

[{"selector":"#anim-b659e167-ddf6-4271-a85e-4599e8e9cd2e [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-b16c02a6-4de8-4cc7-a711-eadf2bcf4bd9","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-0c83f121-de4f-40b0-a805-8dbf4b6b9485","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

The incident shows how AI can sometimes hallucinate and produce aberrant information or behavior

[{"selector":"#anim-647edcc4-c647-4920-94df-45dcd74a4f8c [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-2b876c13-9ada-44ca-b64a-142a57fe1cd1","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-348699a4-6570-4f16-a850-adafdafe5cbe","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

This unpredictability poses a problem for businesses using generative AI, as Snap found out this week

[{"selector":"#anim-97c867f5-9a46-49b5-be8a-5fdc3d39e45b [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-fb7636cf-b97c-4b73-b03c-9de10947758b","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-a2d965be-f27d-4e79-a45a-3bc81175d2e4","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Snap has a history of breaching data protection rights and exposing user details and content

[{"selector":"#anim-ae474d56-6be2-4280-8b02-9189a8dec3fe [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-2c848f91-7116-445b-bb05-d8ebcbbee1af","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-43ef5133-d87e-46bb-b913-d4f2a15eb182","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

The My AI feature has already sparked safety concerns for its inappropriate responses and lack of options to remove or disable it

[{"selector":"#anim-927686f4-6315-4518-bbfe-d8587c2d0603 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(0.017856839567439983%, 0, 0) translate(25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-76ba7a7d-f78f-4800-96d7-d10b54210a1c","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-87b4232a-4cc3-4bfe-9c12-30dc3c43c527","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]