Apple and Google face pressure to pull X and Grok over AI deepfake concerns

Grok AI

• Senators urge Apple, Google to remove X and Grok apps for illegal content
• Lawmakers cite deepfake sexual images and racist content generated by Grok and X
• Apple and Google ban child abuse and explicit content in app stores.

Did our AI summary help?

Three Democratic senators have called on Apple and Google to remove the X and Grok apps from their respective app stores unless Elon Musk addresses what they describe as serious and potentially illegal content issues on the platforms.

In an open letter sent Friday to Apple CEO Tim Cook and Google CEO Sundar Pichai, Senators Ron Wyden, Ed Markey and Ben Ray Luján urged the companies to act immediately, arguing that allowing the apps to remain available undermines claims that curated app stores offer safer user experiences than sideloaded software.

The senators cited widespread reports that Grok and X have enabled users to generate and share nonconsensual sexual images, including deepfake-style explicit content and depictions involving minors. They also pointed to examples of racially and ethnically degrading imagery produced using the AI system.

One incident highlighted involved the generation of an explicit image of a descendant of Holocaust survivors, created through prompts that placed her in a sexualised context at the Auschwitz concentration camp site. The case has fuelled criticism from civil society groups and lawmakers alike.

While several international regulators in Europe, Australia, Malaysia and India have reportedly opened probes into xAI and X, US agencies such as the Federal Trade Commission and Department of Justice have not yet announced investigations.

Musk and X said earlier this month that users who prompt Grok to create illegal content would face consequences similar to those for uploading illegal material. However, critics argue safeguards remain insufficient. Even after X restricted Grok’s image generation features to paying subscribers, the standalone Grok app and website reportedly continued to allow users to create explicit content without consent.

The letter also emphasised that both Apple and Google have strict developer policies that prohibit child sexual abuse material and harmful explicit content. In the past, platforms such as Tumblr and Telegram have faced temporary suspensions from app stores over similar moderation failures.

The renewed scrutiny comes as xAI announced a $20 billion funding round this week, backed by investors including Nvidia, Cisco Investments, and several major institutional and sovereign funds. Despite the capital infusion, lawmakers say investor confidence should not override basic safety and compliance standards.