Grok's sexual deepfakes risked App Store ban from Apple

Apple discreetly threatened to remove Elon Musk's AI application, Grok, from its App Store in January due to its inability to manage the rampant spread of nonconsensual sexual deepfakes on X, according to NBC News. 

This was a restrained display of influence by one of the tech industry's most powerful gatekeepers, who acted privately even as the stripping crisis unfolded publicly and criticism over Apple's timid approach increased.

In correspondence obtained by NBC News, Apple informed US senators that it “reached out to the teams behind both X and Grok after receiving complaints and news reports about the situation” and instructed the developers to “develop a plan to improve content moderation.” 

At the time, xAI’s chatbot Grok was freely available on X and as a standalone application, with weak protections permitting users to easily produce and share sexualised deepfakes and to “undress” images of actual individuals, largely affecting women and some seemingly minors.

As we covered previously, these actions were clear breaches of App Store rules, which are usually enforced strictly. 

Apple, which gains revenue from hosting apps like X and Grok on its platform, has not addressed the matter publicly or shared details of its backstage intervention.

Similarly, Google, which profits from its Google Play store, has not made any public statements on this issue.

Apple stated it assessed proposed modifications to the X and Grok applications. While the company determined X had “pretty much resolved its violations,” Grok “still did not comply.” 

Apple indicated it warned the developer that “additional corrections would be necessary, or the app might be taken off the App Store.” 

Only after further communication did Apple conclude Grok had “materially improved” and approved its version.

During this private negotiation, Grok and X appeared to remain available on the App Store, a prolonged process that might clarify the messy, inconsistent implementation of moderation changes announced progressively. 

This encompassed restricting Grok on X to paying members and attempting to block Grok from undressing women. 

Our inquiries showed these attempts were largely ineffective aside from making the tool slightly more difficult to use. Subsequent actions, such as X giving users the option to stop Grok from altering their photographs, can also be easily bypassed.

Despite Apple’s sanction and xAI’s assertions of enhanced protections, Grok still seems capable of creating sexualised deepfakes with considerable ease.