Apple and Google have received cease-and-desist letters demanding the removal of applications that facilitate the creation of “undressing” images, as revealed by WIRED. These legal notices call for the companies to cease profiting from harmful technology.
On Thursday, San Francisco City Attorney David Chiu issued legal notices to Apple and Google, urging them to remove 13 face-swapping applications capable of generating non-consensual AI-generated nude images from their app stores. The letter emphasizes that major Silicon Valley companies must stop “aiding and abetting” the sale of explicit deepfake images and must “sever” ties with app developers who create them.
“Producing non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu stated in an interview with WIRED. Previously, he filed lawsuits against 16 popular deepfake websites, emphasizing that Apple and Google likely “earn millions of dollars in fees” from apps featuring nudity. He called on both companies to enhance their moderation processes to prevent such applications from appearing in the first place.
“These companies have a responsibility to ensure that the apps on their platforms do not facilitate sexual abuse,” Chiu remarked. The state’s legal letter confirms that this practice is prohibited under California law. For further reference, see support services related to the creation of deepfake pornography. The letter highlights that these apps utilize in-app payments, from which the tech giants also profit. “The fact that some of the world’s largest and most established technology companies are facilitating this must come to an end.”
Researchers have consistently found and reported that apps available in Apple’s App Store and Google’s Play Store are capable of generating sexual images, including some labeled as suitable for children. The push for new regulations aims to tackle the proliferation of deepfakes online, where tech and social media companies continuously direct millions toward harmful technologies.
Both Apple and Google maintain developer policies that forbid pornography, abuse, and harassment on their platforms. Both companies have removed numerous nudify and deepfake apps following reports from researchers and journalists.
A spokesperson for Google, Dan Jackson, informed WIRED that the company has eliminated “hundreds” of apps that violated their policies, including five Android apps flagged by Chiu’s office. Additional steps have been taken to limit access to these applications.
“Google Play does not permit apps containing sexual content, and we actively take measures to detect and eliminate harmful content,” Jackson stated. “When we receive reports of violations, we investigate promptly and take decisive action, which has included suspending hundreds of non-compliant apps and restricting related search terms like ‘nudity’ in the Store.”
Apple did not provide any comments prior to publication.
In the past five years, the emergence of lucrative deepfake “nude” technology has gained traction online. For example, in January, xAI’s Grok was utilized to generate millions of sexual images. Countless apps, websites, and bots enable users (predominantly men) to upload images of individuals (mostly women and girls) and have them digitally “undressed” or placed in explicit scenarios.
Creating a sexual deepfake typically necessitates only a reference photo and a few clicks, with results achievable in mere seconds. As generative AI technology advances, the realism of images and videos has improved; some services offer results for free, while others charge a nominal fee for producing harmful content. Previous investigations by WIRED and Indicator Media indicated that over 90 schools fell victim to the creation of deepfake sexual abuse images targeting minors.
Source: www.wired.com