Concerns have emerged regarding biases present in Apple’s Image Playground application, following claims made by a machine learning expert. Jochem Gietema, who heads the Machine Learning Science team at Onfido, published a detailed blog post that includes various outputs produced by the app, asserting that it inaccurately represented skin tones and hair textures on several occasions, often correlating these inaccuracies with certain racial stereotypes. It remains unclear whether these instances are isolated or indicative of a larger pattern. The app was integrated into the Apple Intelligence suite with the release of iOS 18.2.
Potential Bias Issues in Apple’s Image Playground App
In his blog, Gietema pointed out different outputs generated by the app, highlighting instances of racial bias inherent in the large language model that powers it. Interestingly, staff members at Gadgets 360 did not observe similar biases during their testing of the application.
“While experimenting, I noticed that the app changed my skin tone and hair based on the prompts provided. Different professions, such as investment banker versus farmer, resulted in images with varying skin tones. Similar disparities appeared in contexts like skiing versus basketball, street fashion versus formal attire, and, most concerning, affluent versus impoverished portrayals,” Gietema noted in a LinkedIn post.
Alleged biased outputs generated using the Image Playground app
Photo Credit: Jochem Gietema
These kinds of inaccuracies are not uncommon in large language models (LLMs), which are trained on extensive datasets that may contain similar stereotypical representations. For instance, Google’s Gemini AI model faced significant criticism last year for analogous issues. Nevertheless, companies typically implement various safeguards to mitigate such generations.
Apple’s Image Playground app incorporates certain restrictions aimed at reducing the risks associated with AI-generated outputs. It currently supports only cartoon and illustration styles in order to avert the creation of deepfakes. Furthermore, the app restricts the generated images to a narrow field of vision, typically capturing just the face and minimal additional details. This design choice also aims to limit instances of bias and inaccuracies.
Additionally, the tech company prohibits prompts containing negative terms, celebrity names, or public figures, with the intention of preventing misuse of the application. However, if the allegations are substantiated, Apple might need to enhance its safety measures to ensure that users do not encounter discriminatory experiences while utilizing the app.
