Sony recently stirred controversy with a leaked internal video showing an AI-powered version of Aloy, the protagonist from the Horizon game series. The prototype, developed by Sony’s PlayStation Studios Advanced Technology Group in collaboration with Guerrilla Games, demonstrates AI technology that enables players to have real-time conversations with the character.
The AI Aloy uses OpenAI’s Whisper for speech-to-text conversion, GPT-4 and Llama 3 for decision-making, Sony’s Emotional Voice Synthesis for speech generation, and Mockingbird technology for facial animation. In the demonstration, Aloy responds to questions about herself and her origins, then enters the game world and responds to questions as she moves around.
Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment, narrated the video, clarifying that this is just a prototype. He mentioned that while the demo runs on PC, Sony has also experimented with running parts of this technology directly on PS5 consoles with “little overhead.”
Voice Actor’s Response
Ashly Burch, the voice actor behind Aloy, expressed concerns after seeing the tech demo. She noted that Guerrilla Games contacted her to explain that the demo didn’t reflect anything actively in development and didn’t use any of her performance data.
“I feel worried about this art form. Game performance as an art form,” Burch stated in an Instagram video. As a participant in the ongoing SAG-AFTRA strike against the video game industry, Burch emphasized three key protections actors are fighting for:
- Mandatory consent before creating AI versions of actors
- Fair compensation for AI usage
- Transparency about how AI doubles are used
Similar Posts
Ethical Concerns
The gaming community has noted the irony that Aloy, a character from a series about humanity battling the consequences of rogue AI, is being used in an AI prototype. The leaked demonstration comes at a time when AI’s role in creative industries is under intense scrutiny.
According to a Game Developers Conference 2024 survey, 49% of 3,000 respondents said generative AI tools are currently being used in their workplace, with 31% personally using them. This widespread adoption raises questions about job security in an industry already facing significant layoffs.
Sony isn’t alone in exploring AI characters in games. Nvidia has been developing similar technology called Ace for AI-powered NPCs, while Microsoft has partnered with Inworld AI to bring AI characters to Xbox and created its own Muse AI model for generating gameplay environments.
What Happens Next?
After The Verge published their story about the leaked video, it was removed from YouTube due to a copyright claim from Muso, a copyright enforcement company that lists Sony Interactive Entertainment as a client.
Sony has not officially commented on the leaked internal video, leaving questions about whether such technology will eventually make its way into commercial games.
As the gaming industry gathers for GDC in San Francisco, this revelation will likely fuel further discussion about AI’s role in game development and its potential impact on actors, developers, and the creative process.
Frequently Asked Questions
What technology is Sony using for the AI Aloy prototype? +
Sony’s AI Aloy prototype uses several technologies: OpenAI’s Whisper for speech-to-text conversion, GPT-4 and Llama 3 for conversations and decision-making, Sony’s Emotional Voice Synthesis (EVS) for speech generation, and Sony’s Mockingbird technology for facial animation.
Was Ashly Burch’s voice or performance used in the AI Aloy demo? +
No. According to Ashly Burch, Guerrilla Games informed her that none of her facial or voice data was used for the demo. The prototype used AI-generated voice and animations.
What concerns did Ashly Burch express about the AI prototype? +
Burch expressed concern about “game performance as an art form.” As part of the SAG-AFTRA strike, she emphasized the need for three key protections: mandatory consent before creating AI versions of actors, fair compensation for AI usage, and transparency about how AI doubles are used.
Is this AI Aloy technology going to be used in future PlayStation games? +
Sony has not officially commented on whether this technology will be implemented in commercial games. Sharwin Raghoebardajal, who narrated the demonstration, described it as “just a prototype” and “a glimpse of what is possible,” suggesting it’s still in experimental stages.
Are other gaming companies working on similar AI character technology? +
Yes. Nvidia has been developing technology called Ace for AI-powered NPCs in games. Microsoft has partnered with Inworld AI to bring AI characters to Xbox and has created its own Muse AI model for generating gameplay environments for prototyping.
How common is AI usage in game development currently? +
According to a Game Developers Conference 2024 survey of 3,000 respondents, 49% reported that generative AI tools are currently being used in their workplace, with 31% saying they personally use these tools in their work.