Hyper-realistic AI technology creates avatars from a single photo

Electronics and Telecommunications Research Institute (ETRI) has developed hyper-realistic AI technology that can create an avatar that speaks naturally like a real person using only a single portrait photo.

The technology is being seen as a next-generation interface that enables intuitive interaction between vehicles and humans in preparation for the era of fully autonomous driving, and is expected to spread across the digital human industry.

While traditional current speech-driven AI assistants in office environments or navigation systems in vehicles are limited to simply carrying out commands, ETRI’s hyper-realistic AI avatars have sophisticated facial expressions and mouth movements that enable natural, human-like conversations.

This allows for a more human-centered human-machine interaction, such as an in-vehicle AI driver talking to the driver or interacting with pedestrians.

The core of this technology is a unique algorithm that, unlike traditional generative AI, selectively learns and synthesizes parts of the face that are directly related to utterance, such as the lips and chin. This approach reduces unnecessary information learning and allows for more sophisticated facial expressions, including mouth shapes, teeth, and skin wrinkles.

ETRI explained that the technology has demonstrated superior performance in terms of synthetic visual quality and lip synchronization accuracy as presented at major international conferences such as CVPR and AAAI.

In addition to autonomous vehicles, this technology can be utilized in various industries such as kiosks, bank counters, news presentations, advertising models, and is expected to drive innovation in the AI-based digital human industry.

ETRI’s Mobility User Experience Research Section is currently focusing on human-machine interaction (HMI) technologies, and is also developing AI-based driver interface technologies that analyze driver and pedestrian emotions, fatigue, concentration, etc.

Daesub Yoon, Director of the Mobility User Experience Research Section, said, “As mobility technology becomes more advanced, the elderly and socially disadvantaged may be marginalized. We hope that this AI avatar technology will contribute to improving digital literacy and make smart mobility services more accessible to all.”

And Senior Researcher Daewoong Choi also said, “We plan to further advance our generative AI technology so that AI avatars can naturally talk and move like real people. In the future, we’re aiming for interactions that can replace some human labor for ordering, consulting, and more.”

The technology is currently registered on the ETRI Technology Transfer site as “A Framework for Photorealistic Talking Face Generation.” The researchers will also actively pursue technology transfer and strategies for commercialization in various industries.