Some filmmakers are using an AI tool called TrueSync by Flawless AI to lipsync the English dub of their film to the original footage.

A bit more about the tech:
https://variety.com/2025/artisans/ne...ng-1236383810/
As that article notes, the first film released using this tech is the Swedish language film "UFO Sweden" was retitled as "Watch the Skies," dubbed into English by the original actors, and screening at AMC theaters in the US starting tomorrow, May 9, 2025.

The trailer doesn't really show it off too well. It's a lot of voiceover, but the little you see of people talking, it seems to match:


One reviewer thinks the lipsync works:
https://www.slashfilm.com/1854900/wa...-watch-movies/

A making-of video covering the tech, with longer clips for examples/comparisons, and it shows they also did visual translation of the text in the film too:



One side effect appears to be the original Swedish language version isn't being screened in theaters. One wonders about the eventual streaming and disc releases, if the original language with English subtitles will be an option.

Did they use AI to recreate Martin Mull?

I've always thought at some point they'll use this technique to make looping easier. The actor won't have to concentrate on matching the original lip movements, they can just take the best audio take and use AI to tweak the on-camera mouth movements to sync up.

Smart use of AI.

I prefer to see Swedish films how they were meant to be seen, in their original Swedish, black and white, and depressing.

This would be an issue for me, albeit a minor one. Even though I think this technology neatly solves the ‘mismatched mouths’ problem of virtually all dubbing up to now, depending on the film, sometimes I want to hear the original language with subs. It just sounds right when I’m watching a film from countries where those characters wouldn’t be speaking English to each other. However, in a high-concept sci-fi thriller like WATCH THE SKIES set in a country where most of the population does speak English as a second language – I’d have no problem having the ‘English mouth’ option, especially in the actors’ own voices. That latter bit is probably the real game changer here and will undoubtedly mean the end of a lot of present-day dubbing companies where the English dub voices don’t really ’fit’ the faces of the original actors, and the dubbers themselves clearly don’t have the acting chops of the original performers, with their generic voices and performance quality often seemingly more suited to anime dubbing or commercial voiceover work.

And as long as they include all the options on the discs or streaming, I’m all for letting current dub techniques become a thing of the past.

I’d wager that the stats for the number of people choosing English dub tracks for everything they watch on streaming services and/or discs made the development of this technology a no-brainer. The ‘I hate subs’ crowd is pretty substantial, after all.

I noticed that the article mentioned the Korean film SMUGGLERS (which I saw at TIFF 2023) had this technology applied to it as well. That’s an example where an entire cast of characters speaking English – even in their own voices, and even with Korean accents – would make no sense in the film’s reality. They’re Koreans in South Korea, for heaven’s sake! But, because the film would’ve had an English dub track created for it anyway, and because the dub tracks on the DVDs and streams for most Korean films are kinda bland and feel like they’re all done by the same handful of actors, this technology offers a huge advantage in erasing the longstanding divide between original performances and the dubs laid over top of them.

One drawback might be when western casting directors start considering actors in ‘foreign’ films for North American film projects based on their English performances in the anglicized versions of the films, only to discover that the actors speak absolutely no English. Mind you, this technology could be employed to smooth that over for the release version in any country. The only confusion might arise on set, when actors have to play their parts in their native tongues and possibly not truly understand each other, but that’s been happening for decades. And again, this kind of tech could simply switch all of them to any language on earth as per distribution agreements.

Incidentally, the director of WATCH THE SKIES made his debut (after several short films) a couple of years ago with THE UNTHINKABLE, and highly ambitious fail that nevertheless appears to have set the stage for the new film.

This would have worked wonders with all those kung fu films shown in American grindhouses in the '70s.

Maybe they used AI in the Hindi version of Scarface?

I am a video editor and we record a lot of remote videos where people are not looking right into their cameras while they're talking. I've been using a service (Descript) to fix that - it makes their eyes look into the camera. I remember Facebook was talking about doing that with their remote calling software so I went to see if anyone else was too.

It works SHOCKINGLY well. I've had pretty extreme cases were someone was literally looking at something several feet away from the camera and it brings their eyes back to center.

We're going to see all kinds of stuff like this.

In Germany "dubbing" is kind of an art form. I prefer the original version, but most people in Germany only watched dubbed movies and shows and you actually don't see that it's dubbed because the dubbing directors and voice actors put a lot of effort into it. I was in my early 20s when I "stopped" watching dubbed content, because I wanted to watch Lost. Now the german voice actors are pretty scared since it's only a matter of time until AI can dub a movie in the original actor's voice

Apple has had that a while on FaceTime, called Eye Contact:
https://www.inverse.com/gear/how-to-...feature-in-ios

Nvidia has it in their Broadcast App
https://www.nvidia.com/en-us/geforce...broadcast-app/

Nvidia first made it available as part of their Maxine AI development kit:

They also showed off "Face Alignment" a few years back (end of this video):

Those are all "real-time" processes though, for streaming/video conferencing. One possible advantage of the TrueSync is that it doesn't have to be generated in real-time, so maybe can be even more realistic.

I think this movie had a big advantage in that all the original actors could also speak English and thus could dub their own voices.

Not directly tied to this film or the tech they used, but I have seen a demonstration of AI translation that I think is adapter the original speaker's voice, and is also doing lipsync.

This was in regards to a proposed AI news channel:

I think they were utilizing this tech from HeyGen:

The translated voices do sound similar to the people's voice, but based on the English translation of the French guy, I think it sounds a little too flat, robotic, with not as much emotion as in the original. So there may be

This will be a slightly tricky bit, since for a Blu-ray, it's no longer as simple as a second dialogue track, but they need an alternate video for all the dialogue scenes as well. The shots of translated text on newspapers and such through reminded me that Pixar already does translated text for other languages for their films on Blu-ray, I believe using the "Multiple Angle" feature where several video streams are running concurrently. I wonder if it could affect compression/disc space though.

Streaming seems easier since it could be a completely alternate video stream, or even an alternate title. You see this with dubbed vs subbed anime titles already sometimes.

Good point about the requirement of alternate video for the scenes in other languages (if not the entire film). Streaming would be a no-brainer, because all the variations just sit there for whomever wants to stream them, but putting multiple versions of the film on one disc would presumably impact the video quality at some point, unless they fine-tuned it on a region-by-region basis (with various countries getting, say, a Hollywood movie in the original English plus one ‘video dub’ version in their own language). Even then, some disc companies might not consider it worth the effort.

I could swear I read recently that similar AI tech was being used to allow some big pop stars to release songs in multiple languages in their ‘own voices’, so I’d imagine this would be an easy trick to pull off in music videos now, too.

This exact thought crossed my mind when I was writing my earlier post last night and glancing at my massive Hong Kong movie collection, nearly all of which is in Cantonese (or earlier Mandarin) with English subs, often of a dubious translation quality. I wouldn’t be surprised if some low-end outfit takes a shot at using this technology to anglicize the voices and mouths of actors in some old public-domain kung-fu film – in their own voices, of course – as a test case. I know the ‘old school’ dub tracks of HK films are cherished by fans who discovered them that way. I find them amusing but almost uniformly cheap, badly performed, and poorly matched. I’d be curious, though, to see big Hong Kong stars – many of whom absolutely could speak fluent English albeit with a Cantonese accent – ‘performing’ entirely in English with voices that actually matched their own, as long as the original language tracks were still made available on various formats.

I would assume the same overall pattern of dubs on the discs would pan out, such as English, French, and Spanish dubs for North American discs.

It's possible Multiview Video Coding (MVC) could be used. That was developed for 3D Blu-ray, and the secondary image is compressed based on the first image, since they're so similar to each other. Theoretically, only parts of the alternate lipsync image would be different, namely the lips, so it should compress well. Support for it isn't universal on Blu-ray players though, although anything new enough to play 3D Blu-ray should work. At worst, viewers would be left without the alternate viewing options on unsupported players.
https://en.wikipedia.org/wiki/Multiview_Video_Coding

I found a few articles about it being tried out.
https://www.weforum.org/videos/this-...-thanks-to-ai/
https://www.billboard.com/music/pop/...ky-1235465936/

Both articles are from 2023, so it hasn't taken off yet.

Going back to catalog titles would likely depend on how much it'd cost to record a new dub, and then run this software and render the new lipsync, compared to how many additional sales it will generate. At this point in time, it may make sense only for new films with theatrical releases that could "go big" if only they were in English.

There's probably a point where it will largely be automated, and maybe done after-market. Like a tool a streaming service provides, similar to Youtube's auto-generated subtitles and auto-translations to other languages. There may be a point where it becomes real-time and basically auto-translates the audio and modifies the lips as you watch, maybe with a 5-10 second buffer/delay.

Personally, I've never really had an issue watching foreign language films with subtitles, and I tend to prefer that over the dubs, even if the dubs are good. But I realize I'm in the minority.

Is this the same tech that Ridley Scott's kid, Jake Scott, was developing after he retired from directing?

Yeah, no thanks. Original language for me. Dubs are lame.