- Luma Labs’ Dream Machine can now add audio to video clips for free
- You can prompt the audio or let the AI come up with something it decides is appropriate
- Sora and other AI video makers mostly lack even an imperfect AI audio creator to go with their visuals
Luma Labs has added a score to the AI videos produced on its Dream Machine platform. The new feature brings audio to your video, custom-generated to match a written prompt or created by the AI, and is based solely on what’s happening in the video. That could mean chirping birds at the sunrise scene, a spaceship’s distant hum for your sci-fi animation, the chaotic clatter of a busy café, or anything else you care to hear.
The new feature is free in beta for all users. After generating a video with Dream Machine, you’ll see a new “Audio” button along the row at the bottom of the video next to the existing “Extend” and “Enhance” buttons. Click it, and you get two choices: let the AI decide the best fitting sounds on its own, or take the wheel and provide a text prompt describing exactly what you want. Maybe you’ve got a dreamy nature scene and want to hear a distant waterfall, or maybe you want to hear how the AI does it; either way, it works.
Sound Idea
This update is big because AI-generated videos, while sometimes visually stunning, have always felt incomplete without sound. It’s a lot of work to painstakingly add audio yourself. Even some of the biggest names in AI video don’t have audio as an option yet, including OpenAI’s Sora.
Of course, AI sound generation on its own isn’t unique. There are a lot of AI music makers, even full voice and song producers. But, the production within the platform linked to the video already there makes Dream Machine a real standout. That said, it isn’t perfect. You can tell from the way the motion and sound don’t quite match with this dog as it swims.
On the other hand, when prompted correctly, this crackling fire and laughter of people around it sounds pretty good.
But, I wouldn’t rely on Dream Machine to create sound on its own without any guidance in a prompt. With a blank audio prompt, the AI took the same short clip of people around a fire and came up with something a lot spookier.
Leave a comment