r/aivideo

Bring back this nostalgic game!!

This is not a promotion of any of the tools used, everything mentioned is open-source and the homemade ones are a vibe-coded mess.

This song is part of Residual Instabilities, an album that uses large-scale astrophysics as metaphors for very personal experiences.

For this track, I used tidal locking as a metaphor for autistic masking: keeping one carefully controlled side facing outward while the rest remains unseen.

I’ve seen a lot of questions here about how people make full music videos, so I thought I’d break down my process.

I built two tools for this: Beatcutter and Scenify.

The basic pipeline is Beatcutter → Scenify → Wan2GP → Beatcutter.

Beatcutter detects the BPM and calculates an ideal clip length so scene changes land on the beat. It also acts as the final video editor. Traditional editors can obviously cut to music, but they don’t make it especially easy to build an entire video around consistently timed AI-generated clips.

I first create the overall storyline. Then I give Scenify the song, the clip length from Beatcutter, and the storyline.

Scenify uses a local Gemma 4 12B model to split the song into scenes. For each scene, it creates a starting-frame prompt and a movement prompt describing what happens during the clip. It then exports everything as a ZIP file containing the scene details and prompts.

That ZIP goes into Wan2GP, where I rendered the scenes using LTX 2.3 and the audio-reactive LoRA from fal.

An important detail is that the video model received the actual audio from that moment in the song. So when things pulse, distort, move, fracture, or react to the music, those moments are not just coincidental or manually synced afterward. The model was generating while conditioned on that specific section of audio.

This video was a best-of-two. I rendered two versions of each of the 52 scenes and picked the better one. I wasn’t generating dozens of variations and cherry-picking one usable result.

On my hardware, an RTX 4070, each clip takes around 12 minutes to render. At 52 scenes with two versions per scene, that comes to roughly 21 hours of total rendering time. Assuming the computer draws around 400 W, the full render used about 8.3 kWh of electricity, which works out to roughly €3 at average local household electricity prices.

The selected clips then go back into Beatcutter, which places them on the timing grid and assembles the final video.

The total manual work was roughly 30 minutes before rendering to prepare everything, and about an hour or 2 after rendering to choose clips and assemble the final edit. The rest was rendering time.

Suno: https://suno.com/s/agmRausUbsH8qPzI

YT: https://www.youtube.com/watch?v=yFpLlcOrFbQ