Making a music video can be a time-consuming and laborious process, especially if you’re doing it all on your own. But what if you could use AI to speed things up? In this post, I’ll cover the steps I took to create both the music and the video using a combination of traditional creative techniques and cutting-edge AI technology.
Creating the Music
After playing with electronic music tools for over a decade, I’m operating more on instinct and old habits than anything else when I produce these days. I started this project by turning on Native Instruments Maschine to create the bulk of my drums, I wanted something light with a couple of rim hits and found a couple of samples that sat well without needing too much altering. After that I automated Kinetic Metal to create a shaker effect and add some subtle variations in the progression. For the driving bass line, I used Spectrasonics Trillian, and for the core synths, Spectrasonics Omnisphere 2. To add a bit of grit and distortion to my synths, I’ve used the OTT plugin. On top of all that, Gross Beat has been my go to plugin to automate ducking allowing the melodies and drum progressions to work in harmony.
Always been a fan of Hans Zimmer and would like to make a movie score one day, I’m no where near that skill level but a couple of orchestral elements have mananged to sneak into my last couple of tracks and I’ve got my eye on a couple of expensive plugins. I just wish I had more hours in the day to do this kind of work.
Creating the Video
Without much of a break, I began planning out the concept. I wanted the video to have an anime/cartoon style with a glitching effect that made it seem like the character was moving through different locations. Having used Deforum within Automatic1111 I had figured out that I could use green screen footage of a walk cycle as a base layer. Stable Diffusion doesn’t create transparent exports just yet, so keying out the green would be the logical next best thing.
Using Automatic1111, I created a cartoon-style prompt and used a single seed on the whole batch of images. This created a sequence of PNGs in a similar style to each other. I then brought those PNGs into After Effects and used keylight to remove the green screen background. I used a posterize time effect and set it to 12 frames per second as well to match the old school 2s animation style.
For the backgrounds, I used Automatic1111 to create nearly 50 images from the perspective of a road with high-resolution settings enabled. I brought those images into After Effects and used keyframes to position and scale them, creating a seamless transition from one background to the next. To stitch everything together, I used a glitch transition pack from Motion Bro.
The Result
Thanks to the combination of traditional creative techniques and AI technology, I was able to create both the music in under 12 hours. Without the help of AI, I reckon it would have taken a year or more to complete the same project. It’s not perfect by any stretch, the changing faces are definitely something that I’d like to see fixed but the technology is moving so quickly. It’s probably only a matter of time until there’s a solution.
