A.I | A.I Animation | A.I Art | Animation | Music | Pomp Yin | Stable Diffusion

How I Made My First A.I Music Video

October 11, 2022
James Cunliffe

If you’re interested in exploring the world of A.I music videos, you’re in luck! Thanks to the amazing work of Sebastian Kamph, an Art Director from Sweden, it’s now possible to create your very own music video using Stable Diffusion and A.I tools. In this blog post, we’ll take a look at how I made my first A.I music video using the recommended sd-v1-4 model and following Sebs instructions.

Step 1:

Isolate your musics drum track, export it and then upload it to Audio to Keyframe String Generator. This tool allows you to extract the rhythmic patterns and beats from your audio track and convert them into keyframe data that can be used to create your A.I music video.

Step 2:

Browse Lexica.art for stylised prompts to make your A.I music video more visually interesting. Once I found something I was happy with, I copied the non-descriptive “stylisation” words and added my own description after. This helped me create a unique and personalised video.

Step 3:

Use Deforum to add a series of prompts that will be called in the image generation process on specific frames. Deforum is a tool that allows you to generate images using a combination of prompts and pre-trained A.I models. For my music video, I used the 2D animation mode, toggled the wrap border and pasted the keyframe data into the angle and zoom fields, manually toggling the amount it would be affected.

Step 4:

Render out the images and make final tweaks in Premiere and After Effects. This includes adjusting the colour and fixing a couple of sync issues.

As a bit of an experiment, I was fairly happy with the results. The final video reminded me of some of the old media player animations in the late 90s/early 2000s. The A.I music video was visually engaging although I do reckon this style of animation will probably be overused this year. I can see it being a bit of a fad.

In conclusion, creating your very own A.I music video is not as difficult as you may think. Thanks to the amazing tools and resources available, it’s now easier than ever to create stunning visuals that perfectly complement your music. If you’re interested in giving it a try, be sure to check out Sebastian Kamph’s tutorials on A.I music videos using the Audio to Keyframe String Generator and Deforum. With a little bit of creativity and experimentation, you too can create an A.I music video that will leave your audience mesmerised!

New Skool Community

The Hidden Archive: Free Creative Tools, Resources and a New Teleprompter App Over the past few years, I’ve created a slightly ridiculous number of things. Books. Apps. Courses. Music. Videos. Experiments. Resources. Some useful, some strange, and quite a few that sit somewhere in between. One of the things I’ve wanted to do for a while is create a single place where I can bring more of this work together and make it accessible to people who might find it useful. That place is The Hidden Archive Skool community. The idea is to build a growing library of creative

The Hidden Archive

What is The Hidden Archive? The YouTube channel finally got a rebrand. I realised two years ago that my YouTube page was starting to feel uncurated. The simple truth was that being multidisciplinary, and wanting to explore a wide variety of mediums, had led to a kind of anti-niching. In my own way, I was navigating the principle of correspondence. In creative abstraction terms, that essentially means gaining an understanding of multiple fields and then connecting the dots between them to deepen my understanding of creativity as a whole. That said, I realised that on the surface, this

What Are UFOs?

What Are UFOs and What Do They Say About Humanity? Back in 2024, I went down the rabbit hole of UFOs/ UAPs. It was around the time David Grusch hit the scene and it really did seem like we were going to get disclosure any day. There was an energy around the topic, it felt charged, so much so that I found I could rarely think of much else. I read American Cosmic by Diana Pasulka, picked up Elizondo’s Imminent, watched all the congressional hearings on the topic, and got hooked on AJ’s The Why Files. The whole

The Gnostic Creation Story

Why This Story Is Valuable While researching the Red Book series, I’d occasionally read comments linking Jung’s work to Gnosticism. I was raised by Christians, and although I’m agnostic myself (if I had to put a label on it), I’m still interested in the history of religion and more recently whether the stories hold any value in the modern world. I eventually made my way to an Alex O’Connor video with Bart Erhman where they outlined the core aspects of the Gnostic Creation Story, from the Apocryphon of John. Which then led me to a translation of those

Creative & Multimedia Artist

A full-time creative sharing the journey.

Previously an animation technician for a subsidiary of Epic Games. As well as a digital producer, creative director, and gaming media content lead for multiple award-winning agencies.

I lead end-to-end content delivery, shaping the idea, building the plan, and directing execution through production + post.

WATCH HERE