I love Midjourney, use it almost every day to generate the illustrations that are an important component of this blog. In fact, this blog’s name change from Crafty’s to Crafty’s Illustrated happened entirely because of Midjourney, which made possible the “Illustrated” part. I happily pay $10/mo for these benefits.
I’d gladly fork out similar money for something like Midjourney but for video. To that end, some months back I signed up on a waitlist (or something) with Runway. Today I received an announcement that Runway Gen-2 is available. So I decided to give it a workout and assess the current state of generative AI for video.
We Need a Posse
In my last essay about writing, I touched on IndieWeb’s POSSE principles and that seemed like a great opportunity for an illustration. I used this Midjourney prompt:
a group of men on horseback, riding at high speed, in pursuit of a cattle thief, cinematic spaghetti western technicolor
and this was the winning image:
Midjourney actually generated four images for that prompt, and the others were quite good also:
From prompt to initial images was maybe 30 seconds, and the final image was web-ready in 5-10 minutes including tweaking and exporting to .webp using Pixelmator Pro.
Lights, Camera, Action
So, we might as well try Runway Gen-2. Same prompt, shall we? I gave it three tries, and it was a win in the sense that I got this fun little essay as a result. But Midjourney-for-video? Not so much.
Take 1: What?!
Here we see galloping at high speed … seated backwards on their horses? And in the background, the horses seem to be stampeding in reverse or something. A pretty amazing clip–though not exactly what we were looking for.
Take 2: Huh?
Here we have … a group of men riding while sitting on raised platforms fastened to their horses, kind of a war-elephant setup. Hats do not appear to be western in style, they have a Central Asian mongol-hordes feel. The hat second to left morphs in stages from wizard to Cat-in-the-Hat. And on closer inspection, the horses’ anatomy and mechanics are nightmarish.
Take 3: Look Out!
The first moment here is promising–but then a Reverse Rider, possibly a fugitive from Take 1, blows through the pack like a southbound drunk driver on the northbound Edens Expressway. Chaos ensues.
Nope … But Let’s Be Fair
We’re ready to call this race: Midjourney for video has not yet arrived. But a few things to bear in mind:
- Midjourney generates its share of outtakes, including some true horrors (see examples at the bottom of the page). “That’s such a beautiful little girl, OH GOD SHE’S GOT 8 FINGERS ON EACH HAND” Or “What a cute little bunny AAAGHHH IT HAS AN EXTRA LEG COMING OUT OF ITS BACK”
- A group of men on horseback is admittedly a fairly challenging scene.
- Things move fast in generative AI. It won’t be long.
So, nope, not yet. But I predict it won’t be long.
