A Midjourney Through Space
Perhaps you’ve been hearing the buzz about the Midjourney algorithm lately…let me confirm, it puts DALL-E to shame. Midjourney is an app/bot/AI that utilizes multiple ML models to generate images based on text input. There’s not much information about which models or ML techniques it uses to do its magic, but the founder claims they’ll do a press release at some point to reveal their tech stack. I for one am EXTREMELY interested in what they’re using on the back end. There’s definitely a Generated Adversarial Network involved (prob StackGANS and StyleGANS), some kind of large language model like GPT-3 for the NLP, trained on a multimodal dataset that any data scientist would salivate to get their hands on. As for the servers…as a Google Cloud Platform employee I’d be thrilled to learn they’re using our virtual machines, but it’s anyone’s guess.
You use Midjourney through Discord — that’s your only option as of now. You can join the Midjourney server and send around 25 requests to the AI before you need to pay up. Its $20 a month for 200 requests, but I was having so much fun I sprang for the $30/month unlimited option. Yes I ball hard.
I’ve written a Science Fiction novel, and behind it I’ve designed an entire universe, so of course that’s what I’m going to try to recreate with Midjourney. Let’s see how I did.
I started with a scene from the novel that I wanted to someday have illustrated. I had a pretty clear mental image of what I wanted with this one, and let me tell you now, if you’re trying to exactly recreate what you see in your head, you’ll be slightly disappointed by the results. My first prompt was “a husband and wife stand in front of a spaceship shaped like a fish, night sky, stars, deep space”
“a husband and wife stand in front of a spaceship shaped like a fish, night sky, stars, deep space”
Not even close to what I wanted. Let’s try something slightly different.
“a man and woman stand in front of a spaceship shaped like a shark, night sky, stars, deep space, realistic”
Adding “realistic” to the prompt always gives you…believe it or not…a more realistic image. It also seemed to understand the concept of a shark-shaped spaceship better than a fish-shaped one. Also, Midjourney cares not for your antiquated conceptions of matrimony…man and woman will do just fine. The first image bears a slight resemblance to the image in my head, so I asked Midjourney to make a few variations of that one.
Variation on generated image 1
Upscaled version of image 1
Not too bad! It’s wicked fast considering how complex the underlying ML architecture must be…part of what makes it so quick is that it’s not doing a full prediction every time you send it a prompt. Midjourney has already precalculated how every word in the english language is related to every image on the internet. When you send in a request, it has to find all the “relation values” and essentially add them all up. Then there’s a Diffusion Model that takes that result and re-creates the image it thinks would match the numbers it was given. That’s an incredibly reductive explanation, and I don’t know it’s true cause Midjourney hasn’t released any info about it’s tech stack, but that’s typically how these AI’s work.
I wonder if it could create one of my alien species called the Argruarians…
“shiba inu with kangaroo legs, black fur, yellow eyes, photo realistic”
Oh god, no. Just no. It seemed to do well with the spaceship. I’ve imagined a general idea of what spaceships looked like in my universe, but have never drawn anything detailed. After many iterations, I was able to create some pretty sick concepts.
“spaceship that looks like a sunfish, sleek, pointy fins, jet engine in the back”
“spaceship that looks like a sunfish, sleek, pointy fins, jet engine in the back”
You’ll see those both started with the same prompt, but produced very different results. It is ridiculous fun crafting the prompts, then selecting which variations to iterate on. In my next entry, I’ll walk you through my midjourney to create a new book cover (spoiler alter, the results are SICK).
If you’d like to read my book and/or help get it published, sign up for my newsletter at remywelch.com!
