Ein Baum und seine Abenteuer



The Gods of Diffusion


I have rencently been spending quite some time with text2image generation. And while looking around for great styles for a project I generated a "Chromatic Goddess of Colors" that I really liked. This sparked the idea within me to generate a whole pantheon of gods and see how far I could push this!

I labeled them "The Gods of Diffusion" and here they are:

The Pantheon of Diffusion
The Pantheon of Diffusion

For text2image tool, you might have heared of Dall-E and Midjourney as popular platforms. While both of these are paid services, there is also an Open Source solution called Stable Diffusion you can experience it online using DreamStudio or even run it locally on your machine, for free. For this experiment I ran Stable Diffusion on my Razer Blade 14" with a 100 Watt RTX 3070. I set it up using AUTOMATIC1111/stable-diffusion-webui and used the weight for Stable Diffusion 1.5.

During the process I generated over 120 images with each taking about 15 seconds to be created. Let me introduce you to the results and walk you through the process.

The Pantheon of Diffusion

Named in a referance to the Diffusion part of Stable Diffusion, there are six gods in the Pantheon.

For each god I will share all parameters for generation that differ between all of them. Read the next section for a deeper look at prompt crafting and other parameters used.

The Chromatic Goddess of Colors

She was the first in the pantheon of Diffusion. I image her to be the guardian of colors and everything that brings them into our lifes like play, theater, arts and music. She is the one that makes sure that we see the world in all its colors and that we enjoy it thoroughly enjoy them.

In modern society she would likely be the guardian of diversity and inclusion, making sure everyone can add their color to each others life.

The Chromatic Goddess of Colors
The Chromatic Goddess of Colors
chromatic goddess of colors
Seed: 1501367136, Model hash: 81761151

The Vibrant God of Inspiration

His image radiates strength and the bravery required to go out into that unknown space of posiblity where we can find inspiration. I see him inspiring all of us to take on the challenges in our lifes. His image reminds us to strive for those "break through" moments.

The Vibrant God of Inspiration
The Vibrant God of Inspiration
vibrant god of inspiration
Seed: 1501367136

The Brilliant Goddess of Perseverance

The Goddess of Perseverance was born from a long process, perfectrly illustrating her virtues. She reminds us that perseverance makes us shine brilliantly. Already in her image you can see how she inspired the artist to stick with those details, persevearing through the process that is the creation of an artworks. For me she is a symbol to trust the process and belive in the results.

The Brilliant Goddess of Perseverance
The Brilliant Goddess of Perseverance
brilliant goddess of perseverance
Seed: 4286468546

The Sparkling God of Divination

The Goddess of Perseverances partner is the God of Divination, his eyes see into the "might be", they sparkle with the posibilities that lie ahead of us. His vague shapes in contrast to those of his partners remind us that the future is yet to be written. While his eyes are seeing all the "could bees", it's up to us to make the "will be"!

The Sparkling God of Divination
The Sparkling God of Divination
face of sparkling god of divination
Seed: 4286468531

The Alleviated Goddess of Creation

As goddesses go she is on a vague side of things. Brimming with the possibilities of creation her shape never really settled and is instead constantly being re-created over and over again. She created everything and everything is creating her. Thinking about her forming hair reminds us that everything is being created all the time and as she created us, so are we creating the world.

The Alleviated Goddess of Creation
The Alleviated Goddess of Creation
alleviated goddess of creation
Seed: 4219165820

The Steadfast God of Entropy

He accompanies Creation. As Creation is the start of everything, he is it's end. Everything goes to entropy, even the god himself has a whole in his chest that get's bigger all the time without every growing. All that is not conciously maintained and re-created goes to entropy, falls into disrepair and eventually, disappears.

The Steadfast God of Entropy
The Steadfast God of Entropy
face of steadfast god of entropy
Seed: 862733685

The process

All gods were generated using the same basic parameters.

{prompt}, dramatic lighting, illustration by greg rutkowski, yoji shinkawa, 4k, digital art, concept art, trending on artstation
Steps: 60, Sampler: LMS, CFG scale: 7, Seed: 862733685, Size: 512x512, Model hash: 81761151

The things that vary between them are the seed and the prompt. For some gods I had to add face of to the front of the prompt, as Stable Diffusion would generate full bodies from just their name. If you don't know what the rest of the parameters do, that is enough content for it's own blog post.

The Goddess of Colors was born more by accident. Following her images, I tried generating more gods using the same seed, to imitate the style. That worked perfectly for the God of Inspiration and, seeing the amazing results from the first two images I wanted to continue with the same seed to ensure a consistent art style. But soon I had to realize that for one, this would be boring and two, many gods I came up with didn't yield nice results. So, after I struggled to get good images out of the same seed and as I already felt attached to the names I had come up with, I decided to take a different route: open up the seed and go for pairs of gods.

As you can see above the pairs of gods always share the same seed for generation. This gives them a style that is somewhat related. The part of the prompt that gives the overall "style" was inspired by Concept Art images on Lexica that I likes and combined.

To generate individual gods I ran generations in batches of 16 images to make sure I coudl find images that truely resonated with the gods.

Retouching

Generation using Stable Diffusion in fact worked so good that I only had to touch up one of the gods. For the God of Divination I didn't like his original right eye, so i replaced that with the left one and ran him through img2img to let Stable Diffusion fix any artifacts.

Fixing the right eye of a god
Fixing the right eye of a god

Upscaling

My initial workflow produced images in the size of 512x512. To get something more useable I used the SD Upscale feature of the webui. This feature upscales overlapping tiles of the image and runs them through img2img using the same prompt and seed as the original generation to imoprove quality of the upscale. For this process I used a 150 steps to really bring out all the details, a Denoising Strength of 0.23, a widtha nd height of 640 (following the guides recommandation) and everything else set to the same as during original generation.

Upscaling a god - left 512x512 and right 1024x1024
Upscaling a god - left 512x512 and right 1024x1024

The upscaling process reveils more details and enables display on larger screen or usage for other media.

Final thoughts

This was a super fun evening of diving into image generation. If you feel inspired to try something like this yourself, as of writing this Lexica is the best place to experience text2image generation yourself, it only required a Google login to get started. While all service offer a free trial, Lexica will get you up to speed with truely good looking images.

I hope you enjoyed this journey into the Pantheon of Diffusion.

Trident Emoji
Trident Emoji

Author

Hendrik

I am a JavaScript Enthusiast and developer for the fun of it!
Here I write about webdev, technology, personal thoughts and anything I finds interesting.

More about me

Read next

Generative AI is here to stay!

"A path into the AI future" - generated using Midjourney
"A path into the AI future" - generated using Midjourney

Some reflections on what 2022 showed us about the future.

Generative AI describes AI that takes a prompt and generates something from it. Prime examples include Copilot which takes code or comments as a prompt to complete the code I want to write, ChatGPT which takes a chat message and replies with a human-like answer and Stable-Diffusion (or Dall-E, or Midjourney) which takes a prompt and returns a fitting image. All of these AIs made massive progress in 2022!

Trying my hands at Deno v1.0

Denos awesome Dinosaur logo.
Denos awesome Dinosaur logo.

Following the release of Deno v1.0 I got excited to try my hands at it. These are my first experiences writing a simple tool in Deno.

A super fast introduction to Deno: Deno is the spiritual successor of Node trying to fix design mistakes that were made early on but recognized only late into the project. Deno supports TypeScript out of the box and relies on web-standards. In Deno you can import ES modules from any URL and use fetch like you would in the browser. To help unify the community on processes and workflows Deno provides a wide array of stdLibs and has build in solutions for bundling, testing and code formatting. You can read more in the Deno v1 release post.

Log to Elasticsearch using curl

Good old tools can do amazing jobs.
Good old tools can do amazing jobs.

Once you get a centralized logging solution like Elasticsearch setup you open up an incredible amount of possibilities. That is if you actually send logs to your central place. On a recent project, we wanted to "just log our deploys from the Jenkins Server". Thus we set out to find the "easiest" way to log to our Elasticsearch. We finally reached curl as our solution and today I will walk you through how to do that.