STABLE DIGEST #3
Artist spotlight - Sagans join us straight from the studios of Linkin Park, Lorn and Die Antwoord for an AI animation deep dive
THIS ISSUE: SAGANS JOIN THE STABLE SOCIETY IN OUR SPOTLIGHT INTERVIEW
Welcome back to your AI art and culture companion: Stable Digest!
Straight from the studios of Linkin Park, Lorn and Die Antwoord come Sagans, joining us in our artist spotlight feature to reveal the secrets behind their art and music!
Your roundup from across the Stable community returns with artwork by our mod team, our POW champions and your favourite Models and Embeddings.
And we’re wrapping things up with a Stable Diffusion model training masterclass with Claire Silver!
Thank you for joining us on another journey into the wonderful world of all things Stable!
BY THE COMMUNITY
MODS MADE THIS
We’re back for another look at what our Discord moderator team have been up to!
Here’s are some of the brilliant artwork they’ve created since our last issue.
We’re always looking for helpful members of the community to join our Discord volunteer team!
If you’re interested in being one of our Stable Guides or Moderators, please fill out the application form here!
PICTURE OF THE WEEK
Now for a journey into the wonderful world of Picture of the Week!
Each week, our community of awesome artists rise to the challenge and create the most spectacular themed artwork, as voted for by you.
In this issue’s feature, we’re taking a trip through a cave of wonders, where vcm07 discovered a paradise world of Desires.
Deep below the endless oceans of this magical world we discovered strange and spectacular Underwater Worlds alongside our POW champion The Jingo.
Don’t forget to drop by the Community Events Centre and join the POW fun!
Stay tuned for more exciting events you can get involved in very soon!
MODELS AND EMBEDDINGS
Our community continues to create and share a selection of spectacular Models and Embeddings in our Discord forum.
This issue, we’re taking a look back at one of the biggest hits on the share forum: Joachim’s Double Exposure Embedding for SD 2.X!
This incredible embedding can be used to create photorealistic artwork with brilliant double exposure styles.
And hot off the press we have the long-awaited expansion to Joachim’s masterpiece: Double Exposure (Photo Edition) for SD 2.X!
Come and see the latest models and embeddings being shared in the forum.
If you’ve spun up a creation of your very own, we’d love for you to share it!
DREAMER COMMUNITIES
Our Dreamer Community forums are full of amazing artwork created by our community.
Stop by and browse a range of categories, discuss your favourites and post your own masterpieces!
This issue, we’re taking a look at all things Fantasy!
Weird and wonderful creatures, landscapes and characters all call this dreamer community home. Take a look at some of these fantastical pieces created by our community!
WITH THE COMMUNITY
STABLE SOCIETY DEEP DIVE WITH SAGANS
"Always create with the aim of pleasing yourself first"
Welcome! Thank you for being here with us today for our artist spotlight!
First of all, “Sagans” is a wonderfully intriguing name! Can you give us an introduction as to who, or what, Sagans is?
Hey, thank you for having us today!
The name Sagans is a direct reference to Carl Sagan.
When we watch videos of him, we feel such tremendous poetic energy and always think that the world needs someone like him to bring more and more beauty into it.Sagans with an "s" is a kind of joke, a unit of measurement equivalent to at least four billion - an expression derived from the astronomer's frequent use of the phrase "billions and billions of stars." This expression makes a lot of sense when applied to the billions of calculations performed by AI to create art.
Now that you know a little more about the name, what is Sagans?
We are a small collective of AI enthusiasts, designers and music producers. We are trying to push the boundaries of what is possible with artificial intelligence in the world of music and art - we believe that AI can open up new levels of creativity and expression.
As artists and musicians blazing the trail across both mediums, we’re really interested to find out your biggest inspirations!
If you had to choose one song and one music video, which would you say have influenced you the most on your journey to becoming AI-fuelled musicians and animators?
This is a difficult question, we could name hundreds of songs that have inspired our journey. If there was only one to choose, it would be Acid Rain by Lorn. We have always had a very special connection to that song and to Lorn's entire discography in general. Working together over the last few months has only made that connection stronger.
As for the videos, we take inspiration from many different things, from Paul Thomas Anderson's Daydreaming music video for Radiohead to anime like Akira by Katsuhiro Otomo.
If we had to name one music video, we would name an old one that definitely had an immense influence on our overall work:
Bjork - All is full of love by Chris Cunningham, this music video is so poetic and was so futuristic at the time.
Your Instagram is a real treasure trove of amazing anime-style shorts!
What inspired you to adopt this iconic visual style?
We are big fans of anime, from "Akira", "Paprika" to "Terror In Resonance", and of course also massive fans of the magical ones by Studio Ghibli or Makoto Shinkai.
We have been working on our anime style since the beginning of 2022, when Disco Diffusion was released. Since the release of Stable Diffusion, our style has steadily improved and became more specific, a kind of mix of everything we like, with a style research done in a very empirical way.We use Instagram as a freeform medium to post whatever comes to mind without thinking too much. The short video format allows us to be very spontaneous and try out the latest AI techniques - some techniques that we can then expand on in longer videos.
The amount of work and processes behind each and every one of your animations must be extraordinary!
Can you tell us a little about your workflow, and any unexpected tips, tricks or techniques you’ve discovered along the way?
Our technique is evolving every day, but there are definitely a lot of good practises that remain unchanged:
First, training the model is the most important thing. We usually train the model step by step and test it at every step to make sure the model is not overtrained, but also to make sure we are just below the overtraining threshold to get the best consistency.
When we first started working with artificial intelligence, we did a lot of work with green screens and spent hours pre-processing the video before applying artificial intelligence to it. Now that we have more powerful tools like WarpFusion with Controlnet at our disposal, we can work with a full-frame approach - but sometimes we have to resort to a more compositional approach if the character is too specific and needs to be isolated to be properly understood by the AI.When using WarpFusion the most important thing is to get the first frame right: something highly stylised but with all the elements perfectly placed.
Once we get the first image right, the big task is to keep the style as long as possible (thanks to high Optical Flow) without increasing the style strength too much (which leads to flicker).
Actually, it should be a very automatic technique, but to improve the quality, we always work very manually and with a lot of back and forth.
You’ve directed and produced AI music videos for a number of huge artists, showcasing your signature anime style in Die Antwoord’s “Age of Illusion” video for example.
How do you begin to visualise lyrics and music in the early stages of a project such as this as you build concepts and establish themes?
The fact that we have a musical background helps a lot.
When we compose music, we are used to having a lot of visual ideas. When we work for other music artists, we pick the people we work with to make sure we have the same vision. We want the visual work to remain a pleasure, so the most important thing is to work for inspiring music: you have to be inspired if you want to get the right visual ideas quickly.
So far, we have had the pleasure of working with music artists who did not give us any specific guidelines other than a very strong sense of aesthetics ("we want an anime", "we want something realistic"). The freedom these artists gave us only encouraged our creativity - and when there are no limits, you can not limit yourself, you want to go as far as the technique allows.Since you are talking about "Age of Illusion": this music video was extremely quick to make, as we all wanted something in a trippy anime style.
We ended up going for something extremely colourful with lots of glitches and funny characters inspired by Die Antwoord.
At that time Dreambooth did not exist yet, but Die Antwoord were famous enough to be part of the SD model. It was amazing to be able to work in that way. Age of Illusion is set in the land of illusion, where everything turns into something else.
This was made possible thanks to a 50/50 mix of Deforum for the latent videos and WarpFusion for the Video Init-style transfers.
Your video created for LORN’s “Entropyyy” is quite the tonal shift from your previous work, with its dark, grungy visuals taking us through a world of decaying machinery and urban sprawls.
Can you give us an insight into the imagery we see in the video, and any challenges you faced in animating this new and unique style?
There's a difference between the anime style we are developing on Instagram and the more defined styles we are approaching when working on some longer video formats.
In June 2022 we released a music video for our own music "Coherence" set in an infinite city, inspired by a futuristic London where you feel small and lonely at night.
After the release of this video, we started talking to Lorn about his future works and we all felt that "Entropyyy" should be in the continuity of Coherence, but with a darker and dustier atmosphere.
Lorn's universe is known for being in black and white, and for us it was clear that black and white would suit this new video best. Black and white is a bold choice, it's not easy to play with, but it can also be a great ally when you are trying to achieve a really good consistency in style, especially at the time when Dreambooth was not yet available.
Your most recent project saw you working on the video for Linkin Park’s “Lost”, a song that recently saw release 20 years after its original recording.
An AI-powered music video feels especially fitting to this song in particular; What was it like to work on this project?
This project was directed by two amazing people we have been following for a long time: Maciej Kuciara and Pplpleasr, known for many renowned projects and more recently for founding a modern video platform / production company called Shibuya.xyz.
The two directors, along with Mike Shinoda and Joe Hahn from the band, wanted to take a modern approach for this music video. The track was recorded 20 years ago and only released today, so they needed a concept where we could visualise old footage while giving this older footage a modern look. AI was then the perfect solution for this, using AI to transform each member into an anime.
When we started working on the project, we all decided to take the most ethical approach to realise this idea:
Since Shibuya.xyz created their own anime "White Rabbit", we all wanted to make sure that the AI style would only be trained on this anime. To this end, we first had to train all the images from "White Rabbit" to get a "ckpt" containing that specific style. Then we stylised all the images we could find of each member with that style. Once we had enough images of each member turned into a consistent anime, we could train a model of all the members with different tokens. Some shots were complicated to style as some facial expressions were difficult for the AI to understand at a time without controlnet, but it all worked out in the end. Investing a lot of time in good training was definitely key here to save some time during the production process.
The finished video is amazing and we were very surprised when we saw the exponential number of views on Youtube, which recently reached 28 million, that's just incredible!
Congratulations to the great team that worked on this video!
Die Antwoord, LORN, Linkin Park… The list keeps growing!
Can you give us a small hint on the exciting projects you’re working on right now?
And while that list continues to grow, who would be your dream artist to collaborate with?
We'd love to tell you more about the upcoming projects, it's so hard to keep secrets - especially when there's sometimes such a long time between the end of production and release, especially with larger projects. We have been working on a big TV show for months, I think that's the thing we are most impatient about, but we can not reveal more about it.
We are very lucky to be able to work with people we have admired for so long. There are still a lot of people we would love to work with in the future. If we had to name just one, it would be FKA twigs.
As more and more people start to discover the potential of AI as an art medium, they’re going to want to get in on the animation action, however daunting a prospect that may be!
What advice would you give someone just starting out on their journey as an AI animator, and where would you suggest they begin?
Starting AI now is a blessing compared to a year ago.
There are so many tutorials on Youtube now, so many resources on Reddit and great communities on Discord.
We personally think WarpFusion is an amazing tool for new animators. It takes a bit of time to understand all the settings, but we think this is one of the most amazing tools for consistency.
To complement this tool you need good models and the resources on civitai.com are amazing when it comes to anime. Most of the time we train our own models to get something personal, but using these huge model databases is the best way to get great results quickly.
Thank you for an awesome interview and insight into the wonderful world of AI animation!
To round us out, is there anyone in particular you’d like to shout out or highlight?
And finally, for our quote of issue, tell us the first thing that comes to mind!
Thank you so much for hosting us today!
We would like to highlight the work of Remi Molettee - thank you for always raising the quality of your work, both artistically and technically!
"Always create with the aim of pleasing yourself first"
FOR THE COMMUNITY
CLAIRE SILVER - STABLE DIFFUSION MODEL TRAINING AND WORKFLOW
Back during the Stability Twitch channel launch, amazing AI artist Claire Silver gave a fascinating deep dive into her workflow, unveiling some of her Stable Diffusion model training secrets…
Rediscover Claire’s brilliant insights in this recap video!
And here we are at the end of another issue of Stable Digest! We hope you’ve enjoyed another insight into the amazing art and artists in our communities.
If you’d like to keep up to date with all the latest as soon as it happens, join our vibrant community over on the Discord server!
Got any great ideas or feedback for Stable Digest? Anyone you’d really like to see us shine the artist spotlight on for a deep dive interview? We’d love to hear from you.
Drop us an email at community@stability.ai and let us know!
Thank you for being here with us - We’ll see you soon in issue #4!