Stable Diffusion Goes Public and the Internet Freaks Out – DevOps.com

Welcome to The Long Viewwhere we peruse the news of the week and strip it to the essentials. Lets work out what really matters.

Unless youve been living under a rock for the past week, youll have seen something about Stable Diffusion. Its the new open source machine learning model for creating images from text and even other images.

Like DALL-E and Midjourney, you give it a textual prompt and it generates amazing images (or sometimes utter garbage). Unlike those other models, its open source, so were already seeing an explosion of innovation.

Mark Hachman calls it The new killer app

Fine-tune your algorithmic artAI art is fascinating. Enter a prompt, and the algorithm will generate an image to your specifications. Generally, this all takes place on the Web, with algorithms like DALL-E. [But] Stability.Ai and its Stable Diffusion model broke that moldwith a model that is publicly available and can run on consumer GPUs.For now, Stability.Ai recommends that you have a GPU with at least 6.9GB of video RAM. Unfortunately, only Nvidia GPUs are currently supported. [But] if you own a powerful PC, you can take all the time youd like to fine-tune your algorithmic art and come up with something truly impressive.

From the horses mouth, its Emad Mostaque: Stable Diffusion Public Release

Use this in an ethical, moral and legal mannerIt is our pleasure to announce the public release of stable diffusion. Over the last few weeks we all have been overwhelmed by the response and have been working hard to ensure a safe and ethical release, incorporating data from our beta model tests and community for the developers to act on.As these models were trained on image-text pairs from a broad internet scrape, the model may reproduce some societal biases and produce unsafe content, so open mitigation strategies as well as an open discussion about those biases can bring everyone to this conversation. We hope everyone will use this in an ethical, moral and legal manner and contribute both to the community and discourse around it.

Yeah, right. Have you ever been on the Internet? Kyle Wiggers sounds worried: Deepfakes for all

90% are of womenStable Diffusionis now in use by art generator services like Artbreeder, Pixelz.ai and more. But the models unfiltered nature means not all the use has been completely above board.Other AI art-generating systems, like OpenAIs DALL-E 2, have implemented strict filters for pornographic material. Moreover, many dont have the ability to create art of public figures. Women, unfortunately, are most likely by far to be the victims of this. A study carried out in 2019 revealed that, of the 90% to 95% of deepfakes that are non-consensual, about 90% are of women.

Why is it such a big deal? Just ask Simon Willison:

Science fiction is realStable Diffusion is a really big deal. If you havent been paying attention to whats going onyou really should be. Its similar to models like Open AIs DALL-E, but with one crucial difference: they released the whole thing.In just a few days, there has been an explosion of innovation around it. The things people are building are absolutely astonishing. Generating images from text is one thing, but generating images from other images is a whole new ballgame. Imagine having an on-demand concept artist that can generate anything you can imagine, and can iterate with you towards your ideal result.Science fiction is real now. Machine learning generative models are here, and the rate with which they are improving is unreal. Its worth paying real attention to.

How does it compare to the DALL-E? Just ask Beyondo:

Personally, stable diffusion is better. OpenAI makes it sounds like they created the holy grail of image generation models but their images dont impress anyone who used stable diffusion.

@fabianstelzer did a bunch of comparative tests:

These image synths are like instruments its amazing well get so many of them, each with a unique sound. DALL-Es really great for facial expressions. [Midjourney] wipes the floor with the others when it comes toprompts aiming for textural details. DALL-Es usually my go to for scenes involving 2 or more clear actors. DALL-E and SD being better at photosStable Diffusion can do incredible photosbut you need to be careful to not overload the scene.The moment you put art into a prompt, Midjourney just goes nuts. DALL-Es imperfections look very digital, unlike MJs. When it comes to copying specific styles, SD is absolutely [but] DALL-E wont let you do a Botticelli painting of Trump.

And what of the training data? Heres Andy Baio:

One of the biggest frustrations of text-to-image generation AI models is that they feel like a black box. We know they were trained on images pulled from the web, but which ones? The team behind Stable Diffusion have been very transparent about how their model is trained. Since it was released publicly last week, Stable Diffusion has exploded in popularity, in large part because of its free and permissive licensing.Simon Willison [and I] grabbed the data for over 12 million images used to train Stable Diffusion. [It] was trained off three massive datasets collected by LAION. All of LAIONs image datasets are built off of Common Crawl, [which] scrapes billions of webpages monthly and releases them as massive datasets. Nearly half of the images, about 47%, were sourced from only 100 domains, with the largest number of images coming from Pinterest. WordPress-hosted blogs on wp.com and wordpress.com represented6.8% of all images. Other photo, art, and blogging sites includedSmugmugBlogspotFlickrDeviantArtWikimedia500px, andTumblr.

Meanwhile, how does it work? Letitia Parcalabescu is easy for her to say:

How do Latent Diffusion Models work? If you want answers to these questions, weve got you covered!

You have been readingThe Long ViewbyRichiJennings. You can contact him at@RiCHior[emailprotected].

Image: Stable Diffusion, via Andy Baio (Creative ML OpenRAIL-M; leveled and cropped)

See the original post here:
Stable Diffusion Goes Public and the Internet Freaks Out - DevOps.com