#controlnet

[users7531]

IllustriousXL personal merge今��で試した中で最強に凄いな それとコントロールネットのCN-anytest_v4 小学生LVの落書きでも自分が真面目に描くより上手くしてくれる・・・ ゲームアニメ業界の絵の工程が５段階に分けると将来 １（ラフ原画）と５（最終仕上げ）の作業だけになりそうですね 「ツインズひなひま」と言うAIアニメがキャラデザラブライブ!虹ヶ咲学園の方で 春に公開されるそうですけどどうなりますかね

[389]

ControlNet for QR Code

[ottopilot-ai]

Anatomy of a Scene: Photobashing in ControlNet for Visual Storytelling and Image Composition

This is a cross-posting of an article I published on Civitai.

Initially, the entire purpose for me to learn generative AI via Stable Diffusion was to create reproducible, royalty-free images for stories without worrying about reputation harm or consent (turns out not everyone wants their likeness associated with fetish smut!).

In the beginning, it was me just hacking through prompting iterations with a shotgun approach, and hoping to get lucky.

I did start the Pygmalion project and the Coven story in 2023 before I got banned (deservedly) for a ToS violation on an old post. Lost all my work without a proper backup, and was too upset to work on it for a while.

I did eventually put in work on planning and doing it, if not right, better this time. Was still having some issues with things like consistent settings and clothing. I could try to train LoRas for that, but seemed like a lot of work and there's really still no guarantees. The other issue is the action-oriented images I wanted were a nightmare to prompt for in 1.5.

I have always looked at ControlNet as frankly, a bit like cheating, but I decided to go to Google University and see what people were doing with image composition. I stumbled on this very interesting video and while that's not exactly what I was looking to do, it got me thinking.

You need to download the controlnet model you want, I use softedge like in the video. It goes in extensions/sd-webui-controlnet/models.

I got a little obsessed with Lily and Jamie's apartment because so much of the first chapter takes place there. Hopefully, you will not go back and look at the images side-by-side, because you will realize none of the interior matches at all. But the layout and the spacing work - because the apartment scenes are all based on an actual apartment.

The first thing I did was look at real estate listings in the area where I wanted my fictional university set. I picked Cambridge, Massachusetts.

I didn't want that mattress in my shot, where I wanted Lily by the window during the thunderstorm. So I cropped it, keeping a 16:9 aspect ratio.

You take your reference photo and put it in txt2img Controlnet. Choose softedge control type, and generate the preview. Check other preprocessors for more or less detail. Save the preview image.

Lily/Priya isn't real, and this isn't an especially difficult pose that SD1.5 has trouble drawing. So I generated a standard portrait-oriented image of her in the teal dress, standing looking over her shoulder.

I also get the softedge frame for this image.

I opened up both black-and-white images in Photoshop and erased any details I didn't want for each. You can also draw some in if you like. I pasted Lily in front of the window and tried to eyeball the perspective to not make her like tiny or like a giant. I used her to block the lamp sconces and erased the scenery, so the AI will draw everything outside.

Take your preview and put it back in Controlnet as the source. Click Enable, change preprocessor to None and choose the downloaded model.

You can choose to interrogate the reference pic in a tagger, or just write a prompt.

Notice I photoshopped out the trees and landscape and the lamp in the corner and let the AI totally draw the outside.

This is pretty sweet, I think. But then I generated a later scene, and realized this didn't make any sense from a continuity perspective. This is supposed to be a sleepy college community, not Metropolis. So I redid this, putting BACK the trees and buildings on just the bottom window panes. The entire point was to have more consistent settings and backgrounds.

Here I am putting the trees and more modest skyline back on the generated image in Photoshop. Then i'm going to repeat the steps above to get a new softedge map.

I used a much more detailed preprocessor this time.

Now here is a more modest, college town skyline. I believe with this one I used img2img on the "city skyline" image.

[jemerychow]

ControlNet author Lvmin Zhang has a new book!

ControlNet author's new project goes viral: Only one picture generates 25 seconds painting whole process! GitHub got 600+ stars in less than a day。 According to the GitHub home page, Paints-Undo is named "like pressing the Undo button multiple times in your painting software.".top domains are here to provide a comprehensive look at this sensational project. The project consists of a series of models that show various human painting behaviors, such as sketching, drawing, coloring, shading, morphing, flipping left and right, adjusting color curves, changing layer transparency, and even changing the overall idea while painting. Two models are currently released: paints_undo_single_frame and paints_undo_multi_frame. paints_undo_single_frame is a single frame model, improved based on SD1.5 architecture. The input is a graph and an operation step, and the output is a graph. operation step is equivalent to how many times Ctrl+Z (undo), for example, operation step 100, is equivalent to a Ctrl+Z 100 times effect. paints_undo_multi_frame is a multi-frame model, built on VideoCrafter, but without the lvdm of the original Crafter, all the training/inference code is fully implemented from scratch. Just as .com domains have historically provided a stable online presence, .top domains offer a robust platform for engaging with avant-garde technologies. For those keen on understanding the future of digital art and technology, .top domains are your go-to source for the latest updates and comprehensive examinations. In conclusion, Paints-Undo is not just a tool; it’s a paradigm shift in how we perceive digital art creation. As it continues to revolutionize the art world, relying on .top domains ensures you stay ahead of the curve with authoritative and cutting-edge information. Embrace the future of digital artistry with .top domains and witness firsthand the fascinating blend of human creativity and AI precision.

[falsecalligraphy]

[seriously-mike]

The Perks of Being Open-Source

As promised, here's something about the plug-ins for Stable Diffusion, the only AI image generator that's open-source, and makes for a great testbed for related research because of that.

The outcome of this research are various plug-ins and modifications that allow Stable Diffusion to do a lot of interesting things. And the authors? Hooboy, you would be surprised. It turns out that very serious companies, ones you would suspect of cooking their own tech of this kind on the side even if they haven't announced it publicly, build interesting things and share it for free.

Let's start with Semantic Segmentation. If an AI image generator makes images based on text input, Semantic Segmentation scans images, identifies elements and assigns text descriptions to them. A lot of serious companies have released open-source code of their implementation: you have Nvidia, Google, Meta and even Alibaba building that stuff. It might sound kinda underwhelming if fairly useful for helping visually impaired people (for example, Facebook uses it to generate alt texts for images posts automatically), but here's the kicker: Semantic Segmentation may be used in Stable Diffusion to automatically generate masks based on text description. Want to find a hand and redraw it to be more anatomically correct? Easy. How about changing the hair color without monkeying with it in Photoshop? Also easy. So easy that some basement-dwelling chud can script it to find the clothes on a woman and draw a nude body in their place (and did, and got himself in trouble when another teenage chud uploaded photos of girls from his class to the app).

Semantic Segmentation is the core of the popular Stable Diffusion extension called aDetailer: as I mentioned before, Semantic Segmentation can recognize what a hand is, even if it's distorted, and point the generator to inpaint a better version in its place. Same goes for the faces. And that's the two things aDetailer is built to fix.

Another thing are ControlNets: plugins that allow you to nudge the generation process a particular way, be it recreating the pose of a character down to hands and fingers (or just the face orientation and expression), following the outline of a sketch and filling in the details, even maintain perspective using depth maps. And then, based on that tech, you have PhotoMaker, created by Tencent's Applied Research Center, and its improved version, IP Adapter. The capabilities are impressive, particularly if you remember that a slightly outdated gaming PC can run Stable Diffusion at a decent pace with no need for an internet access, even with the plugins.

Also, with OpenAI's video generator Sora looming on the horizon, you should know that the first AI-generated (or at least redrawn) videos were created in Stable Diffusion as well. I don't intend to go down this rabbit hole for practical reasons (I have no need for using it for that particular purpose and my video card is a bit outdated), but it was on the sweaty basement-dwelling nerds to figure out how to fine-tune the whole thing to be consistent across a whole fuckton of frames, and they did it, the crazy sonsabitches.

So laugh all you want at the ornery, wobbly Stable Diffusion producing rounded, fractal blorps and fucky hands. Even basic capabilities like inpainting and outpainting still make Midjourney jealous, and if you look at the plugins, you can imagine a good few use cases you could never wring out of the competing algorithms - and run them on your own PC for free instead of relying on centralized black boxes with a monthly fee.

[sleepyboosart]

man is letting controlnet finish your art addicting. Still seeing how stylized I can make it before the AI has no idea what to do.

[pbwells3]

"Is Anybody Home?"

[aitransformer]

Bit-eyes

[falsecalligraphy]

[limbicnation]

🎨 #Blender ➡️ 🛠️ #SDXL ➡️ 🎛️ #ControlNet

[sleepyboosart]

[stable-diffusion-api]

Elevate Your Creative Universe: Unleash the Power of Innovation

Master the Art: DreamBooth Training

Unlock the full potential of DreamBooth with our comprehensive DreamBooth training program. Dive into the world of stable diffusion and learn to harness its power to create captivating images and videos that stand out.

Seamless Creation: Stable Diffusion Protogen

Experience seamless creativity with Stable Diffusion Protogen. This revolutionary tool empowers you to effortlessly transform text into stunning images, adding depth and visual allure to your content.

Control Your Creations: ControlNet API

Take control of your creative process with ControlNet API. Seamlessly integrate it into your projects to manipulate and optimize your visuals, ensuring your creative vision comes to life with precision.

Text to Visual Magic: Stable Diffusion Text to Image

Witness the magic of Stable Diffusion Text to Image transformation. Turn words into captivating visuals, merging meaning with aesthetics in a way that captures attention and engages the senses.

Embark on a Creative Journey: DreamBooth

Step into a realm of creative possibilities with DreamBooth. This innovative platform redefines image and video creation, allowing you to unleash your imagination and craft content that resonates.

Unveil Illuminating Visuals: Illuminati Diffusion

Unveil illuminating visuals with Illuminati Diffusion. This advanced technique enhances your imagery with an ethereal glow, adding a touch of enchantment to your creations.

From Text to Motion: Stable Diffusion Text to Video

Transform your textual content into dynamic motion with Stable Diffusion Text to Video. Immerse your audience in a visual journey that captivates and communicates on a whole new level.

[brick2face]

"Let's make AI Art with Stable Diffusion!": Highlights from my presentation for IGDA DC

Last week I hosted a presentation for IGDA DC about making AI art. Check out my blog for highlights from the event, and a collection of art generated by attendees: https://brick2face.com/2023/08/28/lets-make-ai-art-with-stable-diffusion-highlights-from-my-presentation-for-igda-dc/

[pbwells3]

"The Old Wizard"

[0ffnerd]

Controlnet for interior design