loras?

#1
by ItzPingCat - opened

what loras do you use with anima? my anima stuff always is terrible, huh

I don't typically use any. I try to use LoRAs as minimally as possible because even well-trained ones have more effect than people realize and they cook outputs very quickly. The two I use are Turbo and Highres Aesthetic Boost, both in the recommended resources on Civit. But neither work quite as well on 1.0.

I make no claims that Anima is perfect. It's pretty jagged; good in some ways and then surprisingly deficient in others. But ready-for-battle's critique is unsubstantiated and doesn't track at all with the way I understand things.

That said I made quite a few technical blunders in this response and I'm gonna be rewriting it completely in the form of an actual blog post at some point.

I have a mildly interesting question for you! Based on your name, do you know or use any inflation LoRAs for anima

Yeah it’s a funny question going off of your username, but genuinely curious here

No! I actually spoke to the person who made the one I used for SDXL and he said it's on his to-do list though.
That said it's actually pretty good at it out of the box. You DO need to finagle it though.

good at it out of the box

You are a wizard, how?????
All I get are either highly degraded outputs, or amorphous blobs of flesh.

good at it out of the box

You are a wizard, how?????
All I get are either highly degraded outputs, or amorphous blobs of flesh.

I mean it's not excellent.

The thing about Anima in general is that while its text encoder uses a modern language model, that language model is very dumb. It helps to give it concise sentences and try not to use terms that could be misconstrued. Neither tags nor natural language are enough on their own, you gotta mix them. It takes to mixing them pretty well. Repeating tags in natural language sometimes helps, sometimes causes issues. Think of the tags as disambiguating the natural language prompt, which is seen as the actual directions being given.

Adaptive Projected Guidance can also help sometimes. Positive Aligned Guidance (f.k.a. Mahiro) sometimes works but can sometimes do more harm than good. Both are native Comfy nodes, IIRC.
Here's a quick dingus I threw together to demonstrate the basics
https://allura.moe/blog/inflatebot/assets/workflows/anima-inf.json

The hard part is compound shapes, so full-body inflation, gets complicated fast. That's where a LoRA would definitely help. Single-part you can make work.

I think if a NegPiP implementation came out for Anima, it'd become magical. Maybe I'll poke hako-mikan about that, although MMDiTs are more sophisticated than the old U-nets.

fuck, maybe I'll vibe-code it lol.

What’s a NegPIP, APG, PAG?
All I know is clip text encode lmfao

APG and PAG are hacks on the way the model's own attention mechanism affects the output.
Average Projected Guidance I don't actually know the mechanism of, but Positive Aligned Guidance basically tries to make the output shift more towards the positive prompt, rather than the difference between the positive and negative prompt.

NegPiP was a technique that let you embed negative weights into a prompt directly, rather than relying on CFG (which, if you didn't know, is what allows us to use negative prompts in the first place.)

See, negative and positive prompts are almost always different lengths, and the way they're calculated together doesn't take token positions into account. So more or less, all of the negative prompt affects all of the positive prompt. It's functional, but coarse. NegPiP was a very effective fix and I'm sad it got ignored. It didn't have the best PR so people thought it was basically the same as using the negative prompt.

It was great for surgically removing confabulations. This is because of how attention works. Basically, tokens that are closer together are associated more strongly. So by having a negatively weighted token right next to a token that's frequently confabulated, you more or less eliminate the association between the token you specified, and the token it gets confabulated with. NegPiP enabling this was actually extremely powerful.

It was especially useful for characters that have multiple forms, where you'd sometimes get form B where you wanted form A and vice versa. You could just put the qualities of form B as negatively weighted. Problem solved, you didn't have to touch the negative prompt, and you didn't affect the image composition as severely.

Unfortunately, it never got implemented for anything newer than Flux.1.

Thanks

Yeah I went and tested it, and it's great for single part.
But yes, it just can't do compound shapes. When's the roadmap for that LoRA?

He did not give any unfortunately. He's not the most technical so he's limited by his local setup.

I decided to have a go at training one myself since I'm something of a data hoarder so I have plenty of samples. If that turns into anything you'll be the first to know :3

Update: It went very well!!

I goofed up and targeted Preview3 instead of 1.0 though, so I'm running it back (with a bit more data.)

AY nice! Is it only for furries or does it do anime characters well?
If you dont have enough data for anime I can provide some

It only has furry data in there explicitly but Anima biases heavily towards human characters anyway. With any luck it won't anthro-fy characters like Tigerpillow's sometimes tends to. It worked okay in both cases but I should test human characters more.

I trained it very sparsely (~60 epochs, not much for a dataset this size) because Anima knows the general concepts and only needed help combining them.

(To be clear I do have human/anime data on hand, I'd just have to dig for it. The furry dataset was already sorted for an older project. But thank you for the offer!!)

So I went and tested it around and it has a few things to be desired.
It HEAVILY biases towards outie navels to the point that I need to add "pregnant, outie navel" to the negative prompt.
And yes, it may randomly furrify your character.
Also slightly distorts the face for human characters (probably because of only furry data)

I got some questions for you!

  1. How big (in images) is your dataset?
  2. Do you have any anime inflation images in it? I don't mean any human images, I mean ones that are in anime style.

Hey! I'm continuing this over on the LoRA repo.

Sign up or log in to comment