u/ArmadstheDoom

So, I'm trying to improve my Lora training for Krea 2, because while I'm able to train basic style/character/concept loras, the problem I have is that they're all too rigid, and not in the way of being overbaked.

As it turns out, Civitai removing all their real people loras entirely decimated the ecosystem and knowledge for training characters that have specific looks or multiple outfits. Which is kind of the issue.

For example, I could easily train just the armor for Iron Man, but the problem is that it bakes in that look, and then when I do what I've done in the past with other models, which is add more variation to the dataset, it only seems to take away from everything. Ergo, iron man suit 1 works, iron man suit 2 works, iron man suit 1 and 2 takes away from both likeness wise.

And this is an issue because I'm not sure how to A. capture the outfit/concept without B. losing the likeness/flexibility. My assumption was to use a trigger word, again like other models, but the issue was that doing so didn't really train much of anything it seemed?

Is this just a dataset issue? Like do you basically just need, like, 20-30 images for each individual concept? Normally a lora might only need around 20 images total, but with more concepts you need more data. But is this a case where you need closer to 100 images, in order to have clear concepts? Or is this a case where having more images means that the model sees each image less, and thus it learns it less well?

Not sure how to really create the flexibility I'm looking for without either A. baking in the costume as concept or B. muddying the training and causing it to learn things poorly.

So I really don't use comfy much; for image generation, my 3090 is mostly fine. 24gb runs basically everything at a decent speed. I've never really been in the market for video, but H3 having come out makes it seem decent enough to experiment with.

Now, I can run it on my pc. the problem is that if I do, I can't really use the PC for anything else, and it takes like 1 minute per second of video. So that makes it kind of unideal to experiment with.

But I don't know if paying for the cloud is worth it, because if it's pay as you go pricing, that means that it's not really financially viable to experiment due to the expense.

I guess my question is: do you use it, if so, is it fast enough that it would be worth it, or would it be roughly the same speed and also too limited in generation to be worth using for anything that's not already prepared professional work?

I'm not against using it, I just don't want to shell out like, $100 just to play with a new toy, you know?

So, yesterday the jailbreak worked for Opus 5 just fine, and for Opus 4.8. Now though it his the safeguards every time, and it says "This model has safety measures that flag specific phrases. This can happen to safe, normal chats" in regards to the project instructions.

So it sounds like they changed something so that it could just detect the jailbreak? Anyone else having this issue when they weren't yesterday?

I should also point out that for me, it seems they removed the 'answer quickly' option for all models, including 4.6?

So I'll be up front with you, until Krea 2, I've mostly avoided natural language models, because there's basically nothing "natural" about their language. The problem I've found is that training models on LLM spouted gibberish means it becomes impossible to really articulate what you want, because most people simply do not think in "natural" language.

But Krea 2 has shown me that it is at least worth investigating, and that has pushed me to consider a question that I've not really seen answered anywhere.

So, to compare, when you want to train with tags, it's very easy to add new concepts. Because if you describe everything but the thing, and then you add a tag it doesn't know, it assigns that tag to the concept of what it doesn't know. It's very easy. It's just labeling.

But I've not found a way to do this with 'natural language' without running into what I call the "drawing the elephant" problem. Imagine it's like 1800, and you've never seen an elephant before, and you have a description. So you then tell 50 people who have also never seen elephants before to draw one based on your description. You would get 50 different drawings, none of which were correct and none of which you could say 'yes, this is the thing' because you've never seen it.

And that's how I feel about trying to train unknown concepts in natural language. Characters? Styles? Concepts that are roughly adjacent to what it knows elsewhere? That I can grasp. But I have yet to find a way to train something that it simply doesn't know with natural language that doesn't struggle with the fact that the data is going to have 50 different descriptions for every image.

So basically, how do you go about training a concept it has no idea about? For example, if the model had no concept of a car, you could in a tag system just tag it as 'car' and it would learn. But how would you do this in a natural language system where every caption would read like:

>"This photograph captures a vintage black Ford Model T roadster parked on a gravel path in a forested area. The car, with its classic design, features a black fabric roof, round headlights, and a yellow license plate reading "SHM 149." The vehicle has large, black spoked wheels with white-rimmed tires and a prominent Ford emblem on the grille. The car's body is smooth and glossy, with visible fenders and a simple, elegant front. Surrounding the car are tall trees with green and yellow leaves, and the ground is covered in gravel and sparse grass. The sunlight casts shadows, enhancing the car's vintage appeal."

I just don't really know how you would do this successfully, and I'm looking for help/guidelines/useful tips.

I've found ones for 4.7 and 4.6, but I've not seen anything for Opus 4.8. Did we just give up on that model entirely or was there something I missed?

So I've realized that as a writer, I mostly have an academic methodology for writing; part of this is my background in history as a discipline. I've written fiction before, but the dialectic story structure isn't something I really vibe with; it's not something that I find engaging.

But recently I learned about things like "Fire & Blood" by George RR Martin, or "Cyclonopedia" by Reza Negarestani.

The problem I have is that, by and large, I'm not huge into worldbuilding; I'm an editor at heart and I demand that everything in my finished works have a reason to exist. It's not my nature to build a kingdom if it's not important.

Basically, is there a market, or even a real interest for this style of writing, outside of RPG sourcebooks? Because I'm not sure how much people like academic writing in general in fictional spaces.

So, this is not a post about showing things off. It's not a hype post. This is just what I've found, and I want to compare it to other people's findings to know whether or not I'm seeing what other people are.

Pros: honestly, not that many? It generates fast enough on a 3090. It has better initial generations than Illustrious does, but that's kind of irrelevant when you need to inpaint anyway. When it comes to fixing problems, being 10% or 50% of the way there is the same; both need fixing.

The pros, by and large, seem to be theoretical or conceptual at the moment. Namely, that it's capable of learning more and doing more than Illustrious. Since Illustrious is based on XL architecture, it struggles to learn details and replicate them well. I often use the marvel test to illustrate this; it can learn a character like She Hulk easily, because their costume is simple and doesn't have a lot of details. It suffers with a character like Luna Snow, whose outfit is asymmetrical and has a lot of tiny details. No amount of training or dataset curation can overcome Illustrious' limitations there.

How well it can handle, say, multiple loras at once is debateable right now, because the loras that are being trained are still in the experimental stage where we don't know the best methodology for settings and the like. Which means that the ability to mix character, style, and concept loras all at once is debateable. Illustrious, being the older model, has more in this regard, so I'm looking for this to be better at this.

Cons: There are a number of cons, and it's down to the things that can also be strengths. For one, the quality tags skew away from things. For example, trained into the 'masterpiece, best quality' tags is a specific anime style, which undermines its ability to do other drawn styles natively. But a bigger problem is that, out of the box, it cycles through styles so much that it makes inpainting impossible.

For example, if you tell it to generate a colored pencil sketch, it will cycle through a dozen different art styles if you try to inpaint. this is pretty unhelpful.

The bigger problem though comes with training and lora making. Testing loras, which again is still new, shows that what kind of tagging the lora uses is the most important thing. What I mean is that if the lora maker used tags rather than captions (and really there is zero reason to ever use captions for drawn models but I digress), then it will actively suffer if you try to use captions.

For example, if the lora is trained on tags, and your prompt reads "one man standing on a white background" the lora will actively degrade. This is a problem conceptually, because it means that mixing loras that use captions and tags individually is going to cause problems. If your style lora uses captions but your character or concept lora uses tags, this is going to cause issues.

Truthfully, I suspect that this means that trying to mix captions and natural language has produced a worse version of both, especially if the loras cause more problems.

The core benefits of anima, right now, are all theoretical. The real problems are that it's imprecise and struggles to adhere to prompts. Natural language is terrible for artistic work, because trying to describe an art style with natural language is very difficult. It turns into purple prose garbage. But worse, it becomes almost impossible to just get what you want.

So, my conclusion so far is that it would be much improved by jettisoning the natural language, but lacking that, for some kind of consensus in lora makers to happen. Namely, that they pick one or the other. Because right now, the model fights itself.

The other thing is that, quite honestly? It's functional. But that's all it is. All it's benefits are theoretical right now, so I think it'll come down to whether or not we can realize those theoretical benefits. Absent that, Illustrious is still easier to use and more consistent in its outputs.