u/Sea-Bee4158

https://preview.redd.it/sp9hj97aad1h1.png?width=1660&format=png&auto=webp&s=a42f309e54d03694542ec4c57bcb6ec140b15d22

Released a position paper today with my co-author Timothy on small-dataset LoRA training. Writeup includes a paired A/B of chained vs monotonic schedules on Qwen-Image with full configs and figures, both models up on HuggingFace.

What's in the paper

The argument: the community has converged on practical hyperparameters but not on what "well-trained" actually means. I argue generalization within the trained concept is the load-bearing quality measure - a LoRA that reproduces its training set perfectly but can't compose flexibly hasn't learned the concept, it's memorized it.

Operationalized as five named failure modes (each tied to existing academic literature), readable off a comparison grid:

1. Base capability degradation (open-world forgetting)
2. Concept narrowing / mode collapse
3. Caption-token rigidity
4. Entanglement leak
5. Visual signature reproduction (memorization)

The grid with a no_lora baseline row and diverse-prompt columns IS the diagnostic.

Chained training

If you trained on SD1.5 in 2022 you probably already used a version inherently on TheLastBen's fast-DreamBooth Colab. Modern trainers (kohya, ai-toolkit, OneTrainer) don't expose this anymore. We reconstruct it with an external watchdog script that edits the trainer's config at predetermined step counts or other methods.

Recipe: rotate through dataset subsets across N phases, then reintroduce the combined dataset for a consolidation pass. Proposed mechanism: intentional intermediate forgetting acts as a regularizer; the consolidation phase has to find a parameter-space basin that averages over the subset-specific commitments.

The A/B finding

Both runs produce competent LoRAs. The differences are subtle, not dramatic, and but a difference does exist.

The cleanest finding is a seed-variance test at the publication checkpoint. On a side-profile prompt that appears in the training set, the chained run produces 4 pose-distinct outputs across 4 seeds while the straight baseline collapses to 4 near-identical outputs lifted from a single training image. Base Qwen-Image with no LoRA varies freely on the same prompt — so the collapse is LoRA-induced, not inherited. Textbook Tell #2 (concept narrowing) signature in the straight run that the chained run avoids.

The prompt-length stress test (Ostris-suggested follow-up) shows a milder effect: on 2-3 word prompts the straight baseline introduces extraneous design elements not present in the chained outputs, consistent with mild Tell #5.

Configs

• Base: Qwen-Image
• Rank/alpha: 42/42
• LR: 5e-5, AdamW8bit, EMA 0.99
• Scheduler: flowmatch
• Caption dropout: 0.35 (244-img anime) / 0.25 (27-img character)
• Trainer: ai-toolkit by Ostris, chained mechanism via external watchdog
• Hardware: RTX 6000 Ada (A6000, 48GB)
• Full YAML in Appendix A

Links

[GitHub page]

Both LoRAs are up on HuggingFace as alvdansen/illustration-1.0-qwen-image and alvdansen/illustration-1.0-qwen-image-baseline if anyone wants to run them.

Part 1 of a multi-model series. Happy to dig into methodology, configs, or the diagnostic framework in the comments.