I’ve been building Uisato Studio, a workflow-based AI creation platform for audiovisual work.

This is the Music Video mode: upload an image + audio, and the system analyzes the input, generates visual direction, creates clips, handles b-roll / lip-sync when needed, and assembles everything into a finished music video through a guided pipeline.

I’m trying to move AI video from isolated generation into orchestration; an agentic production system built for more coherent, edit-ready audiovisual output.

I’ve been building this suite for the past year, hope you guys enjoy it: https://uisato.studio/

Made entirely on the "intelligent" implementation of Seedance 2 on Uisato Studio.

For Klein 9B using the qwen_3_8b, the prompt path is basically:

your prompt;

1-wrapped in Qwen chat template

2 - Qwen2 tokenizer

3- Qwen3 8B text encoder

4- hidden layers [9, 18, 27] stacked into conditioning

5- Flux2/Klein transformer cross-attends to that

The local wrapper does this template:

  <|im_start|>user
  YOUR PROMPT<|im_end|>
  <|im_start|>assistant
  <think>

  </think>

So it is not reading your prompt like CLIP tags. It is reading it like an instruction/message.

What It Accepts Well:

It should respond best to natural language with clear relationships:

A woman sitting on a beachfront, looking at the camera, wearing a black dress. The camera is at eye level. Her body is seated facing slightly left. The beach and ocean are behind her.

Strong prompt concepts:

- subject type: woman, man, dog, car

- action/pose: sitting, standing, walking, looking at camera

- location: on a beach, inside a kitchen

- spatial relations: behind her, to her left, in the foreground

- clothing/object attribution: she is wearing, holding, beside

- camera/framing: close-up, full body, eye-level, three-quarter view

- style if phrased plainly: photo, natural lighting, soft shadows

What It Throws Away Or Weakens

The big one: Comfy prompt weighting is disabled for this TE.

So this does not mean much:

((face:1.4)), [body:0.6], (((identity)))

The tokenizer still sees punctuation/text, but the encoder wrapper passes disable_weights=True, so classic CLIP-style

emphasis is not applied as weights.

Also weak:

- giant comma tag soups

- repeated words as fake emphasis

- abstract junk like masterpiece, best quality, ultra detailed

- contradictions: sitting, standing, walking

- vague modifiers not attached to a noun: beautiful, perfect, cinematic

- negative prompt logic, unless the sampler/model path explicitly uses it well

- overly long prompts where important instructions are buried

What Matters Most

Because this is Qwen-style chat encoding, write prompt chunks as sentences with ownership:

Bad:

beach, woman, camera, sitting, black dress, looking, ocean, realistic

Better:

A realistic photo of a woman sitting on a beach. She is looking at the camera. She is wearing a black dress. The ocean is behind her.

For identity/reference workflows "Identity feature transfer", avoid asking the TE to redefine the subject too much. Let the node carry identity, and let prompt carry scene/action:

Keep the same woman. Change only the location: she is sitting on a beachfront, looking at the camera. Natural daylight photo.

Best Prompt Shape For Your Use:

Use this structure:

[identity constraint].

[scene/location change].

[pose/action].

[clothing/body constraint].

[camera/framing].

[lighting/style].

Example:

Keep the same woman from the reference image.
Move her to a sunny beachfront.
She is sitting and looking directly at the camera.
Preserve her face, body proportions, hairstyle, and clothing shape.
Eye-level photo, natural daylight, realistic beach background.

The TE will not literally “obey” every clause, but this format gives Qwen the best chance to encode relationships instead of treating the prompt as a bag of tags.

An experimental temporal ghosting / long-exposure toolkit for TouchDesigner, built for turning prerecorded and real-time footage into smeared, split-exposure, echo-like motion.

The system layers delayed frames, masks the active subject region, and adds optional feedback persistence to generate distorted portrait, face, and full-body trails that sit somewhere between long exposure, temporal rupture, and spectral motion blur.

This release also includes:

— a custom FLUX-2 LoRA trained on experimental photography [the one used in this demonstration]
— the pertinent ComfyUI workflow for FLUX-2.dev + LoRA text-to-image generation

Available now through my Tools Store.

Both music and visuals by myself, deeply inspired by the recent BoC-related events.

Shipped this for the AMD x lablab hackathon. Attached video is one of the actual reels the pipeline produced - one English sentence in, finished mp4 with characters, story, music, and voice-over out. ~45 minutes end-to-end on a single AMD Instinct MI300X. Every model is Apache 2.0 or MIT.

Pipeline (8 stages, all sequential on the same GPU):

1. Director Agent - Qwen3.5-35B-A3B (vLLM + AITER MoE) plans 6 shots from one sentence, returns structured JSON with character bibles, shot prompts, music brief, per-shot voice-over script, narration language
2. Character masters - FLUX.2 [klein] paints one canonical portrait per character. No LoRA training step - reference editing pins identity across shots by construction
3. Per-shot keyframes - FLUX.2 again with reference image. Sub-second per keyframe after warmup
4. Animation - Wan2.2-I2V-A14B, 81 frames @ 16 fps native. FLF2V for cut:false continuation arcs (last frame of shot N anchors first frame of shot N+1)
5. Vision critic - same Qwen3.5-35B reloaded with 10 structured failure labels (character drift, extras invade frame, camera ignored, walking backwards, object morphing, hand/finger artifact, wardrobe drift, neon glow leak, stylized AI look, random intimacy). Bad clips re-render with targeted retry strategies (different seed, FLF2V anchor, prompt simplification)
6. Music - ACE-Step v1 generates a 30s instrumental from Director's brief
7. Narration - Kokoro-82M, 9 languages. Director picks language to match setting (Tokyo→Japanese, Paris→French, Mumbai→Hindi)
8. Mix - ffmpeg with per-shot vo aligned via adelay

Wan 2.2 specifics (the bit this sub will care about):

• 1280×720, not 640×640 default. Costs more but matches what producers want
• 121 frames at 24 fps was my first attempt - gave temporal rippling. Switched to 81 @ 16 fps native (the distribution Wan was trained on) and it cleaned up
• flow_shift = 5 for hero shots, 8 for b-roll (upstream wan_i2v_A14B.py defaults)
• Negative prompt: verbatim Chinese trained negative from shared_config.py. umT5 was multilingual-pretrained against those exact tokens. English translation is observably weaker
• Camera language: ONE camera verb per shot, sentence-case, placed first ("Tracking shot following from behind"). Multiple verbs in one prompt cancel each other out
• Avoid the word "cinematic" - triggers Wan's stylization branch, gives the AI look. Use lens/film tags instead ("Arri Alexa, anamorphic, 35mm film grain")

Performance work:

• ParaAttention FBCache (lossless 2× on Wan2.2)
• torch.compile on transformer_2 (selective, the dual-expert MoE makes full compile flaky) - another 1.2×
• AITER MoE acceleration on Qwen director (vLLM)
• End-to-end: 25.9 min → 10.4 min per 720p clip on MI300X

Why a single MI300X: 192 GB HBM3 lets a 35B MoE, 4B diffusion, 14B I2V MoE, 3.5B music, and a TTS share the same card sequentially. Same stack on a 24 GB consumer GPU would need 4-5 boxes wired together.

Code (public, Apache 2.0): https://github.com/bladedevoff/studiomi300

Hugging Face (documentation, like this space 🙏) https://huggingface.co/spaces/lablab-ai-amd-developer-hackathon/studiomi300

Live demo on HF Space is temporarily offline while infra restores - should be back within hours. In the meantime the showcase reels in the repo are real pipeline outputs, no human re-edited shots.

Happy to dig into AITER MoE setup, FBCache tuning, FLF2V anchoring, or the vision critic's failure taxonomy in comments.

This is my 1st post on reddit so apologies in advance for any mistake i make in my post.

I have been probing the flux.2 klein 9b model for some time and based on my findings i have created a lot of nodes for better photorealism and consistency. This one in particular node is a combination of many different nodes i have created and utilises many different techniques. The main objective for creating this was identity consistency with a bit of realism.

I have very primitive knowledge about python so this node has been created through vibe coding but it still took like 3 AIs and 1.5 weeks to get the work done.

The node act as a balancer between input reference image and prompt and it adjusts accordingly to give you a balance between both identity and the creativity.

Just some inportant info: i have tested this only on flux.2 klein 9b FP8 distilled version. i have limited resource of vram (rtx 2060) so the testing was limited but i stopped when i thought i got good results. i exclusively used normal ksampler not the custom or advance ones so i have no idea about their impact. I have attached screenshot of Jason Statham in various scenes using prompts from chatgpt. i hope this is allowed.

https://github.com/Magirad/Flux_ID_Adjuster/

special thanks to https://www.reddit.com/user/Capitan01R-/ as i was able to solve some tricky issues by referring to his enhancer node pack.

---------------------------------------------------------

For people getting bad skin texture try changing the identity_blocks  6-15 or 8-16. Flux processes texture during the 17-23 blocks. the default 8-19 blocks works better to artistic themes.

as suggested by https://www.reddit.com/user/skyrimer3d/ use LCM/beta for better facial consistency.

I’ve been retouching jewelry photos for a while and honestly it’s the hardest thing I’ve ever edited. Reflections pick up everything, dust becomes boulders, and keeping gold looking like actual gold across dozens of shots is brutal. I got obsessed with how big brands like Tiffany or Mejuri keep their entire catalog visually cohesive so I started experimenting with AI, not to replace the craft but to speed up the boring parts.

What surprised me most is that once you have a clean consistent dataset of a single stone, training a LoRA on a specific brand's lighting style actually works. You can make a diamond look like it was shot in their studio, same warmth, same shadow depth, same mood. It's wild.

I ended up shooting 100 frames of the same emerald cut diamond at 4K because I needed a perfect base to train from. It made such a difference that I wanted to share it, not to sell anything, but because I wish someone had told me earlier that the quality of your training images matters more than the prompt. If you're stuck fighting inconsistent source material, the AI can't learn the subtleties.

Anyway, just wanted to share what I've been tinkering with. If anyone else here retouches shiny reflective stuff I'd love to know your pain points. This niche is lonely.

I've been experimenting with FLUX.1-Fill-dev lately and kept running into the same wall: the Fill model is great for mask-based edits, but there's no built-in way to feed it a ControlNet signal (depth, canny, pose, etc.) at the same time.

So I built one.

The idea is simple:
FLUX Fill handles the mask-based edit, while ControlNet guides the structure using inputs like depth, canny, pose, tile, blur, gray, or low-quality conditioning. This makes the inpainting more controlled, especially when you want the generated object or edit to follow a specific structure or composition.

Since FLUX.1-Fill-dev was not originally trained jointly with ControlNet, this is more of an experimental/community implementation. In practice, it works well for structured inpainting, but results depend a lot on the mask quality, control image alignment, and conditioning strength.

Links

• Personal Repo : https://github.com/pratim4dasude/pipline_flux_fill_controlnet_Inpaint
• Pipeline file (Diffusers community): https://github.com/huggingface/diffusers/blob/main/examples/community/pipline_flux_fill_controlnet_Inpaint.py
• Community Pipelines README (FLUX Fill ControlNet section): https://github.com/huggingface/diffusers/tree/main/examples/community#flux-fill-controlnet-pipeline
• FLUX Pipelines docs: https://huggingface.co/docs/diffusers/api/pipelines/flux
• ControlNet in Diffusers docs: https://huggingface.co/docs/diffusers/api/pipelines/controlnet_flux

Code example

    import torch
    from diffusers import FluxControlNetModel
    from diffusers.utils import load_image
    from pipline_flux_fill_controlnet_Inpaint import FluxControlNetFillInpaintPipeline
    
    dtype = torch.bfloat16
    device = "cuda"
    
    controlnet = FluxControlNetModel.from_pretrained(
        "Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro-2.0",
        torch_dtype=dtype,
    )
    
    fill_pipe = FluxControlNetFillInpaintPipeline.from_pretrained(
        "black-forest-labs/FLUX.1-Fill-dev",
        controlnet=controlnet,
        torch_dtype=dtype,
    ).to(device)
    
    img  = load_image("imgs/background.jpg")
    mask = load_image("imgs/mask.png")
    ctrl = load_image("imgs/dog_depth_2.png")
    
    result = fill_pipe(
        prompt="a dog on a bench",
        image=img,
        mask_image=mask,
        control_image=ctrl,
        control_mode=[2],                    
    # canny=0, tile=1, depth=2, blur=3, pose=4
        controlnet_conditioning_scale=0.9,
        control_guidance_start=0.0,
        control_guidance_end=0.8,
        height=1024, width=1024,
        strength=1.0,
        guidance_scale=50.0,
        num_inference_steps=60,
        max_sequence_length=512,
    )
    
    result.images[0].save("output.jpg")

If you find this useful, a GitHub star ⭐ would really help support the project.

The new ecosystem for AI orchestrated filmmaking.

A full year of the heaviest experimentation I’ve ever put myself through, now avaialble.

Link: https://uisato.studio/

I created a website where you can use Flux ai for free. All you have to do is watch ads to earn credits and then use them to generate images.