Tune my Strata setup for longer context: 131K, 262K first, then 512K if it holds up. I'm on Strata v<FILL VERSION, e.g. 0.1.39>.

My hardware:
- GPU(s): <model + VRAM each, e.g. 1x RTX 4090 24 GB / 2x RTX 5090 32 GB>
- System RAM: <e.g. 32 GB / 192 GB>
- CPU: <model, cores>
- Platform: <bare metal or VM (Proxmox/ESXi/etc.)>, OS: <distro/kernel>
- Storage for model files: <NVMe/SATA, model>

Model: Qwen3.8-Flash-Next. Suggest a quant after reviewing my hardware.
REPO: https://github.com/Niko1221/Strata

Ground rules:
- Read docs/DETAILS.md in the Strata repo first, especially the sections on --max-context, --kv, --kv-resident, --prefill, the expert cache, multi-GPU split, and "Context extension past 262K (rope scaling, EXPERIMENTAL)". Go by the docs and `strata --help` for my installed version, not by memory. If a flag below doesn't exist in my build, tell me; don't invent one.
- Change one thing at a time. Before you change anything, record a baseline of my current config: decode tok/s on a short prompt, decode tok/s and prompt-read speed at depth (one ~100K-token prompt, then one near the target context), plus a needle-recall check at depth. Run every candidate the same way and alternate baseline and candidate at least twice, so drift doesn't show up as a gain.
- Watch host RAM and VRAM on every GPU during each run (nvidia-smi, free -g). Report the minimum free RAM, not just the speed.
- Don't run a big compile while the Strata server is loaded. Stop it first.
- Back up the working config before every edit, and give me the exact undo.

What a single-4090, 32 GB RAM box (IQ2_XS) learned, as starting points to test, not answers. Scale these to my hardware:

1. 262K is native for this model. At 262K, --kv q4_0 kept the KV cache in VRAM and stayed fast; recall held at ~238K. If I have more VRAM than that box (24 GB), check whether int8 or k8v4 KV fits at 262K. Measure both against q4_0; higher-precision KV is worth it if it fits.
2. Past 262K needs rope scaling: --rope-scaling yarn --rope-scale 2 for 512K, together with --max-context 524288 and a streamed KV cache (--kv-resident <tokens>, e.g. 32768) so most of the KV lives in host RAM. A 477K-token prompt read in ~150 s and decoded at ~85 tok/s at that depth on that box. It's marked experimental, so run a recall test at 400K+ before trusting it. Estimate how much host RAM streamed KV needs at 512K for my setup and tell me up front if my RAM can't support it.
3. --prefill auto (default) beat --prefill auto:32768 by about 3x on long prompt reads for us. Measure prompt-read speed for any prefill setting you try.
4. "fit_max_tokens": true in the server config keeps requests within the context.
5. If I have more than one GPU: check how my version splits layers, experts and KV across the cards, and whether auto placement accounts for each card's PCIe link (confirm each GPU's link width/gen with nvidia-smi -q and lspci -vv). Tell me where the KV ends up at each context size. Skip this if single-GPU.
6. If I'm in a VM: check whether the guest sees RAM as one NUMA node, whether hugepages or THP are in use for the expert and KV arenas, that vCPUs are pinned, and that GPUs show their full PCIe link inside the guest. Report what you find; don't change the hypervisor host without asking me. On bare metal, still check NUMA layout and THP/hugepages.

If I mention a stat that used to show up and is now missing (e.g. "VRAM hit rate"), check the changes between my previous and current version (git log, CHANGELOG, server /health, /props, /metrics, and the engine log) and tell me whether it was renamed, moved, hidden when experts are all resident, or removed. Quote the commit or line that answers it.

Deliver: a table of each config tried (context, KV type, kv-resident, rope, prefill) with short and deep decode tok/s, prompt-read speed, recall pass/fail, min free RAM and per-GPU VRAM; a recommended daily config and an optional 512K config (or a clear "not viable on this hardware" with the reason); and the exact config diff plus undo for each.

Posting for all the new users and current.

how to use the tuning prompt

1. copy the whole prompt block above.
2. fill in the hardware section at the top: gpu model and vram per card, system ram, cpu, bare metal or vm, os, and what drive the model lives on. also put your strata version in (run `strata --version` if you're not sure).
3. paste it into an agent that can run commands on your box. codex or claude code both work. use opus 5.5 or astra if you have access. this prompt has it reading the repo docs, running a bunch of benchmark passes, checking numa/pcie/hugepages, and diffing git history, and the smaller models tend to lose the thread halfway through or start making up flags.
4. let it run the baseline first. don't skip this. without a baseline you can't tell if a change helped or if your box just warmed up.
5. expect it to take a while. every config gets run at least twice against the baseline, and long-context prompt reads at 262k+ aren't fast.
6. when it's done you get a table of every config it tried, a recommended daily config, an optional 512k config (or a straight answer that your hardware can't do it), and the exact diff plus undo for each change.

notes

- the numbers in the prompt come from a single 4090 with 32 gb ram. they're starting points, not targets. your results will differ.

- 512k uses rope scaling and is marked experimental. trust it only if the recall test at 400k+ passes.

- streamed kv at 512k leans on system ram. if you're on 32 gb or less, 262k is probably your ceiling.

- if you're in a vm, it'll report on the host side but won't touch your hypervisor without asking.

- back up your config before you start anyway. the prompt tells it to, but don't rely on that alone.

EDIT: if you use the guide, please just post a quick update if it helped your setup. Reach out if you have any issues please.

512K context on a single RTX 4090 within the low budget RAM

total vram + sys ram = 50GB

Active strata contributor, working k8v4. Ask whatever you want

OS: Arch Linux
WM: Hyprland
Shell: Quickshell (custom, bar/launcher/notifications/OSD/lock)
Terminal: Kitty
Color engine: wallust
Wallpaper: awww
File manager: Thunar
Font: Inter
Config: https://github.com/T-Crypt/aphotic-hypr

[deleted]

Every Bash/Read/Edit call your Claude Code session makes lands on a live, zoomable node graph as it happens. 200 nodes across 2 active sessions, rendered in real time.

Scrub back through the whole run with the replay transport.

dotfiles: https://github.com/T-Crypt/aphotic-hypr/releases/tag/v1.2.0

OS: Arch Linux
WM: Hyprland
Shell: Quickshell
Terminal: Kitty
Bar/Launcher/Notifications/OSD/Lock: all custom Quickshell modules
Color engine: wallust
Wallpaper: awww
File manager: Thunar
Font: Inter
Config: https://github.com/T-Crypt/aphotic-hypr

OS: Arch Linux
WM: Hyprland
Shell: Quickshell
Terminal: Kitty
Bar/Launcher/Notifications/OSD/Lock: all custom Quickshell modules
Color engine: wallust
Wallpaper: awww
File manager: Thunar
Font: Inter
Config: https://github.com/T-Crypt/aphotic-hypr

I’ve been part of the rice community since around 2022, back when Vaxry first announced Hyprland.

At the time, I was a main contributor to Hyprdots, a project that became fairly well known in the Hyprland community. I spent a ridiculous amount of time developing themes, maintaining the repo, building rofi modules, and generally learning Arch Linux the hard way.

And early Hyprland was rough. Updates breaking AUR packages, official branches, Hyprland-Nvidia branches, Qt changes… you name it. A lot of my early Linux experience was fixing things that broke five minutes after I had finally gotten them working.

Eventually life got busier, and I stepped away.

Then Qt6/QML projects like end-4 and Caelestia caught my attention and pulled me back in.

Aphotic-Hypr started from my old “dots”, including a theming system built around pywal. It has now evolved into something much larger.

The goal isn’t really to make dotfiles for you.

It’s to make something you can make yours.

I’m 30 now, have three kids, and work full time. I still spend my weekends at my desk working on personal projects, homelab stuff, development, security, and Linux but I don’t have the time I had years ago to spend 8 hours getting my desktop functional before I can actually start using it.

That isn’t because I can’t do it anymore. I just don’t want to.

And honestly, I think that’s where AI can be useful.

If you don’t like AI, that’s completely fine. Block, move on, and enjoy your setup. I’m not trying to convince anyone otherwise.

For me, AI means I can spend less time fighting configuration and more time actually building things.

Aphotic-Hypr is my attempt at creating a bridge between AI, gaming, development, and security while still keeping the thing that made ricing fun in the first place: customization.

I also want this to become a community-driven project over time, not just another collection of configs that gets abandoned after the screenshots stop getting attention.

If that’s your kind of thing, take a look.

If it isn’t, that’s fine too.

At the end of the day, it’s Arch.

dotfiles: https://github.com/T-Crypt/aphotic-hypr

So I had quite the morning.

Started off as usual just heading to my desk for work (WFH) and I sit down and fire up the PC. Shortly after logging in, my screen started flickering. My immediate reaction was to cut the system off so I could figure out if it was electrical or coolant. I don’t typically have issues with my PC and have never had a leak in one of my loops.

I found that the CPU inlet was barely leaking droplets of coolant onto my PCIe riser. Threw some paper towels in there to get the immediate liquid out and moved my PC to find a mini pool of coolant under it.

I’m not sure when it started because I hadn’t noticed it due to the placement of the leak but I immediately started taking it apart.

Dried everything off, removed the riser cable completely, put 99% ISO on the whole PCIe lane for the GPU and verified cables at the bottom of the PC. Left it to dry off most of the morning and I started to bend new tubes (Acrylic).

I had to swap to a horizontal config so I didn’t have to make a run to Microcenter (2hrs) and get a new riser so the GPU currently does NOT fit in the Lian Li O11 Dynamic.

I did all of this before 10am. The bends came out quite clean I think and Im liking the clear tubing.

(providing ‘before’ vertical GPU mount reference and ‘after’ horizontal mount reference photos)

TLDR: CPU inlet had a leak, water dripped onto the PCIe riser cable, GPU flickered, and I rebuilt the loop