u/MajorZesty

I'm debating picking up a Galaxy XR since it looks like a better option for what I do vs the Steam Frame. My gaming PC is currently running Bazzite with a 9070 XT. Do any of y'all stream from a Linux machine? It worked fine with my Quest 3 and Steam Link, but I'm not sure if sideloading the Steam Link beta is the best option.

TL;DR from the article:

• FP8 via --kv-cache-dtype fp8 remains the best default for KV-cache quantization: it provides 2x KV-cache capacity with negligible accuracy loss, while matching BF16 on most performance metrics and substantially improving them in memory-constrained serving scenarios.
• TurboQuant k8v4 does not provide any significant advantage over FP8: it only provides modest KV-cache savings (2.4x vs 2x) which are not worth the consistent negative impact on throughput and latency metrics.
• TurboQuant 4bit-nc is likely the most practical TurboQuant variant: it helps under KV-cache memory pressure, but trades the extra capacity for moderate accuracy, latency, and throughput costs. It may still be viable for edge deployments where memory is the dominant constraint.
• TurboQuant k3v4-nc and 3bit-nc show meaningful accuracy drops, especially on reasoning and very long-context tasks, while also substantially degrading latency and throughput. This makes them poor candidates for production deployments.

I feel like restricting what the model can do based on allow/deny lists is a losing battle if it has bash access.

Has anyone messed around with the idea of using containers/microvms for boundaries instead? For instance, if we don't want the AI to write back to the project instead of preventing it from using bash we'd stock it in its own VM/container that has the project mounted read only with an overlayfs on top. It can do whatever, but it won't get back to our main code. Then for the actions that we want to modify our project code base we'd use the same container/VM without the overlay and mount it read-write.

I've messed with this a bit, but my personal laptop is customized to the point where my implementation wouldn't work as a general solution. It is possible with bubblewrap in Linux and podman in Linux and macos. I couldn't find a good non-podman solution on macos, which is annoying since I was hoping to use the built-in features. I'm guessing there's a microvm solution there, but I haven't dug into it yet.

I've been using pi for the last month and I'm finding that the AIs have a lot of bias towards certain actions when they don't have enough context. So far I see:

• Make as many things configurable as possible, providing the user with a long list of options that overwhelm
• make the smallest change possible, leaving issues in place when larger reflectors would work better
• migration paths for everything
• keep legacy code instead of removing it, even if it mirrors the updated functionality
• tests strongly coupled to implementation, leaving good coverage but brittle tests. Every change involves updating the code in two or more places
• defaulting to keeping everything in one place. This coupled with making the smallest change seems to prevent the models from being willing to split the code into smaller files.

What else have y'all seen AIs want to do when you're not providing specific instructions in your prompt?

The bottom of my framework 13 gets really hot. I feel like it's gotten worse over time. I'm guessing it also doesn't get any real airflow, so it soaks up heat without a chance to dissipate it properly. I've used multiple main boards over the years. Right now I'm using the 7840u.

Anyone else have this issue? How do you deal with it?

I usually use vertical / 3 way diff views when I'm on my work computer. However, most of my personal use is on a smaller laptop screen. For those that use smaller screens how do y'all handle it? Vertical diffs? What's your setup?