u/Iajah

Insider does not show the model thinking streaming anymore, most annoying to try catch reasoning errors from your models.

Azahar does not support Surface Duo dual screen out-of-the-box. There is a fork that does, you can download the APK from the release assets: https://github.com/Slion/azahar/releases
That should make it a lot easier to get up and running. It's WIP, rough edges expected.
See also that post.

Surface Duo 2 on GameSir G8+

Can I mount an EEB motherboard in my Dark Base Pro 900 rev2?
Here is the mobo: https://www.asus.com/motherboards-components/motherboards/workstation/pro-ws-w790e-sage-se/techspec/

Do bequiet have a heatsink for LGA 4677?

Having 96GB of VRAM I wanted to run BF16 variant on Windows using LM Studio v0.4.16 instead of the community Q8 variant. I managed to convert the official BF16 safetensors using llama.cpp doing something like:
python .\convert_hf_to_gguf.py ".\models\Qwen_Qwen3.6-27B" --outfile Qwen_Qwen3.6-27B-bf16.gguf --outtype bf16
Had to edit that model.yaml for LM Studio to pick it up.

https://preview.redd.it/683lnwxcn16h1.png?width=760&format=png&auto=webp&s=883acfb3d2ed45bd52bb6df362b53ad187d6897a

It shows as supporting MTP where the Q8 variant did not.

https://preview.redd.it/2azfbbafm16h1.png?width=699&format=png&auto=webp&s=8a17a3fa4928d97fc89761a94df3c5e63b0ef91b

MTP Settings:

https://preview.redd.it/smy8y21fo16h1.png?width=394&format=png&auto=webp&s=a54382178639c8b784276a576ea3125e60e0daf9

I had to lower the context from 128K to just above 100K and reduce concurrency to 1 for it to fit in VRAM. When I tried using KV quant it did reduce VRAM usage estimations but then it failed to load with OOM.

VS Code Copilot agent seems to be running fine so far though I have yet to try it with full context.

I had earlier tried to use Transformers backend through TextGen but could not get it work, it kept OOM on load. I found TextGen is convenient to download the HF safetensors models but there is certainly a less twisted way to get it done.

When counting from 0 to 100 I get those stats:

https://preview.redd.it/p4pjvamfy16h1.png?width=582&format=png&auto=webp&s=67bc6749adcbf7452b4c4f0346a71b869e68d73e

Which slower than what I got with vLLM on Linux. It also feels like it would use less VRAM with vLLM and I could have more context somehow.

I first tried Qwen on Linux using some custom vLLM proxy to get thinking to work in VS Code Copilot Chat and now on Windows using LM Studio as it is much easier to try out various models. I ended up disabling thinking with Qwen as it just does not work well with Copilot, it keeps erroring out as Qwen does some tool call during thinking or something like that. Without thinking I was able to get some serious work done. Like add an SPZ loader to my C++ 3D model viewer for instance. Without thinking Qwen on Copilot can work on its own really well and perform some not so trivial tasks to completion.

However Qwen could not get my splater shader to render properly after a few tries. I thought this is the right time to try out Gemma. Sure enough Gemma seems to be unto something and it feels real fast and thinking is working. However it kept stopping. Not in error but it just decides it wants to stop saying things like I did this and that and we may need to do that next. Sometimes it just stops every two minutes. Really annoying compared to Qwen. Shall I disabled thinking for Gemma too? Now it just stops after saying "I'll read lines 450 to 480 very carefully.". Are you guys having such issues too? And now eventually stopped with an error: "Sorry, no response was returned.". I guess it just does not work well with Copilot either. Maybe I should try Continue.dev.

Switched back to Qwen.

Can we get idle fans support on RTX PRO 6000 Workstation? Having them spin at a minimum of 1200 RPM is a bit pointless. Failing that is there some aftermarket mod that could help with that without having to go for water cooling?

Can I also use it to cool an RTX 6000?

Multi-Token Prediction (MTP) allows the model to predict multiple tokens ahead simultaneously. The num_speculative_tokens parameter controls how many tokens vLLM will speculate on per decoding step:

• MTP 2 (num_speculative_tokens: 2) — predicts 2 tokens ahead, validates both in one forward pass.
• MTP 3 (num_speculative_tokens: 3) — predicts 3 tokens ahead, validating all three together. More speculative tokens yield higher throughput on highly predictable sequences, with diminishing returns on more complex prompts.

Configuration	Predictable/short prompts	Realistic prompt
No MTP	~26 TPS	—
MTP 2	~60 TPS (+131%)	~40–45 TPS (+54–73%)
MTP 3	>70 TPS (+169%)	~40–45 TPS (+54–73%)

That RTX Pro 6K Workstation was running with a 400W power limit. Going to 600W yields minimum gain up to 75 TPS for simple prompts and next to nothing for longer ones. The GPU did not actually draw 600W it remained below 450W AFAICT.

Component	Version
OS	Ubuntu 24.04.4 LTS
Kernel	6.8.0-117-generic
CPU	Intel Core i7-11700K @ 3.60GHz RAM 64GB
GPU	NVIDIA RTX PRO 6000 Blackwell (96 GB) + RTX 5060 (8 GB, display)
NVIDIA Driver	595.71.05
vLLM	0.21.0

Predictable prompt:

Count from 1 to 100, one number per line.

Realistic prompt:

Write a detailed technical blog post (at least 2000 words) comparing the architecture of modern GPU-based LLM inference engines. Cover: vLLM's PagedAttention, TensorRT-LLM, SGLang, and Ollama. For each, discuss memory management, batching strategy, quantization support, and deployment model tradeoffs. Conclude with a recommendation matrix for different workloads.

Prompts were done through VS Code Copilot over a custom python proxy basically doing the translation from vLLM to Copilot. Mostly to be able to show reasoning in Copilot and compute stats.

Here is my config:

Environment="PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True"
Environment="SAFETENSORS_FAST_GPU=1"
ExecStart=vllm serve /models/Qwen3.6-27B \
--served-model-name Qwen3.6-27B \
--host 0.0.0.0 \
--port 8000 \
--dtype bfloat16 \
--gpu-memory-utilization 0.92 \
--max-model-len 196608 \
--max-num-seqs 2 \
--mamba-ssm-cache-dtype float16 \
--mamba-cache-dtype float16 \
--disable-custom-all-reduce \
--chat-template /LLM/chat-templates/qwen3.6-enhanced.jinja \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--override-generation-config '{"repetition_penalty":1.05,"frequency_penalty":0.3,"min_tokens":10}' \
--enable-prefix-caching \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 3}'

I have yet to try it for actual production work but my feeling is that this jump from 26 TPS to 40/70 TPS should make it a lot more usable. It would be interesting to try MTP 4 but seeing at how MTP 3 does not bring anything over MTP 2 for complex prompts I doubt it would be worth it.

Day two working on my local LLM setup. I kept using Qwen 3.6 MoE for testing and development of the VS Code Copilot integration. I had it implement a space invaders in HTML/JS. I got pretty far along with MoE alone until I wanted to fix the asymmetry of the player ship sprite and it just could not work this out even though this is pretty basic math and pattern.

Out of frustration I got the Dense 27b model up and running and sure enough it fixed it on first try. Though Dense it running a lot slower than MoE on my RTX Pro 6000.

Received the GPU today, that's my first local LLM. Had to use a proxy between VS Code and vLLM to get it working. Using customoai in VS Code Insider. Thanks Claude Opus 4.7 for helping me putting it all together in record time. Looking forward to try it some more. First impression: it's fast!

https://preview.redd.it/1uoam3bw601h1.png?width=539&format=png&auto=webp&s=40d0ac35c91dd0379c83f45451a8b49463330f65

https://preview.redd.it/welhy4ar701h1.png?width=567&format=png&auto=webp&s=499a53e8556bafcf40b3f22d187fb72359a076a1

Because the newer fibre optic 5690 only runs OS v8.22 and the 7590 AX runs a newer OS v8.25. When is 8.25 coming to 5690?

https://preview.redd.it/2wygm3y1nhzg1.jpg?width=3072&format=pjpg&auto=webp&s=09ad27ca527f07f22a33005f64105ffea32c908f

Any tips on how to remove that collar without scratching it? There is no screws I can see around the collar.

Will I be able to convert my annual Pro+ to monthly before the June debuff? I'm concerned that if I pick the refund I won't be able to subscribe to monthly Pro+ since it is currently not taking new subscribers. I mean they clearly want to prevent people from keeping on using their annual subscriptions given the silly multipliers. Just hoping they have an upgrade path to usage-based billing rather than just crippling us and locking us out.