I've been looking at ways to reduce power consumption on my RTX Pro 6000 GPUs. First some stats:

SGLang, Qwen 3.8 27B, BF16, prefill = 32k prompt (cold), decode = 1024 tokens on a cached context of 105k.

Prefill

Power (W)	Speed (tok/s)	Change (%)
600	5426	-
450	4298	-21
350	3446	-37
300	2949	-46
250	2456	-55
200	1968	-64

Prefill draws whatever power it is allowed, and its speed is roughly proportional to power. Capping prefill saves little energy; it only makes prefill slower. Running prefill at 600 W costs about 8% more energy per token and halves the time.

Decode

Power (W)	Speed (tok/s)	Change (%)
600	66.8	-
450	66.7	0
350	65.2	-2
300	61.9	-7
250	51.3	-23
200	36.8	-45

Decode at 600 W draws only about 475 W. It does not use the full limit, so any cap between 450 W and 600 W changes nothing for decode.

The knee is at about 300–350 W. Down to 350 W, decode loses approx 3% speed while using approx 25% less energy per token than at 600 W. At 300 W it loses approx 8% and uses 30% less energy.

250 W costs a lot of speed for almost no further saving. Decode drops approx 23% against 600 W, but energy per token is only about 1% better than at 300 W. At 200 W, energy per token gets worse again.

Summary

The optimal strategy seems to be: use 600W for prefill, 300W for decode.

I've installed a daemon that monitors SGLang log files, and switches the power cap depending if SGLang is in Prefill or Decode. I'm monitoring this on real-world workloads now.

Is anybody else doing this? I haven't seen this strategy mentioned before.