u/Adam161000

I came across a project called AethelStream that proposes virtualizing model weights by streaming them layer-by-layer from SSD to RAM to GPU instead of loading the entire model into VRAM.

​

The idea is to overlap I/O and computation so only the layer currently being executed lives in VRAM, while the rest stays on disk or in RAM. It also uses activation recomputation to reduce memory usage during training.

​

On paper, it sounds like an interesting way to make experimentation with larger models possible on consumer GPUs.

​

I'm curious what people here think:

- Is this technically feasible at scale?

- Would PCIe/NVMe bandwidth become the main bottleneck?

- How does this compare with approaches like DeepSpeed ZeRO, FSDP, or vLLM?

- Are there existing projects doing something similar?

​

I'd love to hear opinions from people who've worked on LLM infrastructure.