Is your AI agent's data actually safe?
AI agents have already evolved into real business tools, running workflows and generating revenue. Yet, clear standards for protecting agent data, prompts, and outputs are still missing.
These risks are already visible across the industry:
- Providers can revoke access or change terms at any time
- Failures can lead to data loss or deletion
- Accounts, workspaces, or histories can become unavailable unexpectedly
- Agents themselves can overwrite or corrupt the data they manage
What’s needed is a standard: mandatory, regular backups stored in a reliable system built for these needs.
DeNet has long been building decentralized storage - secure, sovereign, and resilient by design. Today, this is becoming critical not just for individuals and businesses, but for AI systems as well.
In the meantime, we want to hear from you: how do you currently secure your AI agent’s data? Your experience will help shape what comes next.
u/DeNet_Storage — 21 days ago
▲ 51 r/LocalLLaMA
DifussionGemma 4 on 4x7900xtx
Just got 100 tps on generation, but in total time it around 45-60 t/s in case of prompt processing waiting.
Available memory show:
GPU KV cache size: 152,671 tokens
Maximum concurrency for 131,072 tokens per request: 1.16x
amd-smi monitor for this gpu:
GPU XCP POWER GPU_T MEM_T GFX_CLK GFX% MEM% ENC% DEC% VRAM_USAGE
3 0 183 W 82 °C 84 °C 3036 MHz 100 % 5 % N/A 0 % 23.6/ 24.0 GB
5 0 161 W 81 °C 88 °C 3101 MHz 100 % 0 % N/A 0 % 23.7/ 24.0 GB
7 0 165 W 78 °C 86 °C 3095 MHz 100 % 1 % N/A 0 % 23.7/ 24.0 GB
8 0 154 W 80 °C 88 °C 3090 MHz 100 % 0 % N/A 0 % 23.6/ 24.0 GB
# DiffusionGemma 26B on vllm dgemma branch (4x 7900 XTX)
set -uo pipefail
docker run --name "$1" \
--rm --tty --ipc=host --shm-size=32g \
--device /dev/kfd:/dev/kfd \
--device /dev/dri/renderD131:/dev/dri/renderD131 \
--device /dev/dri/renderD133:/dev/dri/renderD133 \
--device /dev/dri/renderD136:/dev/dri/renderD136 \
--device /dev/dri/renderD135:/dev/dri/renderD135 \
--device /dev/mem:/dev/mem \
--security-opt seccomp=unconfined \
--group-add video \
-e HIP_VISIBLE_DEVICES=0,1,2,3 \
-e ROCR_VISIBLE_DEVICES=0,1,2,3 \
-v /mnt/tb_disk/llm:/app/models:ro \
-v /mnt/tb_disk/llm/torch_compile_cache:/root/.cache/vllm/torch_compile_cache \
-v /opt/services/llama-swap/moe_configs/E=128,N=176,device_name=AMD_Radeon_RX7900XTX.json:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/fused_moe/configs/E=128,N=176,device_name=AMD_Radeon_RX7900XTX.json:ro \
-e TRUST_REMOTE_CODE=1 \
-e OMP_NUM_THREADS=8 \
-e PYTORCH_TUNABLEOP_ENABLED=1 \
-e GPU_MAX_HW_QUEUES=1 \
-e VLLM_ROCM_USE_AITER=0 \
-e VLLM_ROCM_USE_AITER_MOE=0 \
-e VLLM_USE_V2_MODEL_RUNNER=1 \
-e PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:256 \
-p "$2":8000 \
--entrypoint vllm \
vllm-dgemma:nocompile \
serve \
/app/models/models/vllm/diffusiongemma-26B-A4B-it \
--served-model-name "$1" --host 0.0.0.0 --port 8000 --trust-remote-code \
--gpu-memory-utilization 0.65 --tensor-parallel-size 4 \
--tool-call-parser gemma4 --enable-auto-tool-choice \
--reasoning-parser gemma4 \
--attention-backend TRITON_ATTN \
--max-num-seqs 2 --max-model-len 131072 \
--generation-config vllm \
--hf-overrides '{"diffusion_sampler": "entropy_bound", "diffusion_entropy_bound": 0.1}'
So it's work, but to launch it we spend 2-3M of deepseek-v4-pro tokens to prepare docker image.
u/djdeniro — 25 days ago
▲ 0 r/ROCm
vLLM + Step-3.7-Flash-FP8 R9700 seeking optimization
At 100 req i got 800 t/s output speed, but let's go deeper:
i have an config to launch step 3.7 flash for fp8 quntization, and got around 35-37 t/s for one concruency request, do we have any suggestion to get more speed?
MTP does not working, got only 12 t/s output speed. I use Triton kenrels.
Thanks! Bellow my launch coinfig:
#!/bin/bash
docker rm -f "$1-cached" 2>/dev/null || true
docker run --name "$1-cached" \
--rm --tty --ipc=host --shm-size=128g \
--device /dev/kfd:/dev/kfd \
--device /dev/dri/renderD128:/dev/dri/renderD128 \
--device /dev/dri/renderD129:/dev/dri/renderD129 \
--device /dev/dri/renderD130:/dev/dri/renderD130 \
--device /dev/dri/renderD132:/dev/dri/renderD132 \
--device /dev/dri/renderD137:/dev/dri/renderD137 \
--device /dev/dri/renderD138:/dev/dri/renderD138 \
--device /dev/dri/renderD139:/dev/dri/renderD139 \
--device /dev/dri/renderD140:/dev/dri/renderD140 \
--device /dev/mem:/dev/mem \
-e HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-e ROCR_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-e VLLM_ROCM_USE_AITER=0 \
-e PYTORCH_TUNABLEOP_ENABLED=1 \
-e PYTORCH_TUNABLEOP_TUNING=0 \
-e PYTORCH_TUNABLEOP_RECORD_UNTUNED=0 \
-e PYTORCH_ALLOC_CONF=expandable_segments:True \
-e PYTORCH_HIP_ALLOC_CONF=expandable_segments:True \
-e TRUST_REMOTE_CODE=1 \
-v /mnt/tb_disk/llm:/app/models:ro \
-v /home/denet/scripts/moe_configs_best:/moe_configs:ro \
-e VLLM_TUNED_CONFIG_FOLDER=/moe_configs \
-p "$2":8000 \
vllm/vllm-openai-rocm:nightly \
/app/models/models/vllm/Step-3.7-Flash-FP8 \
--attention-backend TRITON_ATTN \
--served-model-name "$1" --host 0.0.0.0 --port 8000 --trust-remote-code \
--tensor-parallel-size 8 \
--disable-cascade-attn \
--reasoning-parser step3p5 \
--enable-auto-tool-choice --tool-call-parser step3p5 \
--enable-prefix-caching --gpu-memory-utilization 0.95 \
--max-num-batched-tokens 4096 \
--enable-expert-parallel --max-model-len 262144 --max-num-seqs 128 --enable-expert-parallel \
--override-generation-config '{"max_tokens": 16384, "temperature": 0.7, "top_p": 0.95}'
u/djdeniro — 1 month ago
vLLM + 8XR9700 + DS-V4-FLASH - SUCCESS!
Got DeepSeek-V4-Flash running on 8× Radeon AI PRO R9700 (RDNA4 / gfx1201) — first RDNA4 datapoint I've seen
Spent the day getting DeepSeek-V4-Flash (284B/13B MoE, FP4 experts) up on 8× R9700 with vLLM ROCm nightly, TP=8 + EP=8, VLLM_ROCM_USE_AITER=0. As far as I can tell nobody's run this on RDNA4 before — the official recipes mark every AMD SKU unsupported, and all the upstream work is MI300/MI350 (gfx9).
Surprisingly, almost the whole stack already worked on gfx1201 out of the box on the latest nightly: TP/EP over RCCL, all the mHC TileLang kernels, FP4 MoE via the triton_unfused path, fp8 KV cache. Everything degrades to triton/torch correctly when AITER is off — except one hard raise in the sparse-attention indexer (it assumes AITER-only on ROCm). Redirecting that to the existing triton/torch indexer was the single change that unblocked end-to-end inference.
Worth noting: VLLM_ROCM_USE_AITER=1 is NOT a fix on RDNA4 — it segfaults even earlier in the AITER ck_tile RMSNorm, since gfx1201 isn't in AITER's arch table. So triton/torch is the only viable route here right now.
Now generating correct output (screenshot — it one-shotted a playable HTML5 platformer 🍄). Currently tuning throughput; writing it up for the vLLM tracker so RDNA4 folks have something to start from.
8× R9700 = 256 GB for ~$ a fraction of a single datacenter card, and it runs a frontier MoE. RDNA4 for local LLM serving is more viable than people think — happy to share the launch command / patch if anyone's on the same boat.
I wait in this community someone who also have 8x same GPU
u/djdeniro — 1 month ago