Download speed significantly lower than hardware limits
Hey guys, I've been trying to solve this problem on my own for awhile and can't figure it out. My download speeds through sabnzbd are significantly lower than any limit on the hardware throughout my entire path.
Hardware:
1x 256GB SSD (used for app config storage, not media)
4x 10TB Western Digital Red Plus CMR 512mb cache media pool (On WD website claims up to250MB/s transfer rate)
5 Gigabit ethernet NIC + 5 gigabit ethernet service (speedtested ~4-4.5gigabit)
Truenas Operating system using raidz2 storage structure for media
app config storage on ssd is a solo drive no mirroring but is replicated to media pool nightly
Location: US
Originally I was just using thundernews (unlimited data account with 50 connections max) as my sole main provider with a much lower prio block account on a different provider that is basically never used. I was reaching max speeds of 80MB/s with thundernews using 50 connections. I tried adjusting number of connections to check if it was related to overhead so I did tests at every 5 connections. Lowering connections only made download speed worse. Turning off SSL did not affect download speed at all. I monitored hdd usage using zpool iostat on the truenas shell and the hdd's are sitting idle a majority of the time, same with cpu, so they do not seem to be the bottleneck right now.
I then added a second equal priority provider in Newshosting (unlimited data + 30 connections). This using 30 connections and thundernews still using 50 connections immediately jumped my download speed to ~110MB/s. a near 50% increase but still a far cry from my hardware bottleneck which I assume to be my hard drives. The immediate jump in download speed also signals to me that hardware is not my current limitation, as I did not change hardware at all to reach higher speeds. I don't expect to actually hit the 250MB/s stated rate of the hard drives, but 110MB/s still seems very low.
I know I can move over to ssd's for downloading however they are significantly more expensive for the amount of storage. I've thought about setting up a temporary small storage (~1tb) ssd for downloading and then creating a nightly job that moves downloads from the ssd to the hdd's. I could do this but before I do this I wanted to try and exhaust all my easy options with my current setup since attempting this will complicate the setup a bit more and might be more prone to errors.
Does anyone have any suggestions on why I'm not hitting higher download speeds with this setup? I've seen multiple people say that they have reached 500+ MB/s speeds with thundernews so I'm surprised that I seem unable to go higher with just thundernews or with double providers (yes I know they are using ssd's but I'm not hdd limited yet).
EDIT 1:
forgot to put in original post that I have already done sabnzbd test file and checked sab speed test of download/complete folder. the download folder values change pretty significantly on every refresh:
System performance (Pystone) 623271AMD Ryzen 7 5800X3D 8-Core Processor AVX2
Download folder speed 117.7 MB/s /data/usenet/incomplete
Complete folder speed 154.4 MB/s/data/usenet/complete
Internet Bandwidth 439.42 MB/s3515.36 Mbps
Source [https://sabnzbd.org/tests/test\_download\_10GB.nzb](https://sabnzbd.org/tests/test_download_10GB.nzb)
Download Downloaded in 1 min 28 seconds at an average of 109.2 MB/s
I also ran an fio job on my harddrives and got ~297MB/s which is roughly 2.5x faster than the download speed I'm seeing through sabnzbd:
fio --name=randwrite --rw=randwrite --bs=256k --size=20G --iodepth=16 --filename=/mnt/data/media/testfile --direct=1
randwrite: (g=0): rw=randwrite, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=psync, iodepth=16
fio-3.33
Starting 1 process
randwrite: Laying out IO file (1 file / 20480MiB)
note: both iodepth >= 1 and synchronous I/O engine are selected, queue depth will be capped at 1
Jobs: 1 (f=1): \[w(1)\]\[100.0%\]\[w=221MiB/s\]\[w=885 IOPS\]\[eta 00m:00s\]
randwrite: (groupid=0, jobs=1): err= 0: pid=1912518: Sun Jun 28 12:56:06 2026
write: IOPS=1131, BW=283MiB/s (297MB/s)(20.0GiB/72421msec); 0 zone resets
clat (usec): min=579, max=284015, avg=881.55, stdev=3771.58
lat (usec): min=581, max=284017, avg=883.14, stdev=3771.59
clat percentiles (usec):
| 1.00th=\[ 586\], 5.00th=\[ 594\], 10.00th=\[ 594\], 20.00th=\[ 603\],
| 30.00th=\[ 611\], 40.00th=\[ 619\], 50.00th=\[ 709\], 60.00th=\[ 750\],
| 70.00th=\[ 799\], 80.00th=\[ 832\], 90.00th=\[ 1057\], 95.00th=\[ 1319\],
| 99.00th=\[ 1582\], 99.50th=\[ 1811\], 99.90th=\[ 33817\], 99.95th=\[ 69731\],
| 99.99th=\[191890\]
bw ( KiB/s): min=104448, max=421376, per=100.00%, avg=289655.20, stdev=83868.08, samples=144
iops : min= 408, max= 1646, avg=1131.46, stdev=327.62, samples=144
lat (usec) : 750=58.87%, 1000=30.53%
lat (msec) : 2=10.16%, 4=0.16%, 10=0.08%, 20=0.06%, 50=0.05%
lat (msec) : 100=0.05%, 250=0.03%, 500=0.01%
cpu : usr=0.38%, sys=48.68%, ctx=83201, majf=4, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,81920,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=283MiB/s (297MB/s), 283MiB/s-283MiB/s (297MB/s-297MB/s), io=20.0GiB (21.5GB), run=72421-72421msec