u/Haravikk

While the new scrub/resilver algorithm in ZFS is great when it's working, I've noticed the performance is heavily dependent upon how quickly it scans the metadata, so it can build up its list of sorted records to issue.

But how quickly ZFS does this seems to vary — sometimes it will scan the entire pool, then proceed to issue extremely quickly, other times for some unknown reason it only scans a fraction of the pool then proceeds to languish for hours before bothering to scan any more.

I'm currently in the process of resilvering (replacing) a drive and it's doing the latter — ZFS has only scanned around 440g of a 6t pool, and the issuing is averaging an underwhelming 4mb/sec to a CMR drive I tested as averaging 120mb/sec sequential.

The lack of scanning ahead seems to be the problem here, because ZFS presumably doesn't have the big blocks of sequential data it needs to do this efficiently yet. But what I can't figure out is why it isn't scanning more?

As I say, I've found the performance varies — sometimes a scrub or resilver is extremely fast. I should have more than enough RAM (64gb) to scan all of the metadata, and I've got scan_mem_lim_fact set to 5 which should allow ZFS to use up to 20% of that RAM (12.8gb) for scanning, yet it just doesn't seem to want to (my current SPL memory usage is only 2gb above my current ARC size).

Is there anything else I can do to push ZFS to scan more aggressively? The system isn't under any real load right now, I've got over 20gb of RAM free, and there are no scrubs/resilvers active on other pools.

It just seems to be completely uninterested in scanning ahead, as the amount scanned has not changed since the initial burst right at the start.

For reference, here's what the result of zpool status currently looks like:

  pool: zdata
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Fri May 22 10:36:06 2026
	442G / 6.03T scanned at 124M/s, 15.3G / 5.74T issued at 4.29M/s
	1.93G resilvered, 0.26% done, no estimated completion time
config:

	NAME                                              STATE     READ WRITE CKSUM
	zdata                                             ONLINE       0     0     0
	  mirror-0                                        ONLINE       0     0     0
	    replacing-0                                   ONLINE       0     0     0
	      media-DF6465F2-797D-3D49-BF95-A1B85C24AC46  ONLINE       0     0     0
	      /dev/sde                                    ONLINE       0     0     0  (resilvering)
	    media-9CFD5094-937D-B645-B08B-25185AC1D836    ONLINE       0     0     0
	  mirror-1                                        ONLINE       0     0     0
	    media-5107EA4B-814B-694F-9265-9253F646C3D2    ONLINE       0     0     0
	    media-452A7F56-0FAE-A045-880E-20931B595196    ONLINE       0     0     0

errors: No known data errors