r/zfs

Background

After setting up a syncoid backup on my main zpool, I realized rsyncing my desktop to a zpool is janky and horrible. Instead of reinstalling Void with zfsbootmenu, I decided to do it in-place; /home already has backups, so worst consequence is I reinstall anyway.

File-level backups give me the heebie-jeebies. Several years ago I trusted Deja-Dup/Duplicity, and it couldn't handle the sheer size of my backup when I needed a full restore. rsync is leagues better but still an extremely slow, potentially imperfect copy that can fail.

Here's what I did:

---

• Original state: 80 GiB / and 300 GiB /home on a single 1 TB ext4 partition

• Boot into hrmpf and install ZFS to the livecd environment

• Shrink the ext4; e2fsck; create a full-size partition for a single-vdev pool with -o mountpoint=none and a /home dataset with -o mountpoint=/home

• Mount the / partition to e.g. /mnt/root

• rsync -aHAXP --numeric-ids /mnt/root/home/ /home/

• (ideally verify transfer); rm -rf /mnt/root/home

• umount /mnt/root

• Shrink the ext4 to file size + buffer; 150 GiB in my case; e2fsck

• Make a third partition; zpool add

• Create child datasets with syncoid:no-sync=true for cache/junk files

• Chroot into /mnt/root to ensure ZFS is installed (don't have 30 old kernels like me or it'll take hours!), and to add zpool import -a and zfs mount -a to rc.local or whatever systemd uses.

• Set and verify UID/perms on all the mountpoints

---

Results

It was quite a slow and dangerous operation, and there was one instance of metadata corruption after the second shrink, which e2fsck fixed. Also nearly fat-fingered the new partition overtop the others. However, it worked out in the end. Striping across the two vdevs should be safe because they're partitions on the same NVMe anyways. If I did reinstall, the transfer still would've taken at least 12 hours because AT&T Fiber LAN is limited to 10/100.

---

Killer App: syncoid

If you haven't tried syncoid (which comes with sanoid), you should. It's a fancy script wrapper around zfs send | zfs receive. By default, it replicates a dataset (+ child datasets with -r) to a dataset on another pool along with snapshots; by default it creates a temporary snapshot to synchronize and resume interrupted transfers.

The big advantage is that I can create exact copies of my pools without any configuration, and use the same command in reverse to restore the pool if necessary. It does NOT copy pool attributes because that would interfere with the target dataset's needed attributes.

In my case, my workstation can now run

syncoid pool/home glados:slab/home-backup

In turn, GLaDOS has an @hourly crontab to run

syncoid -r slab backup/slab

When you need to recover a pool/dataset, simply zfs destroy pool/home if necessary and run syncoid in reverse

syncoid glados:slab/home-backup pool/home

Im trying to get it working

Im at this point in the guide

By default ZFS kernel modules are loaded upon detecting a pool. To always load the modules at boot:

echo zfs > /etc/modules-load.d/zfs.conf
By default ZFS kernel modules are loaded upon detecting a pool.
To always load the modules at boot:
echo zfs > /etc/modules-load.d/zfs.conf

I dissabled secure boot to get the previous step to work, this is my input:

~$ sudo echo zfs > /etc/modules-load.d/zfs.conf
-bash: /etc/modules-load.d/zfs.conf: Permission denied

Hello. I'm not too knowledgeable on ZFS as my use case is relatively straightforward. I have 3 18TB drives in raidz1. I recently setup Scrutiny and noticed one of the drives is failing. 'zpool status' still looks fine, no issues there. But I'd rather not wait for the drive to fail completely. Thankfully the drive is still under warranty, and I have already begun an RMA. However, my concern is over the process of actually replacing the drive. I've been searching online and the more I look the more unsure I am about the process.
Is it better to get the new drive first, and connect that to my server while the failing drive is also still connected? And then do 'zpool replace'? Or should I do 'zpool offline' for the failing drive and remove it before I get the new one? I have 4 SATA ports on my motherboard so connecting the new one with the other 3 wouldn't be an issue.
For the failed drive, should I be concerned about privacy? I only have TV shows and movies on the array, but I'd still rather the vendor not be able to look at the data. What's generally recommended here? If it's recommended to wipe the drive, how should I do that? I'd rather not wipe it while it's still connected to the server.
Thanks.
Edit - well the vendor already got back to me and send they're not able to ship a new drive until the old one is returned. In that case I guess my only option is to 'zpool offline' and then 'zpool replace' correct?

So I have 2 severs with a PowerVault ME5 each. Both servers are in different locations, so no FC connection between both ME5212s.

My idea was to use ZFS as filesystem to be able to use ZFS replication over ethernet for specific datasets.

Now the ME5 itself has builtin raid functionallity I have to use.
So I will have one big disk represented to my servers each backed by a hardware raid.

My main features I want to use is replication of datasets / subvolumes over ethernet and the possibility to enable compression to some subvolumes.

Should I give ZFS a chance in this setting or better go for BTRFS in this case?

Should I consider 80% on mirror-0 a concern even though the overall pool is only 32% full? And would there be a meaningful performance/reliability disadvantage (fragmentation on mirror0?) to a situation such as:

mirror-0: 90–100% used
mirror-1: 10–20% used

while the overall pool still has plenty of free space?

I'm particularly interested to know if there is a "vdev-level utilization threshold" that you guys recommend for striped mirrors, as opposed to the usual "keep the pool below 80%" advice.

The workload is mostly cold/immutable data, but new data is continuously added over time.

I'm installing ubuntu 26.04.01 end of month with ZFS.

AI is leaning towards only putting /home and other such parts on ZFS and having the boot stuff on ext4 [even ext4 preferred to btrfs].

Would you agree?

this file server has been sitting for a few years since I let TrueNas try to migrate from bsd to linux and the pool got broken. went back to BSD version, so it's TrueNAS bsd 13.1 now. there may have also been some additional bungling.

[root@micro /tmp]# zpool import    pool: cargo      id: 16478687762405515984   state: UNAVAIL status: The pool was last accessed by another system.  action: The pool cannot be imported due to damaged devices or data.    see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-EY  config: cargo                                                      UNAVAIL  insufficient replicas   mirror-0                                                 UNAVAIL  insufficient replicas     disk/by-partuuid/d0e0014d-59c7-11ea-9d20-3cecef102d24  UNAVAIL  cannot open     1005513936371914804                                    OFFLINE   mirror-1                                                 DEGRADED     16411949616967839104                                   OFFLINE     ada0                                                   ONLINE

There is a bad label. I find the issue a bit overwhelming and today I engaged with gemini pro AI to help, and it did some exploration and recommended a low level disk write to fix the issue. I stopped short of writing the disk

If anyone's game to help me out with this I'd appreciate if you looked over the diagnostic process I did with the AI and let me know if you think the solution it produced is well supported.

here is the chat:

https://share.gemini.google/6OJMBfe434MF

and here is the output of those last commands:

------------------------------------

LABEL 0

------------------------------------

version: 5000

name: 'cargo'

state: 0

txg: 28352142

pool_guid: 16478687762405515984

errata: 0

hostid: 808464432

hostname: 'micro'

top_guid: 11704010057385088391

guid: 1005513936371914804

vdev_children: 2

vdev_tree:

type: 'mirror'

id: 0

guid: 11704010057385088391

metaslab_array: 38

metaslab_shift: 34

ashift: 12

asize: 5999022833664

is_log: 0

create_txg: 4

children[0]:

type: 'disk'

id: 0

guid: 17375504288854665353

path: '/dev/disk/by-partuuid/d0e0014d-59c7-11ea-9d20-3cecef102d24'

phys_path: 'id1,enc@n3061686369656d30/type@0/slot@4/elmdesc@Slot_03/p2'

vdev_enc_sysfs_path: '/sys/class/enclosure/6:0:0:0/Slot 03'

DTL: 410

create_txg: 4

children[1]:

type: 'disk'

id: 1

guid: 1005513936371914804

path: '/dev/disk/by-partuuid/31953669-59f2-11ea-9d20-3cecef102d24'

phys_path: 'id1,enc@n3061686369656d30/type@0/slot@3/elmdesc@Slot_02/p2'

vdev_enc_sysfs_path: '/sys/class/enclosure/6:0:0:0/Slot 02'

DTL: 442

create_txg: 4

features_for_read:

com.delphix:hole_birth

com.delphix:embedded_data

labels = 0 1 2 3 

I'm looking at installing GhostBSD along side Linux. Linux is using BTRFS for its root partition and I have a subvolume for data that I'd like GhostBSD to have access to. BTRFS isn't going to work for BSD so I'm thinking of moving just the data volume to ZFS. How sensitive is Linux these days to updates with kernel changes and compatibility with OpenZFS? Should I just create an EXT4 volume for the share between them rather than ZFS?

Hey r/ZFS! We’re the team at 45Drives, and we’re hosting a free educational webinar on Wednesday, August 12, covering the fundamentals of OpenZFS.

If you’re new to ZFS, looking to strengthen your understanding, or just want a refresher, our Educational Team will walk through:

• RAID types and storage pools
• How ZFS combines a volume manager and file system
• The ZFS storage hierarchy
• Managing ZFS through the 45Drives Houston UI
• Live Q&A with our technical team

📅 Wednesday, August 12, 2026
🕒 3:00–4:00 PM ADT

Register for the free webinar:
https://attendee.gotowebinar.com/register/88845394642749791

Making inroads into ZFS recently reading what I can get my hands on, but still pretty new.

I have an experimental OpenZFS zpool with a single HDD as sole vdev. It so happens, I have to return the drive for book-keeping reasons. The pool is early days and every metaslab 126 or larger shows up in zdb -mmmm as

    metaslab    126   offset  1f800000000   spacemap      0   free      16G
                      segments          1   maxsize     16G   freepct  100%
    In-memory histogram:
                     34:      1 *
    Flush data:
    unflushed txg=0

Now, I could just leave the drive be and call it a day, encryption being enabled from the get-go. But since I'm learning about ZFS I was wondering: Could I zero out the device up to the aforementioned offset 1f800000000 and all data would be gone? Instead of going through it all.