u/AlyxRoberts

Hey there. So I've been running a media server and I went to add a drive to it at the end of May and long story short, it turns out that one of the drives in my array had been down for a while without me realizing. I know. I have since educated myself about better error notifications.

Okay. Fine. I'll have to resilver the drive that's been offline. This is my punishment.

So one month of resilvering later, I'm ready to add the new drive. Aaaand I've somehow messed this up. The drive that was resilvering is now missing its label and the pool wants nothing to do with it. As far as I can tell it's healthy and has all the data, it's just missing its ZFS label.

I'm at my wit's end here. I don't want to start over, but unless I can figure this out, I'm going to have to start over. Can anyone tell me how I might fix the missing label from my resilvered drive and get my pool to come back?

Okay. Big writeup time. I hope I don't miss anything important.

Server:

• Debian Linux
• OpenZFS 2.3.2
• Pool "poool": 3-wide RAIDZ1 across three 8TB Seagate drives (ST8000DM004)
• Pool was originally created with plain /dev/sd* paths

The basic problem:
My pool is UNAVAIL and won't import due to insufficient replicas. One of the three drives (ZR163NBH) has no readable ZFS labels, even though it successfully completed a resilver less than 24 hours ago.

Technical breakdown of what happened:
Drive ZR163NBH was UNAVAIL due to a cable issue. I used zpool replace -f with its by-id path to kick off a resilver, which completed successfully after 23 days (6.07T resilvered, 0 errors). After the resilver completed, the pool showed DEGRADED with a second drive (ZR165FYD) now FAULTED, also due to the drives re-enumerating between reboots (the pool was created with /dev/sd* paths, so drives shuffling causes ZFS to lose track of them).

While attempting to fix the FAULTED drive, I ran wipefs, sgdisk --zap-all, and dd commands targeting what I believed was ZR165FYD. However, due to the /dev/sd* enumeration shuffling between reboots, I believe I accidentally hit ZR163NBH instead at some point, wiping its ZFS labels.

Current state:
sudo zpool import -d /dev shows:

pool: poool
  id: 3794980518434859057
state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
config:
        poool                                UNAVAIL  insufficient replicas
          raidz1-0                           UNAVAIL  insufficient replicas
            sdb                              ONLINE
            13851791371351368010             OFFLINE
            ata-ST8000DM004-2U9188_ZR163NBH  UNAVAIL

sudo zdb -l /dev/sdd (where ZR163NBH currently lives) returns:

failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3

However, sudo zdb -l /dev/sdb1 (ZR164YVN, the most up-to-date drive) shows a clean pool config at txg 2521027 with ZR163NBH listed as a direct vdev child (no longer in a replacing vdev), confirming the resilver completed successfully:

txg: 2521027
guid: 5504141216371227705
children[2]:
    type: 'disk'
    guid: 3892473579986994429
    path: '/dev/disk/by-id/ata-ST8000DM004-2U9188_ZR163NBH-part1'
    devid: 'ata-ST8000DM004-2U9188_ZR163NBH-part1'
    DTL: 691

What I've tried:

• zpool import -f -m -F -d /dev 3794980518434859057 returns "no such pool or dataset"
• zpool import -d /dev -f -m -T 2479927 poool gives me "no such pool or dataset"
• Various combinations of -d /dev/sda -d /dev/sdb -d /dev/sdd give me "no such pool available"

So I guess it all comes down to the questions down below:

1. Is there a way to restore or reconstruct the ZFS labels on /dev/sdd so the pool can recognize it and import?
2. Is there a way to force import a RAIDZ1 pool with one UNAVAIL member, treating it as degraded rather than un-importable?
3. The data on /dev/sdd should still be physically intact since we only wiped labels/partition tables, not the actual data blocks. Is there a recovery path here or am I going to have to start over?

All services are stopped and I am not writing anything to these drives while I wait for advice. Any help would be amazing.

EDIT: Seems like the answer isn't the one I was hoping for. I guess it might be time to rebuild the media library. Oh well. Learning experience, I guess.