OpenShift and item-level restore
Hello all,
I've gotten the fun task of evaluating RH OS for my company's use cases and I am currently in the phase of exploring backup solutions. Specifically, for the virtualization part of the story, namely fileservers.
Now, I've already setup K10 and it's snapshotting happily, but when I try to follow the procedure for item level restore, I keep bumping into the same issue, regardless of the operating system (tried Fedora from template and WS2022 from sysprepepd image).
I've been working with Claude on this, so I asked it to write up a summary, which is found below:
---------------------------------
Start-of-Summary
Environment
- OpenShift Container Platform: 4.22.8
- OpenShift Virtualization (CNV): matching 4.22.x
- Storage: ODF (OpenShift Data Foundation) / Ceph RBD, storage class
ocs-storagecluster-ceph-rbd-virtualization - Kasten K10: reproduced identically on v8.5.8 (OLM/Marketplace install) and v9.0.3 (Helm install)
- VM disks:
volumeMode: Block,accessModes: [ReadWriteMany](required for live migration / failover support) - Guest OS: reproduced on both Windows Server 2022 (NTFS) and Fedora (ext4) — not OS-specific
What fails
Any FileRecoverySession against a restore point sourced from a volumeMode: Block + accessModes: [ReadWriteMany] PVC fails within ~1 minute, regardless of guest OS:
{"message":"getBlkidProperties","function":"kasten.io/bmd/go/flr/session.(*objectAnimatorBase).getBlkidProperties",
"file":"kasten.io/bmd/go/flr/session/animator_common.go:49",
"fields":[{"name":"devOrImagePath","value":"/mnt/v0s0/volume.img"}],
"cause":{"message":"exit status 2"}}
Full chain: getBlkidProperties → Activation error → ActivateMountObjects → 0 new objects → Mount fails → failed mount → failed to start session.
Ruled out as the root cause:
- Guest OS / filesystem type (Windows/NTFS and Linux/ext4 fail identically)
- Partition table complexity (not tested exhaustively, but ruled lower-priority once OS/filesystem was excluded)
- Kasten version (identical failure on 8.5.8 and 9.0.3)
- Export completion status (export reports
Completein all failing cases)
Supporting evidence pointing toward the export/mount pipeline rather than blkid itself: Kanister artifact detail for a failed case showed UploadSize: 0 B for the main VM disk object despite Complete status, and total exported object storage size was on the order of ~100 KB for a disk with several GB of actual guest data.
End-of-Summary
---------------------------------
I did get it working via a Veeam B&R agent, but that's just messy for large-scale management.
I did also get it working via restoring the snapshot to a different Namespace. The VM boots normally and I can restore files via interactive login session.
However, I'm stuck on understanding whether this is how admins are expected to restore files in a RHOS Virtualization environment, what I may have done incorrectly and what kind of a workflow is typical for people using it daily to backup fileserver VMs.
My colleagues are already in the "this is crap" camp, and I'm finding it hard to prove them wrong.
Thanks for any opinions, experiences and advice.