A filesystem benchmark focused on corruption, snapshots, rebuilds and near-full behavior

I thought this might be useful to people here who run ZFS, btrfs, bcachefs or traditional md/LVM storage.

I maintain modern-fs-benchmark, a benchmark suite focused on behavior that is often missing from simple mkfs + fio comparisons:

- Injected corruption, scrubbing and self-healing

- Degraded operation and rebuild

- Snapshot aging, scaling, deletion and space reclamation

- Near-full and hard-ENOSPC behavior

- Compression, encryption and reflinks

- Fsync tail latency and responsiveness under load

The current matrix contains 26 configurations, including multiple ZFS, btrfs and bcachefs layouts, plus ext4 and XFS over md, LVM and dm-integrity as classic-stack baselines.

One test deliberately corrupts data on one redundant device behind the filesystem, runs a scrub and verifies the file contents. This demonstrates the difference between redundancy that can identify the correct copy through checksums and redundancy that only knows its copies disagree.

Dashboard:

https://bartosz.fenski.pl/modern-fs-benchmark/

Experimental real-hardware results:

https://bartosz.fenski.pl/modern-fs-benchmark/real-hw/

Source and methodology:

https://github.com/fenio/modern-fs-benchmark

An important caveat: the main dashboard uses loop devices on GitHub-hosted VMs. It is useful for correctness results, behavioral differences and trends, but absolute throughput should not be treated as a hardware ranking. The separate hardware dashboard contains three completed runs from a two-NVMe machine.

The source is Apache-2.0 licensed and the result datasets are CC BY 4.0.

Suggestions for additional data-hoarding workloads, failure scenarios and storage layouts are welcome. If an existing test treats a filesystem unfairly, I consider that a benchmark bug.

reddit.com
u/bfenski — 23 hours ago
▲ 30 r/linux

I built a Linux filesystem benchmark for corruption, snapshots, rebuilds and ENOSPC across 26 storage layouts

Most filesystem benchmarks measure throughput on a freshly formatted single device. That is useful, but it misses many of the reasons people choose btrfs, ZFS or bcachefs in the first place.

I built modern-fs-benchmark to examine modern filesystems as complete storage systems, including their features, failure modes and behavior over time.

The current matrix contains 26 configurations across btrfs, ZFS, bcachefs, ext4 and XFS over md/LVM, dm-integrity, native and LUKS encryption, parity layouts, and XFS on a ZFS zvol.

Hosted CI dashboard:

https://bartosz.fenski.pl/modern-fs-benchmark/

Experimental real-hardware dashboard:

https://bartosz.fenski.pl/modern-fs-benchmark/real-hw/

Apache-2.0 licensed source and complete methodology:

https://github.com/fenio/modern-fs-benchmark

The raw benchmark datasets are published under CC BY 4.0. Every result records the kernel and filesystem tool/module versions. CI artifacts also contain a full command trace, so the exact workload is inspectable rather than hidden behind a chart.

Why I started this project

Many filesystem benchmarks follow a familiar pattern: create a filesystem with mostly default options, mount it, run fio or another generic workload, and compare throughput.

General-purpose suites such as the Phoronix Test Suite are useful for conventional performance comparisons, but they do not focus on the machinery that makes modern copy-on-write filesystems interesting.

I wanted a benchmark designed around btrfs, ZFS and bcachefs as multi-device storage systems rather than treating them as interchangeable replacements for ext4 on a single freshly formatted disk.

That means testing behavior such as:

- Redundancy and degraded operation

- Snapshot aging, scaling and reclamation

- Compression and encryption

- Reflinks and clone divergence

- Fsync tail latency and responsiveness under load

- Rebuild and scrub behavior

- Near-full and hard-ENOSPC behavior

- Data integrity and recovery from corruption

Ext4 and XFS over md, LVM and dm-integrity are included as classic-stack baselines so the costs and benefits of integrated CoW designs can be compared with layered alternatives.

Silent corruption is one particularly important example. When one redundant copy is deliberately corrupted behind the filesystem, checksumming filesystems such as btrfs, ZFS and bcachefs can identify the damaged copy and recover from a valid replica.

Traditional md/LVM redundancy without data checksums can notice during a scrub that its copies disagree, but it cannot determine which one is correct. In my tests, some classic configurations returned corrupted data successfully without an application-visible error.

The dm-integrity configuration shows that a classic layered stack can obtain integrity protection too, with a measurable performance cost.

An important limitation

The main dashboard uses loop devices on GitHub-hosted VMs. Absolute throughput numbers and small differences between filesystems should not be interpreted as hardware rankings.

The hosted runs are primarily useful for:

- Correctness and integrity outcomes

- Comparisons within the same job

- Large behavioral differences

- Snapshot-aging and near-full shapes

- Trends across repeated runs

Real disks are required for meaningful absolute performance, concurrency scaling, device parallelism and mixed-media topologies.

I am aware of this limitation and do not want the hosted dashboard to suggest more precision than the underlying environment can provide.

The real-hardware experiment

Kent Overstreet, the creator of bcachefs, made one of his Hetzner machines available for a real-hardware experiment.

The server had two physical NVMe devices. It completed three full benchmark runs, which are available in the separate real-hardware dashboard. These runs provided meaningful absolute measurements and exposed concurrency behavior that cannot be observed when several loop devices share one virtual disk.

During the fourth run, one NVMe controller entered the kernel’s dead state. The existing machine environment had its operating system on RAID0 across the two NVMe devices, so the hardware failure also made part of /nix/store unreadable and eventually prevented new SSH sessions.

The incomplete fourth run is not being published as benchmark data. This was a failure of the underlying hardware, not a result attributable to any filesystem being tested.

I am grateful to Kent for providing the machine and making the real-hardware experiment possible. Without that access, the three existing hardware runs would not exist.

Where I would like to take it

Better hardware would not merely produce more reliable throughput numbers. It would enable an entirely new class of tests designed for multi-device and multi-tier filesystems.

I would eventually like to run the suite on a machine containing several storage classes, for example two HDDs, two SSDs and an NVMe device.

That could support scenarios such as:

- HDD, SSD and NVMe baselines using identical workloads

- bcachefs foreground and background targets

- ZFS HDD data vdevs with SSD special vdevs

- Separate ZFS L2ARC and SLOG experiments

- LVM dm-cache in writeback and writethrough modes

- Metadata and small-block placement on faster media

- Foreground latency during background migration

- Contention between fast and slow storage tiers

- Degraded operation and rebuild under application load

- Performance before, during and after promoting or evacuating a storage tier

These are the kinds of scenarios for which multi-device and multi-tier filesystems are built, but they cannot be represented honestly when every “device” is a loop file backed by the same cloud disk.

I am considering either renting a suitable dedicated server or eventually building and hosting my own machine. Providers such as Worldstream offer configurations close to what I need, but the recurring cost is currently outside the project’s budget.

For now, the benchmark will remain in its hosted-CI form for an unknown amount of time. The existing dashboard will continue to be useful for correctness, behavioral comparisons and regression tracking, but it cannot answer every real-hardware performance question or model complex mixed-media topologies.

I am also open to running the suite on hardware provided by someone else. A useful environment would need Linux root access, clearly identified block devices that may be wiped, and enough uninterrupted access to complete repeated runs. The hardware description, methodology and resulting data would remain public.

I would appreciate technical feedback:

- Which current tests are misleading or unfair?

- Which failure scenarios are missing?

- Which mixed-media topologies would be most useful?

- Which additional filesystems or layered stacks should be included?

- Which results deserve deeper investigation?

The methodology, implementation and raw results are public. If a filesystem is being tested in a way that misrepresents it, I consider that a bug in the benchmark.

reddit.com
u/bfenski — 1 day ago
▲ 10 r/bcachefs+1 crossposts

NASty 0.0.14

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

The highlights:

🪪 Active Directory — both directions

NASty can now join an existing AD domain as a member, or be the domain controller itself. As a DC you get integrated DNS and Kerberos, user / group / computer management from the web UI, on-box domain backups, and RSAT compatibility so the Windows admin tools work against it. (Experimental, single-DC for now.)

♻️ Backup restore + disaster recovery

Backups finally got their missing half. Restore a whole snapshot back onto your filesystem — including disaster recovery onto a bare box: point a fresh install at an existing backup repository, pick a snapshot, and pull it back.

🔥 Built-in firewall

An engine-managed nftables firewall, deny-by-default, with per-service source/interface restrictions and custom port rules for anything running outside NASty's service model (IPv4 + IPv6, TCP/UDP, port ranges).

⚡ RDMA over RoCE & InfiniBand

Optional zero-copy transports — iSER, NVMe-oF/RDMA and NFS-RDMA — for RoCE and InfiniBand NICs. The controls only appear when you actually have a capable card.

🖥️ SR-IOV & hardware passthrough

IOMMU-group view, vfio-pci toggles that survive reboots, per-device passthrough, and SR-IOV virtual-function management — VF count plus per-VF VLAN, MAC, trust and spoof-check.

📊 Operations control center

One page to see and drive every bcachefs array operation across your pools — start or cancel a scrub, watch a device evacuation, pause or resume reconcile and copy-GC — each row labeled with its pool.

…and a lot of smaller stuff

  • Calculate folder sizes on demand right in the file browser
  • .env files for Docker Compose stacks
  • Remove leftover Docker networks from the UI (e.g. one an app brought with it)
  • Secure Boot moved to lanzaboote v1.1.0 — validated on real UEFI + TPM2 hardware
  • Linux 6.18.38 and bcachefs 1.38.8
  • iSCSI portal management, VM disk/network editing, and a big pile of fixes + dependency refreshes
github.com
u/bfenski — 9 days ago

Benchmarking what modern filesystems promise, continuously

I started working on this recently because I found out that most if not all filesystem benchmarks still test the way it was done when ext2 was state of the art: one disk, default mkfs, raw throughput. But nobody picks btrfs, ZFS or bcachefs for throughput - you pick them for snapshots, redundancy, checksums, self-healing. Almost nobody publishes numbers on those.

So this suite benchmarks the machinery instead, across 17 configurations (btrfs/ZFS/bcachefs plus ext4/xfs over md/LVM as classic baselines, plus encryption variants): aging under 100 snapshots, corrupting a device behind the filesystem's back and checking whether scrub repairs it, failing a device and timing the rebuild, filling to hard ENOSPC, fsync tail latency, and "how long until my prompt comes back while a big cp runs in the background".

Yes, it runs on ephemeral GitHub runners with loop devices - I know what that means. Absolute MB/s is meaningless there, so the suite is built around shapes, ratios and verdicts, with per-VM calibration probes (lemon runners get auto-rerun) and conclusions drawn from trends over many runs. Real hardware with tiered topologies (NVMe cache over rotational disks, special vdevs) is the next step. I just need to gather hardware for it first.

What I'm really asking for is your eyes. Every number on the dashboard links to a description of exactly what ran, with what parameters, and to the code responsible - so if I picked a wrong mount option, an unfair default, a workload that misrepresents your favorite filesystem, or a broken measurement, you can find it and call it out. Several of the current tests exist because people poked holes in earlier results, and honestly that feedback improved this more than anything I did alone.

Dashboard: https://bartosz.fenski.pl/modern-fs-benchmark/
Repo: https://github.com/fenio/modern-fs-benchmark

reddit.com
u/bfenski — 10 days ago
▲ 42 r/zfs+1 crossposts

Benchmarking what modern filesystems promise, continuously

I started working on this recently because I found out that most if not all filesystem benchmarks still test the way it was done when ext2 was state of the art: one disk, default mkfs, raw throughput. But nobody picks btrfs, ZFS or bcachefs for throughput - you pick them for snapshots, redundancy, checksums, self-healing. Almost nobody publishes numbers on those.

So this suite benchmarks the machinery instead, across 17 configurations (btrfs/ZFS/bcachefs plus ext4/xfs over md/LVM as classic baselines, plus encryption variants): aging under 100 snapshots, corrupting a device behind the filesystem's back and checking whether scrub repairs it, failing a device and timing the rebuild, filling to hard ENOSPC, fsync tail latency, and "how long until my prompt comes back while a big cp runs in the background".

Yes, it runs on ephemeral GitHub runners with loop devices - I know what that means. Absolute MB/s is meaningless there, so the suite is built around shapes, ratios and verdicts, with per-VM calibration probes (lemon runners get auto-rerun) and conclusions drawn from trends over many runs. Real hardware with tiered topologies (NVMe cache over rotational disks, special vdevs) is the next step. I just need to gather hardware for it first.

What I'm really asking for is your eyes. Every number on the dashboard links to a description of exactly what ran, with what parameters, and to the code responsible - so if I picked a wrong mount option, an unfair default, a workload that misrepresents your favorite filesystem, or a broken measurement, you can find it and call it out. Several of the current tests exist because people poked holes in earlier results, and honestly that feedback improved this more than anything I did alone.

Dashboard: https://bartosz.fenski.pl/modern-fs-benchmark/
Repo: https://github.com/fenio/modern-fs-benchmark

reddit.com
u/bfenski — 10 days ago
▲ 14 r/bcachefs+1 crossposts

NASty 0.0.13

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This release leans hard into three things: running VMs, getting your data back, and actually seeing what your disks are doing.

🖥️ Virtual machines grew up

  • Attached virtual disks now survive a host reboot — they stay put.
  • Starting a VM tells you the truth: if it can't launch, you get a real error instead of a fake "running."
  • Edit a VM's network (mode + bridge) and resize its disks right from the VM card.
  • Guest tools are always-on, with a per-box toggle for VMware/Hyper-V environments, and a manual disk-type override for when detection guesses wrong.

💾 Get your data back, two ways

  • Whole-subvolume rollback to a snapshot — quiesce, swap, resume, done.
  • File & folder restore — pull individual files or directories out of a snapshot without rolling back the whole thing.

🔍 See what your storage is doing

  • Live progress for scrub / evacuate / rebalance, straight from bcachefs.
  • An operations panel to cancel a scrub/evacuate or pause reconcile/copygc.
  • Stable device ordering + a guided force-remove flow.
  • New bundled CLI tool diskwatch — a read-only disk-diagnostics TUI (devices, SMART, IO, hot files, bcachefs-aware) for triage over SSH or the built-in terminal.

📦 Docker apps, smoother

  • Multi-port apps route correctly now (no more surprise 502s).
  • TLS certs for app ingress hosts are handled on deploy.
  • Published app ports show up on the Firewall page, with port ranges collapsed sensibly.
  • Host-network deploys, port-range parsing in the paste-a-docker-run creator, and a bunch of app-form polish.

🎨 UI & system

  • New opt-in Common navigation menu — a curated short-list of the pages you actually use, toggle back to Full anytime. Theme switch moved into your profile menu.
  • Custom bridge MTU now works correctly on top of bonds (jumbo frames welcome).
  • SMTP notifications work against no-auth relays.
  • Latest kernel (6.18.37), plus the usual security and dependency housekeeping.

Grab the ISO from the release page. Storage with attitude. 😈

reddit.com
u/bfenski — 22 days ago
▲ 10 r/bcachefs+1 crossposts

NASty 0.0.12

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the sharing & visibility release. NASty learns to hand files out and to show what it's doing. Guest file sharing turns any file or folder into a public link — with expiry, password, download caps and folder-as-ZIP — for people who have no NASty account. SMB shares become macOS Time Machine destinations that auto-appear in the Time Machine picker. Compose stacks gain engine-owned startup ordering with inter-stack delays, and a new always-on system-status band in the sidebar surfaces exotic array operations (device evacuation, scrub, reconcile) as they happen. Round it out with sortable columns across the data tables, bcachefs 1.38.6, journal visibility for every engine subprocess, and a sweep of fixes.

Headline changes

  • Guest File Sharing. Share a file or folder under /fs with a public link for someone who has no NASty account. Create it from the Files page with an optional expiry, password and download limit; folders download as a streamed ZIP (built on the fly, never buffered in RAM). The link is the credential — only its SHA-256 is stored, so it's shown once at creation and can't be retrieved later. Recipients land on a clean no-login page; downloads are always served as attachments (shared content can't render on the app origin) and every unavailable reason — expired, revoked, over-limit — returns the same generic message, so a token guesser gets no oracle. Manage and revoke links under Sharing → Guest Shares.
  • macOS Time Machine destinations. A single toggle turns an SMB share into a Time Machine target: NASty applies the Samba vfs_fruit options Time Machine needs and advertises the share over mDNS (_adisk), so it auto-appears in System Settings → Time Machine → Add Backup Disk with no manual mounting. Optional size cap; pair it with a subvolume quota as a hard backstop.
  • Compose stack startup ordering + inter-stack delays. Let NASty own boot startup for the compose stacks you opt in: they come up in the order you choose, with a configurable settle delay after each — for the common case where a "network" stack must create shared Docker networks before the stacks that depend on them. Managed stacks are pinned to restart: "no" via a generated compose override (your compose file is left untouched) so Docker doesn't race the engine; a stack that fails to start is logged and the sequence continues. Drive it from Apps → Compose Startup Order.
  • Persistent system-status band in the sidebar. An always-visible band under the logo — green Healthy, amber for activity (a device evacuating, scrub or reconcile running), red for critical — so there's no mistaking that an array operation is in flight while you navigate. Click to expand the running operations and alert counts.

Sharing & files

  • Guest shares: public metadata / password-unlock (verify-once → short-lived grant cookie, rate-limited) / single-file download / folder ZIP, plus a management page with status, download/view counts, and revoke; removed shares are kept for audit and hidden behind a "Show removed" toggle. The whole surface lives as a tab on the Sharing page alongside SMB/NFS/iSCSI/NVMe-oF.
  • Time Machine SMB shares are validated as authenticated + writable, with the _adisk advertisement managed dynamically as shares are enrolled/removed.
  • Folders created in the Files browser are now writable through the sharing layer — they get share-friendly permissions on create, so SMB forced-users and NFS squashed uids can write into a freshly-made folder.

Apps & Docker

  • Compose startup ordering + settle delays, with the restart: "no" override and an ordered, failure-tolerant boot sequence run off the boot-phase budget.
  • Removing a compose stack now also clears its metadata sidecar, so a stack later recreated with the same name can't inherit stale startup/ingress settings.
  • Apps boot hardening: a dangling /var/lib/docker data-root symlink is cleared before re-linking, avoiding a dockerd crash-loop when the apps filesystem wasn't mounted.

Storage & bcachefs

  • bcachefs-tools bumped to v1.38.6.
  • The bcachefs update/sync status chip stays accurate after a sync — the engine reads the pin-derived fields fresh and the WebUI reconciles after a switch — and the chip now cleanly separates "update available" from "reboot pending".
  • Filesystems: bcachefs label groups are offered as data targets (foreground / background / promote), and an optional Rotational column surfaces the per-device bcachefs superblock flag.

System & UI

  • Sortable columns across the main data tables — Filesystems (and the per-pool device table), Alerts, Guest Shares, TLS, Users, and Snapshots.
  • Subprocess failures now reach the journal. Every failing engine shell-out (docker compose, smbcontrol, exportfs, …) is logged under the nasty::cmd target with its command + stderr, instead of failing silently — and a managed compose stack that fails to start at boot now says so explicitly.
  • nasty-top bumped to v0.0.8 — the live IO / tuning-advisor CLI now shows background_compression in the header when it differs from the foreground algorithm.
  • UPS support can be enabled: the NUT systemd units are built on the appliance.
  • SSH password-authentication changes made in the WebUI are honoured by the engine.
  • VM Add ISO no longer persists an empty CD-ROM path when editing.
  • Native <select> dropdown text stays legible across themes.
  • Help & Glossary gains Guest Shares, Time Machine and managed-startup entries, plus an r/bcachefs community link.

Alerts

  • The HDD-failure SMART alert is scoped to spinning disks (no false alarms on SSDs) and tolerates blank diagnostics.
reddit.com
u/bfenski — 1 month ago

Anyone running a third-party NAS OS on the ME Pro? Eyeing it for bcachefs tiering + VMs

I work on a small NixOS-based NAS OS called NASty (bcachefs under the hood). It's not on Beelink's official list - they show FNOS, Unraid and TrueNAS - so I'm not here to tell you to run mine. I'm trying to figure out whether the ME Pro is actually a good target before I claim it is.

What caught my eye is the layout: rotational bays plus the extra M.2 NVMe slots. That maps almost perfectly onto how bcachefs does multi-tier storage — fast NVMe as the foreground/promote tier, spinning disks as the background tier, all in one filesystem rather than separate pools. On paper it looks purpose-built for it.

For anyone who actually owns one:

  • Disk topology — how many NVMe slots run at full speed at the same time as the HDD bays are populated? Any PCIe lane sharing, or the NVMe dropping to x1 once the SATA side fills up?
  • Heat — do the M.2 drives cook under sustained writes in that chassis, or is airflow fine? Tiering leans on the NVMe hard.
  • Booting third-party OSes — does it boot a plain UEFI installer cleanly, or is there any locked-down / Secure Boot weirdness to fight?
  • Virtualization — anyone run KVM guests on it? Curious whether VT-x/VT-d are exposed in the BIOS and whether the IOMMU groups are sane enough for any PCIe passthrough (even just the NIC).
  • NIC + idle power — real-world idle draw with disks spun down, and is the onboard NIC well-behaved under Linux?

TIA!

reddit.com
u/bfenski — 1 month ago
▲ 12 r/bcachefs+1 crossposts

NASty 0.0.10

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the Backups + REST API release. Backups now run on a built-in cron scheduler, with credentials sealed at rest, a job-handle pattern keeping long-running ops alive past the WS timeout, and a HTTPS+auth-enforcing receiver. The engine's JSON-RPC dispatcher now also speaks REST at /api/v1/{...} with OpenAPI 3.1 and a built-in Swagger UI. Disk health gets first-class NVMe, SAS, ATA and PCIe-link panels; alerts fire on critical SMART attributes before the drive's own self-assessment flips. Plus a security pass — input validators, configfs secret redaction, systemd restart hardening, scanner-flagged response headers, two new destructive-confirm dialogs.

  • Cron-driven backup scheduler. Backup profiles run on their configured cron expression end-to-end. A new nasty_backup::scheduler parses cron, ticks every 60s, fires each due profile in its own tokio::spawn so a slow run can't starve the scheduler, and seeds last_attempted to now on engine start to avoid a thundering-herd catch-up after a long downtime. Long-running RPCs (backup.repo.init, backup.run, backup.repo.check) return a BackupJob handle that the WebUI polls every 2s, keeping multi-hour init / backup / check operations alive across reloads and well past the WS timeout. The Backups page rehydrates in-flight job badges across reloads, the edit form has the same Hourly/Daily/Weekly chip group as create, and the target is editable post-creation.
  • Backup credentials sealed at rest. Repository passwords + S3/B2 cloud secrets are sealed via systemd-creds --with-key=auto (TPM2+host when available, host-only fallback) before they hit /var/lib/nasty/backups.json. A new nasty_common::secrets module owns the seal/unseal path; on first start after upgrade a boot phase backups.migrate_secrets (30s budget) walks existing profiles and encrypts in place, idempotently. Wire shape stays backward-compatible — plaintext on input still works; output is redacted to "***" or absent. A backup.secrets_status RPC drives a Secrets: TPM-protected / encrypted at rest / plaintext pill in the WebUI so operators see at a glance which protection level applies. Filesystem encryption keys stay on the existing PCR-7 path (Phase 2, separate review).
  • Backup Server: HTTPS, required auth, paste-the-CA flow. The bundled restic-rest-server reads Caddy's already-issued cert for the box's tls_domain and serves HTTPS for free. Authentication is HTTP basic backed by a per-box 32-char alphanumeric password sealed via systemd-creds, with a Receiver credentials card on /services exposing Show / Copy / Rotate buttons. Source-side profiles get first-class Username + Password fields that round-trip as "***" and seal to disk via the same systemd-creds path as BackupProfile.password — credentials live in dedicated fields instead of riding inside the URL, so they stay out of list responses. Source-side profiles can also paste their own trust anchor PEM into a Trusted CA certificate field; the receiver's local CA is exposed inline on /services so the operator doesn't have to bounce through /tls to copy it.
  • In-engine REST API + Swagger UI. The method registry moved into nasty-engine and was backfilled from 106 → 256 methods — every RPC the dispatcher actually accepts now has typed params, role, and result schemas. Six docs-vs-runtime drift bugs fixed in passing. A new REST gateway at /api/v1/{path} translates dotted method names to slash paths with HTTP verbs inferred from the last segment (*.get/.list/.status → GET, *.delete → DELETE, *.set/.update → PUT, else POST). Auth matches /ws: nasty_session cookie or Authorization: Bearer. /api/openapi.json emits a ~1.1 MB OpenAPI 3.1 doc covering all 256 methods + 104 component schemas; /api/docs mounts Swagger UI v5.17.14 vendored via include_dir! — no CDN, works air-gapped, Try it out carries the same session cookie as the WebUI.
  • Disk health: full per-protocol detail. NVMe drives show a full panel reading the controller's health log — endurance %, spare-vs-threshold, critical-warning bits, media errors, host R/W totals, unsafe shutdowns, time above warning/critical temps. SAS drives get an equivalent SCSI panel including a health override that flips health_passed to false when any uncorrected R/W/V counter is non-zero — caught a real failing Seagate that smartctl still reported PASSED. ATA drives get a summary panel with interface-speed downgrade detection, reallocated/pending/uncorrectable tiles, SATA SSD endurance via smartctl 7.5. Topology shows each controller's PCIe gen × width × computed MB/s including for RAID-tunneled drives via /sys/class/scsi_host/, with chipset-integrated controllers annotated. Enumeration via smartctl --scan-open -j covers drives behind MegaRAID and similar SAS HBAs. SMART attribute names + descriptions + critical flags come from a 79-row Backblaze-evidence-backed table adapted from Scrutiny, and a new SmartAttribute alert fires the moment any of the 9 statistically-predictive attributes goes non-zero.

More details on GH.

github.com
u/bfenski — 2 months ago
▲ 23 r/bcachefs+1 crossposts

NASty 0.0.8

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the nginx → Caddy migration release. The reverse proxy and TLS terminator under the WebUI moved from nginx to Caddy. ACME issuance is now driven directly through Caddy (lego dropped), per-app ingress applies at runtime via Caddy's admin API, and the v0.0.7 NetworkManager compatibility scaffolding has been removed — boxes upgrading from 0.0.7 should be reconciled before jumping. Anything still on 0.0.6 or earlier should pass through 0.0.7 first.

Headline changes

  • Caddy replaces nginx as the reverse proxy and TLS terminator. App ingress routes apply through Caddy's admin API at install / remove time — config changes take effect in-process with no file rewrite and no reload. TLS automation is one atomic admin-API PATCH per change, so per-host issuance state shows up live on the TLS page.
  • Per-app subdomain ingress (V1 of #99). Apps can now be served at app.example.com instead of (or alongside) /apps/<name>/. Subdomain mode is selectable at install time and editable later, conflicts are detected before submit, and ingress-incompatible apps (whose absolute-path assets break path-prefix mode) auto-detect themselves and surface a clear reason in the install UI.
  • Self-signed certs now cover both nasty.local and the box's LAN / Tailscale IPs. Direct-IP HTTPS (https://10.x.x.x) validates the cert against the IP directly — only the "untrusted CA" warning remains, which clears once you import Caddy's root via the Download CA Root button on the TLS page. Unknown SNI (tailnet *.ts.net names, anything not on the cert) falls back cleanly to the internal cert.
  • Files page learned copy, move, and bulk actions (#88). Per-row Copy / Move icons + multi-select bulk action bar (Copy / Move / Delete) using the existing PathPicker. The same dialog handles files and directories regardless of which bcachefs pool the destination lives on.
  • NetworkManager compatibility scaffolding from v0.0.7 has been removed. The legacy networking layer, the one-shot migration cutover, and the Phase-X comments are gone. A clean reconcile of orphan interfaces + NM profiles runs at startup, per-connection NM apply errors surface individually in the UI, and DBus type encoding for MAC / DNS fields aligns with what NetworkManager expects.

And bunch of other changes/bugfixes. More details in release notes.

github.com
u/bfenski — 2 months ago
▲ 16 r/bcachefs+1 crossposts

NASty 0.0.7

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF — managed from a single web UI, updated atomically, and rolled back when things go sideways.

⚠ Heads-up: this is the NetworkManager-migration release. v0.0.7 runs both the legacy networking layer and NM in parallel for a transparent migration. v0.0.8 will drop the compat shim, so don't jump from 0.0.6 straight to 0.0.8 — go through 0.0.7 first.

Highlights:

  • Networking on NetworkManager with confirm-or-rollback safety. Network edits stage, apply, and auto-revert if you don't confirm in time — no more SSH-locking yourself out from a typo.
  • Encrypted filesystem lifecycle is end-to-end. Lock / unlock / mount-with-keyring-key all work, the WebUI shows which apps / VMs / shares / backups would break before you lock, with a 🔒 badge linking to the unlock dialog.
  • Hardware passthrough UI. IOMMU groups, USB device list, edit USB passthrough on existing VMs, bridge selector, inline disk-import wizard.
  • Subvolumes overview is the new default landing view. One table grouped by filesystem, real disk-usage progress bars, block-image actual allocation (no more iSCSI / NVMe-oF images reading 100% full).
  • Update flow rebuilt. Dev-build channel actually bumps the kernel now (refreshes all flake inputs), failed rebuilds dump the activation journal, nasty-cleanup is a one-shot fix for /boot full.
  • Live per-app resource usage on the Apps page (CPU%, memory, network I/O, disk I/O), à la docker stats.

Notable bug fix: filesystem quotas were stored 1024× larger than requested (bytes vs KiB units) — a 5 Gi NFS PVC ended up with 5 TiB cap. Engine fixed; startup reconcile auto-corrects existing installs.

aarch64 boxes (Pi / Odroid / Rockchip) now hit cachix for prebuilt binaries instead of compiling Rust + npm on every upgrade.

github.com
u/bfenski — 2 months ago
▲ 22 r/bcachefs+1 crossposts

NASty v0.0.6

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF — managed from a single web UI, updated atomically, and rolled back when things go sideways.

Highlights of 0.0.6:

  • OIDC / Single Sign-On — Log in with Google, Authentik, Keycloak, or any OIDC provider. Configure from Access Control → Identity Providers.

  • Security hardening pass — Browser sessions moved to httpOnly cookies, per-IP login rate-limit with persisted lockouts, WebSocket origin validation, gated WS endpoints, legacy ?token= URL auth removed, tightened HTTP security headers, {@html} XSS sinks killed, compose deploys sandboxed, NFS exports hardened, secret files locked down.

  • Network bridges — Linux bridges as a virtual switch for VMs (and apps), composable with bonds and VLANs (closes #27).

  • MTU configuration — Configurable MTU on physical interfaces, bonds, bridges, and VLANs from the WebUI — including jumbo frames (9000) for SMB / NFS workloads (closes #62).

  • Filesystem wizard upgrades — Drive model / serial / vendor / transport on the disk picker, usable-capacity estimate that matches the filesystems list, and a summary line on the filesystem card showing device count, erasure coding, and encryption.

  • Apps allow_unsafe escape hatch — Deploy compose stacks (or simple apps) that need privileged options with explicit user opt-in, surfaced in the deploy form and app list. Internal port now editable on Apps.

  • Background alert evaluation — Alerts fire from the engine's background notifier instead of waiting for a browser to be connected.

  • Test & CI footprint — fmt / clippy / svelte-check / test gates in CI, pinned Rust toolchain, integration nixosTest that drives JSON-RPC over the appliance, bcachefs smoke test, and unit tests across JSON-RPC framing, alert evaluation, sharing config, storage parsers, update rollback, the WebSocket client, the toast queue, and IO history.

  • Dependency refresh — rusqlite 0.34 → 0.39, openidconnect 3 → 4, vitest 3 → 4, plus major bumps to sha2 / rand / x509-parser / bollard / reqwest, nixpkgs to 549bd84 (2026-05-05), and bcachefs-tools to v1.38.2.

  • Smaller polish — SSH banner is now dismissible and renamed to "Configure SSH", banner buttons actually navigate, VM-detect loop fix, audit log rotation fix, dead nft -f - spawn removed.

github.com
u/bfenski — 2 months ago

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF — managed from a single web UI, updated atomically, and rolled back when things go sideways.

Highlights of 0.0.5:

  • Backup system — Deduplicating, encrypted backups via rustic_core library. Local, S3, SFTP, REST, and B2 targets. Scheduled backups with retention policies. Backup Server (restic REST) as a managed service for NASty-to-NASty backups.

  • Sidebar reorganization — 15 flat menu items collapsed into groups (Storage, Sharing, Protection, Compute, System) with collapsible sections and a search bar for quick navigation.

  • Log viewer — Dedicated Logs page with real-time streaming (follow mode), server-side grep, and client-side search/filter.

  • Notifications — SMTP, Telegram, Webhook, ntfy, and Signal notification channels with test-before-save.

  • Networking — Multi-interface support with per-interface IPv4/IPv6, dynamic nftables firewall with per-service source restrictions, bonds and VLANs.

  • SMB Groups — Group-based share permissions via @groupname, inline user/group creation in share wizard.

  • Services Page — Unified page with per-service Configure panels: NFS, SMB, iSCSI, NVMe-oF, UPS, SSH, Docker, Backup Server.

  • Boot Reliability — Device wait with udevadm settle before mounting, critical alerts on mount failure.

  • ARM Support - ISO for aarch64 is now included.

And obviously bunch of bugfixes and some refactors to make future development easier. Also that's probably last BIG reorg release. Things should now start stabilizing.

Enjoy!

u/bfenski — 3 months ago

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF — managed from a single web UI, updated atomically, and rolled back when things go sideways.

Highlights of 0.0.4:

Docker Apps

The apps runtime has been rewritten. Deploy single containers or paste a docker-compose.yml with syntax highlighting, port conflict detection, and live deploy output. Apps are automatically accessible through the built-in reverse proxy — no firewall changes needed.

bcachefs 1.38

Full compatibility with bcachefs 1.38. Per-subvolume options (data_replicas, compression, tiering targets) are now visible in the WebUI. New dataReplicas StorageClass parameter lets the CSI driver create volumes with reduced replication for expendable data.

Quality of Life

  • Global progress indicator for all operations
  • File preview and download in the file browser
  • nasty-top TUI for live bcachefs monitoring
  • Audit logging for all mutations
  • Firmware update support via fwupd
github.com
u/bfenski — 3 months ago