u/ioncehackedmyschool

What it is: a "gentle" observability layer for training. The training loop calls run.emit(); metrics flow through a lock-free ring buffer into a separate engine that persists an append-only log and serves live dashboards (terminal + self-hosted web)

Why another logger: existing tools tend to (a) require an account + cloud upload, or (b) add latency to the training loop. I wanted something that's local-first and provably can't slow the run down.

Design choices that might interest this sub:

• Non-blocking by construction. emit() is sub-microsecond; every queue between the loop and disk is bounded and drops-and-counts on overflow rather than blocking the training thread. Observability degrades gracefully under load instead of harming the run.
• Event-sourced. An append-only events.jsonl is the audit trail; a wide metrics.jsonl (readable by pandas/jq, importable from other loggers) is the export surface. A killed run still leaves a valid log.
• Deploy modes for HPC. Embedded (in-process), sidecar (separate engine training survives an engine crash; the SLURM pattern), or file-only. Auto-detects SSH/SLURM.
• Engine in Rust (unsafe forbidden), SDK in Python with duck-typed tensor coercion (pass loss directly, no .item()).

pip install emry Apache-2.0, v0.1/alpha. Repo and a quickstart in the README.

Check it out at: https://github.com/femboyisp/emry

I'd love feedback on the backpressure model (drop-and-count vs. block) and the deploy-mode abstraction.

https://github.com/femboyisp/purroute

one tiny rust router that sniffs whatever proxy protocol you throw at it socks5, socks4, http, https and tunnels it upstream, translating between them on the way out ^_^