r/softwarearchitecture

I'm noticing enterprises buying into this new pattern called the orchestrator service. From my experience, I have noticed that it usually starts with a legacy service being too big to handle.

- Lots of contributors

- no clean code boundaries which means a lot of redundancy and ad hoc code

- scope is huge, it does everything

- deployment is hard because no one knows what the code does and what can break

- blast radius is a problem because it just runs too many things at once

So a lead comes up with a plan: "hey guys why not break our payment service into a payment orchestrator, checkout service, rule engine, loan service, customer context service that do one specific thing each".

You buy the idea and you find certain boundaries.

The orchestrator service still has the same API surface as your original legacy service because its the same entry point again (!). So you have changed no complexity at that level.

Then you create all the services and realise that the API surfaces of your child services (checkout, rule engine, loan, customer context) are all tightly coupled.

You accept that because you at least get to reduce some (?) blast radius. For instance, customer context service can go down without affecting other services. Then you realise, no, actually the blast radius problem is also not solved because how does payment work without customer context? You accept it and move on anyway.

Then you think, "well at least it allows different contributors to work across different services independently". Then your PM comes up with new requirements and then you realise you've now made things worse. There's a new tax regime and your change spans rule engine, loan service, customer context and obviously the orchestrator. Not only that, you have to change all API surfaces simultaneously. You still accept it. You make the changes. But then.. what have we really achieved? The orchestrator is always involved in every change?!

To steel-man the orchestrator pattern - it can sometimes work. If for example, the rule engine service can get big enough that entire customer features can span only the rule engine service. But is this usually true?

We sell an on-prem B2B product (Java/Spring Boot + PostgreSQL) that runs on a VM in the customer's own data centre. The site is air-gapped, so no license server, heartbeat or phone-home is possible. The customer's IT has full admin over the hypervisor and root in the guest.

The contract limits usage to N completed transactions per year. Today we enforce it with:

- a license file signed with our private key (the app only holds the public key);

- an append-only, hash-chained usage ledger in the database;

- periodic usage reports the customer sends us, which we verify offline.

The problem: cloning the VM, or reverting it to a snapshot, copies the license, the ledger and the counters together and consistently. Every copy passes every local check. Two clones get 2× the quota, and a snapshot revert resets the count.

What we found so far:

- VM Generation ID is the only signal built for this, but a non-root process on a Linux guest can't read it, and the hypervisor admin can pin or disable it.

- UUID, MAC, machine-id and vTPM are copied with the VM or easy to forge.

- TPM or SGX counters on the server aren't reachable from a VM.

Questions for anyone who has shipped on-prem, usage-based licensing:

1. What actually works for you in air-gapped VMs? USB dongles with on-chip counters (Wibu CodeMeter, Thales Sentinel), FIDO2 keys, a physical appliance, or something else?

2. If you use a dongle in a VM, how do you handle ESXi/Hyper-V passthrough, vMotion/HA and dongle failure?

3. Has anyone used a FIDO2 key's signature counter as a usage meter?

4. Or did you give up on technical enforcement and rely on audits?

Not looking to stop determined attackers, just to make cheating detectable or impractical.

I have Money transaction data in postgres DB, with fields like merchant, MCC, amount, date, card last four digits, and location.

I want users to ask questions like:

• “How much did I spend on restaurants last month?”
• “Show transactions in Dubai on my card ending 4242.”
• “How does my spending compare with last month?”

Low end-to-end latency is the main requirement, while keeping filtering and calculations accurate.

I’m considering using a small LLM to turn questions into structured queries, executing them against the database, and rendering results without a second LLM call.

Has anyone built something similar? What architecture and models worked for you, and what actual latency—especially p95—did you achieve?

I recently broke down the system design of a payment gateway and double-entry ledger platform handling 100M daily transactions (~5,800 peak write TPS).

The article focuses on the real-world engineering challenges: stopping accidental double-charges with distributed idempotency locks, enforcing strict double-entry bookkeeping (Debits = Credits) using insert-only tables, recovering from card network timeouts without duplicate charges, and running midnight three-way reconciliations against bank settlement files.

For those who have built financial or checkout infrastructure: what was your biggest challenge in production? How do you handle partial failures and asynchronous discrepancies across multiple bank networks?

I built JobMaster, a job scheduler for .NET aimed at horizontal scale. Execution is split from the audit trail and the coordination, so workers are not all locking the same rows.

Jobs go into a per-worker bucket first. The Master DB keeps the history. A job due soon stays in the bucket. A job due later is parked there and moved back in a batch when it is close. On a 50k burst with 20 workers, NATS in front of RavenDB did 23.8k jobs/sec, against 6.5k when one database did both.

• Write-up: https://hugoj0s3.dev/blog/jobmaster-achieving-horizontal-scaling-for-net-jobs-while-maintaining-a-centralised-audit-log/
• Docs: https://docs.jobmaster.hugoj0s3.dev/
• Benchmark: https://docs.jobmaster.hugoj0s3.dev/docs/benchmarks/results
• Architecture: https://docs.jobmaster.hugoj0s3.dev/docs/architecture-under-the-hood/architecture-overview
• Repo: https://github.com/hugoj0s3/jobmaster-net

I would appreciate any feedback, especially on the bucket split.