I keep falling into the over-engineering trap—getting the design "just right" is so hard. It's probably my lack of experience causing me to design for fake requirements. I really need to focus on current needs rather than predicting the future. Any advice would be much appreciated!

I built a pretty ambitious project, shared part of it with the community through "silentJson", and wrote a detailed article explaining how to achieve some pretty crazy performance results.

The results were so unrealistic to some people that I was accused of spam, and the article was removed from a few subreddits.

Still, I'm genuinely grateful that many people at least took the time to read the articles, and that some of them actually appreciated the work.

So, here is the continuation.

The problem isn't only that we need to remove unnecessary layers, conversions, or simplify the logic. The bigger problem is that all of this is actually standard practice in production systems. And most people simply don't notice it anymore.

Let me give you a hint where to look.

Take PL/pgSQL, for example. You can put the logic directly inside the database.

Or Elasticsearch. It works in a similar way.

You probably use approaches like these all the time.

And while PostgreSQL itself doesn't necessarily give you some massive raw speed advantage, it can still give you a huge overall performance win because you are not constantly moving information through layers of OOP abstractions and paying the cost of all those calls and transformations.

The business logic is already where it needs to be.

You can use the same idea in your own applications.

But for some reason, we often care more about creating interfaces and generics so that we can mock everything in unit tests instead of simply writing integration tests.

This can significantly reduce the load on the system and, more importantly, make your tests validate real scenarios instead of proving that "sum(1, 2)" returns 3.

So where do you start?

Take a piece of paper.

Write down the actual goal of the system, what you have, and what you want to achieve.

Use blocks, lists, arrows, whatever works for you.

At this stage, completely forget about the programming language.

Seriously.

If you start thinking about Go, Java, Rust, interfaces, frameworks, or whatever else too early, you will probably start designing around the language instead of designing around the problem.

First, work out how the data should actually flow through the system.

Only at that point should you start thinking about what data you need, how it should be stored, and, perhaps even more importantly, what should not be stored or processed at all.

Then start writing code.

But write it according to the data flow and decomposition you already designed.

At every stage, use only what is actually necessary.

For example: maps, interface{}, channels, mutexes, goroutines.

Sometimes it is incredibly convenient to throw together a worker pool using channels. It's simple, readable, and often works very well.

But do you actually need it?

Creating goroutines isn't free.

A WaitGroup isn't automatically the best solution just because it's convenient.

And combining maps with mutexes can very quickly turn into an adventure where you're trying to figure out who locked what, when it happened, and why something never got unlocked.

Sometimes a simple slice, struct, or primitive type is all you need.

Programming has spent decades developing good ways to work with data efficiently. Look at the natural primitives we already have.

Take JSON.

It is simple, universal, and fast enough for an enormous number of real-world applications.

Yet many people consider JSON "slow".

Usually, the problem isn't JSON itself.

The problem is what you put inside it and what you make it do.

Add a "time.Time", a "map[string]interface{}", or several layers of dynamic structures, and suddenly you get the performance penalty you were blaming on JSON.

The same principle applies everywhere else.

If the job is to store data, store it.

If the job is to return data, return it.

Don't build obstacles in between.

The fewer steps the information has to go through, the faster it reaches its destination.

And don't try to return more than you actually have.

Separate responsibilities.

If a service is supposed to provide data, it should provide data.

If a BFF is supposed to compose data, let it compose data.

And let it do that as quickly and simply as possible.

It shouldn't be transforming everything just because it can.

A BFF doesn't need to contain your business logic.

Wait.

What about business logic?

This is where things get interesting.

Business logic can actually operate on different levels.

Some logic changes the data itself. Other logic changes how that data is presented.

The first kind belongs as close as possible to the point where the data is created, retrieved, or indexed.

The second kind can live at the composition layer, frontend, or a dedicated business layer.

Where exactly you put it depends on the architecture and business requirements of the project.

And that's why the architecture should be designed before you start writing the code.

Once the data flow is clear, you can look at the system and ask a much more useful question:

"Where is the bottleneck?"

Then remove everything on the path that creates an unnecessary restriction.

Profile your application.

If you don't know how, ask an LLM. There is no shame in that.

When you've done all of this and finally discover that the bottleneck isn't your architecture anymore, but the hardware itself, then you can start asking whether that abstraction, interface, mock, or extra layer is really worth its cost.

Especially when the only reason for adding it was to prove that one tiny function correctly calculates 2 + 2.

I'm curious how other people approach this.

How do you design, develop, and profile your systems?

And how do you decide when an abstraction is actually useful and when it is just another layer between the data and its destination?

For context , I know basic SYSTEM design concepts, like all the concepts from Gaurav Sen Yt Playlist. I have not done much practice on those concepts therefore the Title.

1-10 how good are you and your team in keeping architectural decisions in documents? Do you have a process or tool to help you with that?

Infrastructure as code can reproduce the same network, container, database, and service account twice. That does not guarantee that the workload running on those resources means the same thing twice.

We ran into this while building a distributed runtime. A service could start with different capabilities than its twin. A consumer could join a runtime it was never intended to join and begin reading work addressed to another process. A required policy value could be missing and quietly fall back to a local default. Every resource matched and every health check was green, but the operational contract had changed.

The problem was not that our infrastructure was insufficiently declarative. We had declared plenty. We had declared resources without declaring enough behavior.

We now separate the infrastructure definition from a versioned runtime policy contract. The contract defines supported profiles, process capabilities, ownership boundaries, refusal conditions, secret-resolution rules, and the evidence required to accept the resulting runtime. It is parsed into a typed immutable model, then projected mechanically into the values consumed by deployment manifests:

runtime policy contract

-> typed validation

-> rendered deployment values

-> service manifest

-> running process

The direction matters. If a capability can be edited in the contract, overwritten in generated configuration, defaulted in the manifest, and changed again inside the application, the deployment is not governed by one declaration. It has four declarations and an undocumented precedence rule.

Repeatability also has to include refusal. If an environment lacks an effects boundary, it should not silently execute the effect inside a general worker. If a required secret cannot be resolved, the runtime should not grab a nearby credential. If a process is not eligible to own a subscription, it should not start that consumer merely because the package is installed.

The contract is not proof by itself. The renderer can be wrong, the application can ignore a value, and a runtime can claim a capability it does not actually provide. Acceptance still needs durable evidence binding the contract revision, rendered projection, validators, selected profile, and boundaries exercised after startup.

Infrastructure as code builds the substrate. The infrastructure contract defines what that substrate must mean, what variation is allowed, and when the deployment must refuse to start.

Where do those behavioral rules live in your systems today?

I have taken a stab at describing and thinking about cognitive debt. I find there are quite a few definitions of what that actually is. My definition is:

The accumulated gap between what an organization collectively knows and what its people can reliably access, trust, interpret, and act upon.

Like technical debt, it begins with reasonable shortcuts: a verbal decision saves time today; an experienced employee becomes the unofficial knowledge base; a process changes but the manual does not; a new system is added without retiring the old one.

Over time, the company pays interest on the accumulated complexity.

I think there are 3 problems

1. Work Takes Longer
2. Intelligence is Diverted
3. Latency - Decisions take longer.

Love your thoughts; please be kind.

I wouldn't consider myself a proper software engineer, but I've been coding for around 10 years, with the things I am coding increasing in importance to the business at the same time that AI coding is increasing in prevalence.

Even with a well-designed spec, I find that the code that AI generates is just ... I don't know. It's perfect when zoomed in, but as soon as parts start interacting and you are not coding from scratch every time, the whole thing falls apart. I mean, it will compile and work, but what comes out of it is riddled with incomprehensible layers of abstraction and errors. When I go in and try to understand those errors or explain why my code made them to others, it takes me days. When I ask the agent that wrote the code to explain what the code does, it seems to struggle even more than me. Fixing things without thoroughly understanding what the AI wrote (in detail), tends to make things worse.

I have tried modifying my .md files to include good software hygiene, agent review loops, heuristics .... none of it seems to make much difference.

Does the entropy that agents inject into a codebase eventually solve itself, and maybe I need to release my grip on the reins for a longer period of time? Is it just highlighting my own incompetence as a software developer? Or am I doing something otherwise wrong? Maybe I am just projecting my own issues onto the technology.

I am thinking about switching to something like cursor tab so I have tighter control over what comes out.

I don't see things going back to a time w/o AI assisted coding.

What has worked for everyone here?

Please feel free to roast me too :)

When building a dealership inventory catalog for sales agents on showroom iPads, we realized standard network-on-every-filter setups were breaking the pace of sales conversations. Every loading spinner broke conversational flow.

We restructured the catalog around a local-first pattern: load the inventory once, keep it on the device, filter in memory, and use a light server signal to invalidate cache only when the inventory actually changes.

1. Upfront Payload for In-Memory Filtering: Instead of paginated API queries on every filter change (brand, price, mileage), we fetch the full vehicle inventory once.

2. Instant Rehydration via IndexedDB: To avoid startup latency on repeat visits, we persist the TanStack Query cache to IndexedDB.

3. Cheap Real-Time Invalidation via RxJS + SSE: To prevent stale data (e.g., sold/repriced cars) without burning serverless budget on polling, we use a single Server-Sent Events (SSE) connection.

Read the full article: https://blaze64.dev/logs/local-first-pwa

Hello, I just want to get some advice about this.

Do you think system design is more important now because of AI?

I’ve been thinking about this lately because AI can produce a lot of code in a short amount of time. Something that might take me a few days to build, AI can sometimes generate in minutes, and sometimes it even catches edge cases that I didn’t think about.

For the past 3 months, I’ve been using AI a lot when coding. Most of the time, I let it generate the code, then I read through it, understand what it’s doing, and validate if it actually makes sense and aligns with the product we’re building.

Because of this, I feel like my job is slowly becoming less about writing every line of code myself and more about knowing what should be built, how things should be designed, and whether the generated code is actually good or not.

Should we be focusing more on system design and architecture now?

PS. I asked AI to rephrase my post because I'm not a native English speaker.

Edit: Thanks guys, I think I won’t be guilty using AI now and will push forward to learn architecture and system design!

Hi everyone, I'd like to ask for your insights based on your experience. Is it possible to use Antigravity to build multi-tier software that includes, for example, a CRM and a planning tool? Can Antigravity also set up databases? I'm skeptical about whether that is actually feasible.

We need to implement a tenant offboarding/data deletion pipeline across multiple microservices using shared tables in PostgreSQL and ClickHouse. All relevant tables contain a tenant_id.

We're weighing two options:

• Static Registry: Manually maintain a mapping of target DBs and tables (hard to scale and maintain).
• Dynamic Discovery: Query metadata tables (information_schema, system.columns) at runtime to find and delete from all tables with a tenant_id.

Are there better architectural patterns or alternative solutions (e.g., event-driven soft deletes, CDC pipelines, or orchestrators) to handle this cleanly at scale?

Working through a multi-module architecture for an enterprise Java service, and the thing I keep fighting is the same thing every team eventually fights: JPA annotations, pagination params, and query logic quietly leaking out of the persistence layer and into services and domain models until nothing's testable without a database running.

The approach that's worked so far: a generic CrudDaoImpl/SearchableDaoImpl hierarchy that every concrete DAO extends, so search/pagination/spec-building logic lives in exactly one place instead of copy-pasted per entity. The domain objects stay pure Java - no framework annotations, immutable audit fields, no setters and the DAO layer is the only thing allowed to touch things like createdAt/updatedAt, set explicitly in code rather than trusted to a DB default (Hibernate doesn't read back DB-computed defaults after save(), so relying on the default alone hands you a null timestamp on the object you just created, I found that one the hard way).

The part I didn't expect to spend as much time on: some entities are append-only and never get an updatedAt at all, so the base class can't statically assume every entity supports it. Ended up with an explicit instanceof check in the shared preCreate/preUpdate hooks rather than forcing every entity through an interface it doesn't need, I felt it like a compromise at first but it reads as the right call in hindsight.

Longer writeup with the actual class hierarchy and repository access: (link in comments)

How do others handle the "some entities need X, some don't" problem in a shared base class without it turning into an interface explosion?

The TIED methodology incorporates strict TDD to produce the correct code. The primary tasks is now capturing and enforcing the user's intent. See this full article to learn how TIED makes TDD work for agentic programming. Do you trust your TDD-compliant code?

I’ve been thinking about something after reading a few discussions around AI assisted development and architecture.

A lot of teams use ADRs, architecture conventions, code owners, reviews, CI/CD, and internal processes to keep systems consistent.

But I’m curious how this works in practice when a team starts using AI coding tools more heavily.

Do teams usually have a way to automatically check whether a PR still follows the project’s architecture decisions and conventions?

For example, things like:

• a change going against an existing ADR
• new logic bypassing an agreed service boundary
• duplicated logic being added in another part of the system
• a new dependency or library being introduced without design review
• a feature implementation needing an architecture discussion before continuing
• a large AI assisted PR being technically correct but not fitting the system design

Or is this mostly still handled manually by senior engineers, architects, and reviewers?

Hey,

Can you share how your company handles databases for local development? I’d really appreciate hearing about your experience and any valuable insights you can share.

I’m a little confused about what the better approach is:

1. Shared cloud DB: If a company uses a shared cloud database for development, how do they handle the situation where one developer makes a breaking change that affects everyone?
2. Individual local DBs: If developers are expected to set up their own local databases, how does the company provide the large amount of initial/seed data needed to get started?

I’d really appreciate it if you could share how your company handles this in practice, or any best practices you’ve seen.

Thanks!

I spent the last nine months building a construction project management software. The biggest challenge I faced so far was not building the app. It was the scheduling engine.

This is a long post. It is the modelling decisions behind that engine and what each one cost.

What construction scheduling actually is

A construction project is a list of things that have to happen in a specific sequence.

excavate foundations -> pour blinding ->fix reinforcement -> pour slab -> strike the formwork

Some of those can happen at the same time. Most cannot. You cannot pour a slab into a hole that has not been dug.

A schedule, or programme, is that list plus every "this must come before that" rule between the items, turned into dates. This matters because of one question a contractor gets asked frequently, 'We are three days late on pouring the slab, does the handover date move?'

Sometimes the answer is no and it costs nothing. Sometimes it is yes and it costs a month. Those two situations look identical on a gantt chart, and telling them apart is the entire job of the engine.

Getting it wrong is expensive in a specific way. Construction contracts carry liquidated damages, a fixed sum per day late. A schedule that quietly under-reports a delay does not just mislead, it costs money per day of being wrong.

And the schedule is never left alone. Rain stops earthworks. A concrete pour cannot happen in a downpour. Tasks spill into the following week, and some of that spill reaches the handover date while some of it disappears into slack that was already there.

Which of those two happened is not obvious from looking at the gantt chart, and the difference is commercial. Weather delay on a task with room in it is absorbed. Weather delay on a task with no room is a claim for an extension of time, and if it is not identified and argued for, the contractor eats the damages instead.

So the backend engine is not calculating a plan once. It is answering the same question every time reality moves. Given what just happened, does the end date move, and which tasks caused it.

The vocabulary

A few terms I will use throughout. These are the construction ones. I am assuming the computer science ones.

Task. One item of work. 'Excavate foundations'. It has a duration and a place in a numbered hierarchy called the WBS, the Work Breakdown Structure, which is why tasks are labelled 1.1, 1.2, 1.2.1 and so on.

Dependency. A rule saying one task must come before another. Not a suggestion or a preference. In my schema it is a row in its own table, not a column on a task, and I will explain why below.

Predecessor and successor. The two ends of a dependency. If excavation must come before pouring, excavation is the predecessor and pouring is the successor. The relationship is one-way and the direction carries the meaning. Reverse it and you have told the crew to pour concrete into ground that has not been dug.

Lag. Delay built into the dependency itself, separate from either task's duration. Pour concrete, then wait seven days before striking the formwork so it can cure. Nobody is on site working during those seven days. It is not a task. It is a property of the link between two tasks, which is why it lives on the dependency row.

A negative lag is legal and is called lead. It means the successor starts before the predecessor finishes, overlapping them deliberately.

Lag unit and Edays. Lag needs a unit, and this is the detail I would not have got right without talking to someone who has run a site.

Concrete cures on Saturday. It cures on a public holiday. A seven-day cure is seven actual days, no matter what the calendar says about work. I call that unit EDAYS, elapsed days, and it is raw calendar math.

Waiting for an inspector is also written as seven days. But inspectors do not work weekends, so that one is seven working days. I call that DAYS, and it routes through the calendar service that skips weekends and holidays.

Same phrase in the contract. Different math in the code. Get it wrong on a cure period and you lose two days per weekend crossed, compounding across every pour on the project.

Forward pass. Runs from the project start to the end and establishes each task's earliest start and finish. In my engine those dates are already resolved when a task's dates are set, so this pass reads them rather than deriving them.

Backward pass. Runs from the project end date backwards and computes the latest each task can finish without pushing the end date out. Every successor has to be computed before the task that feeds it, which is why the ordering matters.

Float, also called slack. How many working days a task can slip before the project end date moves. A task with ten days of float slipping three days costs nothing. A task with zero float slipping one day costs a day off the handover.

Critical path. The chain of zero-float tasks. If anything on it slips, the project end date moves. Everything else has room.

Baseline. The plan, locked. Without it, "behind schedule" has no referent, because if the plan moves every time something slips then nothing is ever late.

Dependency type. I assumed for too long that a dependency just meant "after." It does not. There are four:

Type |Full name |Meaning |Site example
FS |Finish to Start |successor starts after predecessor finishes |Excavate, then pour
SS |Start to Start |successor starts after predecessor starts |Start trenching, then start laying pipe behind the trenching crew
FF |Finish to Finish |successor finishes after predecessor finishes |Two crews on one pour who must stop together
SF |Start to Finish |successor finishes after predecessor starts |Rare. Shift handovers, mostly FS is the common one. SS is the one people forget, and it is the one that breaks a naive model, because SS work runs in parallel rather than in sequence.

1. Dependencies are not a column on the task

The obvious schema is a blocked_by column on the task row.

That breaks the first time a task has two predecessors. Pouring a slab waits on the reinforcement being fixed and on the formwork being up. One column cannot hold both.

It breaks harder once you notice the relationship carries its own data. A dependency has a type and a lag. Those belong to neither task. They belong to the link.

So dependencies are their own table. predecessor_id, successor_id, type, lag, lag_unit

The cost is that reading a schedule is now two queries and an in-memory join. Nearly every operation in my calculator starts by building a predecessor map and a successor map before it can do anything useful.

2. Four dependency types means four formulas everywhere

Supporting SS, FF and SF is not a matter of storing an enum and moving on.

Every place the engine reasons about dates has to branch on the type. My backward pass computes, for each dependency, what that edge implies about the predecessor's latest finish. FS and SS read the successor's latest start. FF and SF read its latest finish. The two start-anchored types then add the predecessor's duration back on.

Four formulas, and each one is a place a sign error can hide without crashing anything.

The cost is test surface. A single formula needs a handful of cases. Four formulas with positive lag, zero lag and negative lag is twelve, before you add weekends.

3. Two lag units means two code paths

Lag routes to different date math depending on its unit. Working days go through the calendar service. Elapsed days are raw calendar addition.

The cost is that every lag calculation has a branch in it, and the branch is invisible in the output. Both paths return a valid-looking date. The only way to know which one was correct is to know what the seven days represented on site.

I have not solved the interface side of that. The person entering a dependency has to pick the unit, and there is nothing in the data that lets me infer it for them.

4. The dependency graph has to be acyclic

Tasks and dependencies are already a directed graph, whether or not anything draws one. Directed because predecessor to successor is one-way, and the direction is the entire meaning.

It also has to be acyclic, meaning you cannot follow the arrows and arrive back where you started.

Here is a cycle, taken from one of my test fixtures.

Alpha -> Beta -> Gamma -> Alpha

Read it as constraints. Alpha before Beta. Beta before Gamma. Gamma before Alpha. Therefore Alpha before Alpha.

That is not a tight schedule or a late schedule. There is no set of dates that satisfies it.

This matters structurally. My backward pass computes each task's latest finish from its successors' dates, so it has to visit every successor before it touches that successor's predecessors. Otherwise it reads values that have not been computed yet.

An ordering with that property is a topological sort, and it exists if and only if the graph is acyclic.

No acyclicity, no valid order, no critical path.

5. Two cycle algorithms, because they answer different questions

I use Kahn's algorithm for the ordering. Track in-degree per task, which is just a countdown of unmet preconditions. Seed a queue with every task at zero. Pop one, emit it, decrement its successors, and any successor that hits zero joins the queue.

I picked it over the depth-first search because the intermediate state means something here. The queue at any moment holds exactly the tasks that are currently unblocked, which is a real thing a site manager wants to see.

I use Kahn's algorithm for the ordering. Track in-degree per task, which is just a countdown of unmet preconditions. Seed a queue with every task at zero. Pop one, emit it, decrement its successors, and any successor that hits zero joins the queue.

I picked it over the depth-first version because the intermediate state means something here. The queue at any moment holds exactly the tasks that are currently unblocked, which is a real thing a site manager wants to see.

Kahn's detects cycles for free. Tasks inside a loop never reach in-degree zero, so the queue starves and you finish with fewer sorted tasks than you started with.

But that only tells me a cycle exists. It cannot tell me which tasks form it, and the starved set includes everything downstream of the loop as well as the loop itself.

"Your schedule contains a circular dependency somewhere among four thousand tasks" is not an error message anyone can act on.

So imports run a second algorithm. A depth-first walk with three states per task. White for not yet visited. Black for fully explored. Gray for the tasks currently on the path from where I started down to where I am standing.

An edge to a gray task is an edge back to an ancestor. That is a cycle. And because I am already tracking the current path, the loop is the slice of that path from the gray task onward.

The import rejects with the actual loop listed by WBS code and task name. The user fixes it in a minute.

Two algorithms for what looks like one job, because "is there a cycle" and "which cycle" are different questions.

6. I have three different cycle policies

Layer |Policy on a cycle
Import |reject, return the loop path, write nothing
Critical path engine |append the unsorted remainder and continue
Cascade |bound the worklist by a step count, log a warning, stop The reasoning is that the import is the gate, so the engine should never see a cyclic graph.

But the engine currently degrades silently rather than throwing. If anything ever reaches it that did not come through the gate, I get wrong dates instead of an error.

I think it should throw. Then the invariant is enforced rather than assumed.

7. Working days are per project

Skipping Saturday and Sunday is not enough. Public holidays move dates, and they are not the same set in any two countries, so the holiday list cannot be hardcoded anywhere in the engine. Different projects also run different working weeks. A road contract working six days is not the same calendar as a commercial fit-out working five.

So there is one calendar service, holidays arrive as a set of date keys rather than being baked in, and the project’s schedule row carries a calendar_id. Adding a country is adding a calendar, not touching the engine. Where those dates come from is a data problem, and there are public holiday APIs that resolve them per country, which is exactly why it belongs at the boundary rather than in the calculator.

The cost is that date math stops being a pure utility function. It becomes a service with a dependency, which makes the parts of the engine that use it harder to test in isolation.

8. Late needs something to be late against

If you measure progress against the current plan, and the plan shifts every time something slips, nothing is ever late. The chart looks healthy permanently.

So the plan gets locked, and the locked version is the baseline. Separate columns on the task, plus a history table because projects do get re-baselined when a variation is agreed.

The cost is duplicating every planned date column on every task row.

9. Not all lateness moves the end date

This is the question the whole engine exists to answer.

Twenty late tasks can cost nothing. One late task can cost a month. The difference is float, and only zero-float tasks move the handover date.

I persist float and the critical flag on the task rather than computing them on read, because the read path is a chart that loads constantly and recomputing a few thousand tasks per load was not viable.

The cost is staleness. Anything that changes dates has to trigger a recalculation, or the flags quietly describe a plan that no longer exists.

There is a version of this where float is derived on read and cached properly.

Conclusion

That is nine months of decisions compressed into one post. It’s been hella crazy learning and implementing this feature btw I have zero construction knowledge when I got started. I learned by getting it wrong first and talking to construction managers.

I am happy to go deeper in any of it in the comments.

Thanks for reading this far.