I made this mistake recently. An agent generated a fix for a bug in our checkout flow, the diff looked reasonable, tests passed, so it shipped with a normal PR review and nothing more.

I assumed passing tests meant the function behaved the same as before under real traffic. It didn't. It handled a currency rounding edge case differently, and it took almost two days of scattered complaints before anyone connected it back to that deploy.

If I did it again, I'd want something checking the function's actual runtime behavior before treating a green test suite as enough. What's the mistake that took you longest to recognize as a pattern?

I kept running into the same problem with codebases:

“What does this repository's architecture actually look like right now?”

So I built arch-auditor — a Python CLI + Streamlit app that analyzes a repository using deterministic, AST-based checks and turns the results into actionable architecture reports.

The core idea is simple: measure first, ask the LLM second.

What My Project Does

arch-auditor points at a Python repository and runs deterministic architecture detectors:

• Import/dependency graph — see how modules depend on each other
• Circular dependencies — detect dependency cycles
• High coupling — identify modules with unusually high fan-in/fan-out
• Oversized modules — flag modules that have grown beyond a configured threshold
• Layer violations — enforce architectural rules defined in YAMLBlast radius — find direct and indirect dependents of a module

It produces a dependency-graph SVG, severity-ranked findings, and impact reports.

There's also an optional Gemini integration.

Gemini doesn't perform the underlying analysis. Instead, it receives the deterministic findings and can turn them into:

• an architecture overview
• root-cause explanations
• step-by-step refactoring plans
• affected files
• potential risks

No API key? No problem. The core analysis works offline without an LLM.

Try it

pip install arch-auditor

arch-auditor demo

The bundled demo repository is intentionally designed to trigger every detector, so you can experiment with the tool and see what each finding means.

Stack

• Python 3.10+
• Python AST / standard library
• NetworkX
• Pydantic
• PyYAML
• Streamlit
• Gemini (optional)
• 33 tests
• Ruff-clean
• Published on PyPI

It's released under GPL-3.0 because I'd like improvements to flow back to the community.

Target Audience

This is primarily aimed at developers working on medium-to-large Python codebases, especially when:

• a project has accumulated architectural debt
• you're preparing for a major refactor
• you need to understand an unfamiliar repository
• you want objective signals before making architectural changes
• you want an LLM to help plan a refactor without making the LLM responsible for discovering the architecture itself

It's not intended to replace a human architect or code review, and it's not a magic “is my architecture good?” score.

The goal is to provide reproducible evidence that helps humans make those decisions.

Comparison

There are already excellent tools for individual parts of this problem — dependency visualization, linters, type checkers, code-quality metrics, and various AI coding assistants.

arch-auditor is trying to connect a few of those ideas around architecture-level analysis.

The main distinction is the separation between measurement and interpretation:

Traditional approach:

Repository → LLM → "Here's what I think your architecture looks like"

arch-auditor:

Repository → deterministic analysis → evidence → optional LLM → refactoring plan

The architecture findings don't depend on whether an LLM happens to interpret the code differently from one run to another.

I'd especially love feedback on whether the detectors and thresholds are useful signals in real-world Python projects, or if there are architectural problems you'd want to see measured that aren't covered yet.

GitHub / docs / demo:
https://github.com/ANIKETHSAI9813/auditor

“What does this codebase’s architecture actually look like right now?
"Instead of asking an LLM to guess the architecture, arch-auditor first analyzes the repository using deterministic, AST-based detectors — then optionally lets Gemini turn those findings into a refactoring plan.
Point it at a repo and it analyzes:
The result is a dependency-graph SVG, severity-ranked findings, and impact reports that show why a module is considered problematic.
Gemini is optional.
The deterministic analysis produces the evidence first. If you provide a Gemini API key via an environment variable, it can use that evidence to generate:
No API key? No problem. The core analysis works offline.
That separation was intentional: I wanted the architecture measurements to be reproducible rather than dependent on an LLM's interpretation.
The bundled demo repo is deliberately designed to trigger every detector, so you can see exactly what each finding means without having to point it at a huge codebase first.
It's released under GPL-3.0 because I'd like improvements to flow back to the community.
GitHub / docs / demo:
https://github.com/ANIKETHSAI9813/auditor
I'd especially love feedback from people who work on large Python codebases:
Are these the architectural signals you'd want to see before starting a refactor?
I'm also happy to discuss how the detectors and thresholds work — and the demo repo is intentionally built to trip all of them.

I spent **$5,419.54 with Replit in roughly seven weeks** trying to build and deploy real business applications.
This wasn’t a hobby experiment or a case where I subscribed, changed my mind, and asked for my money back.
I bought an annual Replit Pro subscription on August 10 and then used Replit Agent extensively to build, troubleshoot, repair, deploy and publish actual projects.
According to Replit’s own review of my account:
**$5,419.54 was successfully paid.**
**1,423 Agent runs were completed.**
**Approximately 96% of the usage was Max mode.**
**My support history contains 34 ticket records.**
And after reviewing all of it, Replit’s refund decision is:
**$0.**
I want to explain how I got there because I think my experience raises a much larger question about how AI coding platforms charge customers when the Agent is being used to troubleshoot problems involving the platform itself.
**I didn’t start out asking Replit for a refund**
I was trying to make Replit work.
I had multiple real projects being developed and deployed through the platform. Replit wasn’t just where I occasionally asked an AI to write code. It had become part of the development and production workflow.
And when something went wrong, one of the tools available to diagnose and fix the problem was **Replit Agent itself**.
That distinction became extremely important.
Every time Agent investigated something, changed something, tried a repair or worked through another problem, usage could continue accumulating.
The basic economic loop became:
**Problem → Agent investigates → paid usage → attempted repair → publish/deploy → another problem → Agent investigates again → more paid usage.**
At first, I kept doing exactly what I assume Replit expects customers to do:
**I kept trying to fix it.**
**The problems didn’t begin with the outage Replit now admits**
There were already problems during August.
By August 11, I was dealing with Agent workflow/progress issues.
During the middle of August, there were problems accessing, locating and independently verifying existing application and production work through the Replit tooling.
By late August, I still didn’t have the dependable end-to-end workflow I needed:
**make the change → test it → publish it → verify it live.**
Then on **August 29**, publishing failed with:
**“Migrations failed validation.”**
That is important because it happened **before** the outage Replit now acknowledges.
Agent investigation around that problem also ran into a timeout.
And the paid usage continued.
**Then Replit’s own platform actually broke**
This part isn’t my accusation.
**Replit has admitted it.**
After reviewing my account, Replit wrote:
“From August 31 to September 4, a platform issue on our side blocked publishing for a number of projects, including Artist Bos. It was not caused by anything you did.”
There was also a support case, **#522753**, involving a server-side disconnection during the database-diff/publishing process.
At the time, I wasn’t demanding thousands of dollars back.
**I wanted the publishing system fixed so I could continue working.**
That’s important to me because this wasn’t a customer looking for an excuse to get out of a bill.
I was still trying to make the platform work.
And while I was doing that, I was continuing to use the product—including its paid Agent.
So now that Replit acknowledges the underlying publishing problem during this period was theirs, I’ve asked them a very specific question:
**How much of my paid Agent usage was consumed diagnosing, retrying, repairing or working around the Replit-side publishing failure?**
I still don’t have that accounting.
**Replit says it fixed the outage September 4**
Replit says engineering fixed the platform issue on September 4.
It also says I successfully published **16 times between September 5 and September 7**.
That’s Replit’s evidence, and I’m including it.
I’ve asked them to identify those publishes—the projects, timestamps, deployment information and production results—because a recorded successful publish doesn’t necessarily answer whether the resulting application was complete and working correctly.
And unfortunately, the production problems didn’t end there.
**September 10: 13 name-matched databases were unexpectedly deleted**
During a broad test on September 10, **13 name-matched databases were unexpectedly deleted**.
I want to be precise about that.
I’m **not claiming that I have established that Replit itself deleted those databases**.
I’m saying that the incident occurred during this development process, the identities and contents were not logged in the information available to us, and deployment work was stopped while the incident was investigated.
That became another problem requiring time and investigation in a development process already consuming substantial Agent usage.
**Then the publishing failures continued**
Beginning September 11, I received Replit-generated notifications with the subject:
**“Publishing for Artist-Bos Failed.”**
Those notifications involved multiple projects/domains.
Another failure notification followed on September 14.
Eventually, we discovered a major difference between development and production.
Production was missing:
**15 required database functions**
and
**20 required trigger bindings**
that existed in development.
The application’s safety check detected that those protections were missing and prevented startup.
Replit now says this later problem wasn’t caused by the earlier Replit platform outage.
According to Replit, an August 30 code change disabled automatic setup of those database objects at startup.
But Replit also explained something that I think developers considering this platform should understand:
**Replit’s publishing process transfers tables and columns, but it does not transfer those functions and triggers.**
So those objects existed in development but weren’t present in production.
Replit considers that an application issue.
I think that raises a legitimate question.
**If an application is being developed inside Replit, with extensive use of Replit Agent, for deployment through Replit, and the development database contains required objects that Replit’s publishing process doesn’t move into production, who is responsible for making sure the customer understands that before production fails?**
I’ve asked Replit whether Agent created or contributed to those database objects and what warning was provided that these dependencies would not be transferred during publishing.
**There was also a 3.3 GB deployment problem**
Replit identified another September 14 failure caused by a **3.3 GB backup directory** exceeding deployment limits.
I’m including that because I’m not claiming every failure was Replit’s fault.
Some problems may have been application-specific.
Some may have been caused by development decisions.
Some may have been caused by Replit.
And at least one significant publishing period **was explicitly acknowledged by Replit as a Replit-side platform problem.**
That’s exactly why I want the Agent usage separated and examined instead of receiving a blanket answer that all Agent usage is non-refundable.
**Support became part of the experience too**
Replit says my account generated **34 support-ticket records**.
Replit says some of those were duplicates that were merged, and that the primary publishing issue was handled through one main support thread.
That’s fair, and I’m not claiming I experienced 34 separate outages.
But I think 34 support records during a relationship this short provides important context about how much time was being spent trying to resolve problems.
Replit itself also admitted:
“Our replies during this period were slower than they should have been, and I’m sorry for that.”
Eventually, I stopped trying to make Replit my long-term production platform.
**I started trying to get my work off it.**
**Meanwhile, this is what I was paying**
Replit has now provided its accounting of successful payments between August 10 and October 1:
**August 10 — $900.00 annual Pro subscription**
Then usage charges:
**$49.10**
**$250.36**
**$250.40**
**$253.36**
**$255.08**
**$250.80**
**$252.45**
**$250.11**
**$250.21**
**$250.94**
**$250.23**
**$250.22**
**$250.08**
**$250.38**
**$250.29**
**$250.60**
**$146.33**
**$250.58**
**$250.28**
**$57.74**
Total successful payments:
**$5,419.54**
Replit says the approximately $250 charges occurred automatically as usage reached billing thresholds.
Again, this happened over roughly **seven weeks**.
Replit says Agent completed:
**1,423 runs**
And approximately:
**96% of the usage was Max mode.**
**There’s another number I want Replit to explain**
During this period, Replit’s own dashboard showed **more than $10,000 in usage/activity**, overwhelmingly associated with Agent.
I am **not saying I paid Replit more than $10,000**.
I didn’t.
Replit’s own accounting now says the successful payments were **$5,419.54**.
I’m asking Replit to reconcile those numbers:
**How much usage was generated?**
**How much became invoices?**
**What credits were applied?**
**What payment attempts failed?**
**What was actually collected?**
**And what Agent activity generated those amounts?**
That’s particularly important because of what Replit says next.
**Replit’s reason for denying the Agent refund**
Replit told me:
“These charges reflect the work Agent did, whether or not a publish succeeded afterward.”
This is the part of the dispute I think matters far beyond my account.
I understand paying an AI Agent for productive development work.
I understand paying Agent to fix mistakes in my own code.
I understand that software development doesn’t guarantee that everything works on the first attempt.
But consider a different situation:
**The platform itself has a problem.**
The customer doesn’t necessarily know that yet.
So the customer asks the platform’s AI Agent to investigate.
**The Agent consumes paid usage.**
The customer tries the suggested changes.
It still doesn’t publish.
The customer asks Agent again.
**More paid usage.**
More investigation.
More changes.
More retries.
**More paid usage.**
Eventually the platform company determines:
**The underlying publishing problem was on our side. It wasn’t caused by you.**
But then it says:
**The Agent still performed work, so all of those Agent charges remain valid.**
That is the issue I’m asking Replit to address.
**And in my case, this isn’t hypothetical**
Replit has already acknowledged:
“A platform issue on our side blocked publishing…”
and:
“It was not caused by anything you did.”
So I want Replit to identify the Agent usage associated with that period and the attempts to diagnose or work around that failure.
If none of my paid Agent activity was related to it, show me.
If $20 was related to it, show me.
If $200 was related to it, show me.
If substantially more was related to it, show me.
**But actually do the accounting.**
Don’t simply tell me all 1,423 Agent runs are non-refundable because computation occurred.
**Replit also denied the annual subscription refund**
The annual Pro subscription cost **$900** on August 10.
Replit says that refund is also denied because the request falls outside its 30-day subscription refund window.
So despite buying a year’s service and reaching the point within weeks where I was trying to move the work elsewhere, Replit’s position is that the annual payment isn’t refundable either.
**Where that leaves me**
In roughly seven weeks:
**$5,419.54 actually paid**
**1,423 Agent runs**
**Approximately 96% Max usage**
**34 support records**
Problems existed before August 31.
Publishing failed August 29.
Replit then acknowledges that from August 31 through September 4, **its own platform blocked publishing through no fault of mine.**
Replit says there were 16 successful publishes afterward.
Then came the September 10 database incident.
Then additional publishing failures.
Then the discovery that production lacked 15 functions and 20 triggers present in development.
Then additional deployment/database repair problems.
Replit acknowledges its support responses were slower than they should have been.
Eventually I stopped relying on Replit and tried to move the work elsewhere.
Replit reviewed the account.
Its refund determination:
**$0**
**I’m not asking people to take my word for everything**
I have the invoices.
I have the publishing-failure notices.
I have the support communications.
I have Replit’s written explanation.
I have Replit’s admission that the August 31–September 4 publishing failure was on its side.
And importantly, **I’m including Replit’s explanations where Replit says a problem was caused by my application rather than its platform.**
I’m not interested in exaggerating this.
The actual numbers and Replit’s own statements are enough.
I’ve escalated the decision and asked Replit for the records necessary to separate legitimate development usage from Agent activity associated with platform/deployment troubleshooting.
If Replit changes its decision or provides evidence that changes my understanding of what happened, **I’ll update this post with that too.**
But there’s one question I think every developer considering heavy Agent usage should think about:
**If an AI coding platform charges you by Agent usage, who pays when that Agent spends your money trying to diagnose and work around a failure in the platform selling you the Agent?**
In my case, Replit’s answer so far appears to be:
**The customer.**

Sometimes the hardest part of debugging isn't finding the error.

It's figuring out whether the error you're looking at is actually the *root cause*.

I've had cases where:

Service A fails → Service B throws an error → API returns 500

The obvious approach is to investigate Service B first.

But the real problem was actually somewhere earlier in the chain.

How do you guys approach these situations?

Do you start from the first error in the logs, use distributed tracing, inspect the execution flow manually, or have another method that works better?

Interested to hear how people handle this, especially in larger codebases.