r/codereview

Coderabbit noise is seriously getting out of hand

feel like CodeRabbit has gone too far in the direction of commenting on everything it possibly can.

on bigger PRs especially, you can end up with a wall of review comments where a few are genuinely useful but a lot are things i'd never block a PR over.

and at some point that makes the good comments less useful too. if engineers get used to dismissing half the bot's comments, they're eventually going to skim the one that actually matters.

saw Theo mention macroscope recently so gave it a try and it seems much better on noise. BugBot too.

at this point i don't really care which reviewer leaves the most comments. i just want the comments it does leave to actually matter.

when does an AI reviewer become too noisy for you?

reddit.com
u/NextSubject227 — 1 day ago
▲ 17 r/codereview+4 crossposts

GitHub - Strategic-Automation/violin: Violin — a supervised, agentic Hermes Agent pentest profile (31 playbooks, 10 references, guard plugin) for authorised recon, exploit validation, and reporting. Hermes-native, no extra keys.

Violin is now at v3.0.1 on master.

It is a Hermes-native profile for supervised, authorised penetration testing, with:

• 31 methodology and vulnerability playbooks

• An execution guard at the target boundary

• Evidence-backed findings and reporting

• Structured scoping and approval gates

• No additional credential broker or provider lock-in

Install:

"hermes profile install https://github.com/Strategic-Automation/violin"

I’m looking for Hermes users and penetration testers to test the installation and engagement workflow and report where the guard, evidence capture, or reporting process creates friction.

github.com
u/WarmAd6505 — 2 days ago
▲ 150 r/codereview+9 crossposts

asm.fm — a chiptune synthesizer in pure x86-64 assembly (no libc, no audio lib)

Learning project that became my favourite: a chiptune synth written entirely in x86-64 assembly (Linux, NASM). No libc, no audio library — just computing raw 16-bit samples and writing a WAV header by hand.

The premise is that sound is just a list of numbers (44100/second) describing where a speaker sits. So the whole synth is: generate the numbers, write them out.

It does four oscillators (square/saw/triangle + LFSR noise), polyphony by mixing voices into one buffer, ADSR envelopes, and FM synthesis with a hand-built sine table. Working on effects next (vibrato, delay, reverb).

github.com/whispem/asm.fm

Feedback on the low-level details welcome — especially the fixed-point math in the FM operator.

u/whispem — 3 days ago

Will you use this? (Not AI generated)

Hi guys,

I have been working with a product for sometime and I believe will be useful for startups whose products are mainly APIs and SDKs.

I have asked for validations before I started building and have seen startups who have built internal tools for this. So the problem should be real upto an extent imo.

While I am working on it, I would like to validate it more. So I want you guys to see if this is relatable for you and will you pay for it. (I have a case study of a startup as well, would love to share and discuss in a gmeet if this aligns with you)

Here is what I am working with,

The problem:

API first companies struggle to keep their backend code, OpenAPI specs, SDKs, API docs, examples, and packages in sync as the API evolves. A coding agent struggles to implement it if there are any drifts in these.

The solution:

A single source of truth that connects your API code, OpenAPI specs, SDKs, docs, examples, and packages, and automatically detects and propagates changes across all of them.

To put in simple terms, you focus on the main product, which is the API. Everything else stays in sync automatically.

\---

Above is the one main problem to focus. But this is also an additional part I am working on:

A coding agent struggles to implement it if there are any drifts in these. So after each evolution, the system can run the API/SDK changes in a sandbox and test whether a coding agent can actually implement and use the updated API/SDK successfully.

But the main problem I am trying to solve is keeping the API ecosystem in sync.

\---

Any inputs, feedback or suggestions will be very helpful. Please comment anything you can think of when you see this. Thank you in advance.

reddit.com
u/Over-Alternative7275 — 3 days ago
▲ 3 r/codereview+3 crossposts

Try Benzi- A coding agwnt that _queries_ your codebase instead of reading it

Benzi is a compiler + runtime tracer + harness and Al agent built to understand code from ground up. Challenging traditional RAG and embedding space approaches, Benzi aims to write code as cleanly as it understands it.

77.4% SWE-bench Verified (#4 on the leaderboard) for less than $30. (using deepseekv4flash. Benzi is model agnostic)

Also included in the benchmarks page is proof for mechanism that makes this possible.

Any feedback is greatly appriciated!

reddit.com
u/DonkeyTheKing — 3 days ago
▲ 1 r/codereview+2 crossposts

Ran a "looks good, solid implementation" AI review through five questions afterward. It failed four of them.

Pulled up an AI code review from a few weeks back that had approved a PR touching retry logic near a payment flow. At the time it read as thorough, numbered comments, a couple of suggestions, a clean summary at the end. Went back afterward and checked it against five specific questions instead of just trusting the tone.

Did it establish what was actually at risk? No. Nothing in the review distinguished the retry logic from a comment on variable naming, both got roughly equal attention. The model wasn't told this touched a payment path, so it had no way to weigh it differently.

Did it check a specific failure mode? No. The comment was "should probably check for duplicates," which sounds like a finding but is actually a hedge. Nothing tested whether the retry would double-charge under a duplicate request with the same idempotency key.

Was severity justified? No. The duplicate-check comment sat at the same visual weight as a docstring suggestion. Nothing forced a distinction between "this could cause an incident" and "this is a nitpick."

Did it state its own scope? No. Silent about whether it had visibility into the caller, or the idempotency key generation happening elsewhere. Silence read as "nothing else to worry about," which is a much stronger claim than "I didn't check that part."

Was there a confidence check on its own findings? No second pass existed. A confidently wrong suggestion and a confidently correct one look identical in tone, the only way to actually tell them apart is asking what would prove it wrong and then checking.

Four out of five, on a review that read as completely fine at the time. Re-ran it with risk context supplied up front, a specific failure scenario framed explicitly, severity tags required with justification, and a stated scope, and the duplicate-charge risk became the one clearly flagged blocking issue instead of one line sitting level with a naming suggestion.

The uncomfortable part wasn't that the AI missed something. It answered confidently either way, whether it had actually checked or not, and confidence was the only signal I had to go on until I started asking these five questions deliberately instead of trusting the shape of the output.

Wrote the full breakdown of the five questions with the before/after comparison here: https://medium.com/@nagatomopedro05/five-questions-your-code-review-should-always-answer-66be919bb200

Worth running your own last "looks good" AI review through the same five questions. Curious how many people find theirs holding up better than mine did.

u/ClickOk5811 — 3 days ago

Greptile: Please cancel my upcoming unexpected $30 OSS billing

UPDATE (Aug 18): RESOLVED

I'm sorry for posting here.

Greptile change the OSS policy recently,

however, I only discovered some of my OSS repositorys was placed on a pay subscribption and that a $30 charge is scheduled for the next billing period, it will be charge 20TH AUG.

During setup, I was not clearly informed that adding a new repository would start a paid subscription. I also encountered a new requirement that an OSS repository must have at least 50 GitHub stars, but I could not find this requirement disclosed on the public OSS or pricing pages.

Please cancel the scheduled Pro renewal, confirm that I will not be charged.

If the $30 charge has already been processed, please refund it.

I was sent at least two emails regard this OSS billing, no any reply, so posting here.

My Github is keyang556

Thank you.

reddit.com
u/keyan556 — 3 days ago
▲ 14 r/codereview+9 crossposts

Open sourced bonsai-ninja: local code intelligence + security analysis built for humans and local first LLMs

Finally making bonsai-ninja public.
It’s a local code intelligence and static analysis engine built around compiler-backed facts instead of giant file dumps.

Use it for code intelligence, security taint analysis across 20 languages, or export call graphs, control flow, dataflow, IDG facts, and other structured compiler data for AI training, code-reasoning research, and agents.

It’s also built with local-first AI in mind. Pair it with your local LLMs to give them focused, compiler-backed context about a codebase instead of burning context windows repeatedly reading files. Models like Qwen and Nemotron are a great place to start, and experimenting with different local models is encouraged.

It’s free, MIT licensed, and runs locally. No subscription required, no hosted service required, and no analysis features held back behind a paid tier. Your source code stays on your machine.

Even if the engine isn’t your thing, the security-patterns alone are worth checking out. There’s an extensive YAML rulepack for modeling sources, sinks, sanitizers, frameworks, and security behavior across the supported languages.

This is just getting started. Contributions, testing, sharing, criticism, forks, whatever. Hell, steal it, improve it, build something better with it. Doesn’t matter. The goal is to put it out there and see what people do with it.

github.com/gromhacks/bonsai-ninja

github.com
u/GromHacks — 4 days ago
▲ 1 r/codereview+1 crossposts

Code review that follows callers across you repos

Started with our own review queue. Most of the code in a PR is generated now, so PRs land faster and bigger than anyone reads properly. First thing that goes is context outside the diff. Nobody opens the other repo to see who calls the function you just changed. Skim, approve.

So we automated that part. It resolves the symbols the diff touches, walks the callers through a code graph, and pulls in how your repos connect: routes, queues, shared tables, package deps. Sub-agents for architecture, security, logic, tests. One review comes out.

Weak spots: GitHub only, no benchmark, noise tuning needs work. Free tier is one repo crawl plus your first reviews, no card.

If you try it: what fraction of the findings would you have acted on?

https://contextgoblin.com/

u/LOSIHOIDAANACCOUNT — 3 days ago
▲ 13 r/codereview+2 crossposts

Code review

Hello, I'm an aspiring software engineer. I've recently finished developing microservice for managing projects and tasks using Spring Boot. I'd appreciate if you could review codebase of my project and provide feedback on it.

GitHub: https://github.com/Simpav-chill/tasktracker

u/Simpav1 — 5 days ago
▲ 5 r/codereview+1 crossposts

Something like LeetCode, but for code review (my experiment)

At my company we started talking about the fact that we generate more code than we can properly review, and bugs have started slipping through. I don't think we're the only ones.

Auto-reviewers don't fully solve this: some things they just don't see, and sometimes it's the opposite, they keep finding endless "improvements". So other than training the review skill itself, I don't really see a way out.

So I made my own experiment: something like LeetCode, but for code review. You open a realistic pull request, leave inline comments, give a verdict (approve / request changes), and find out whether you caught the planted bug. Some PRs have no bug at all: confidently saying "this is fine" is also a skill.

Right now there are pull requests for Python, Node and popular frontend frameworks (React, Angular). There are even a couple in COBOL. Some of the bugs I recreated from real incidents that made the news back in the day.

The project is an experiment for now and completely free: diffdojo.com

Would be interesting to hear what you think, and especially how you keep your guard up when reviewing generated code?

reddit.com
u/IllConfidence1771 — 4 days ago

OOP Project

Hi everyone, I didn't know much about OOP. So I made a project surrounding classes to help me out. I'm self taught so please be honest with your judgment of it. Its a little unfinished testing wise, but it feels bigger than it needs to be. And if anyone has anything I can read or project I can contribute to to learn best practice, I'm all ears! Thank you!

https://github.com/Eidos-bot/ClassBasedModulesPublic

u/Far_Programmer_5724 — 4 days ago
▲ 2 r/codereview+1 crossposts

We built an agent that scores its own output before it opens a PR — the architecture, and the three things that broke

Disclosure up front: I work on this. It's a commercial product (KeplerCrew, by AiChargeLabs). Happy to talk architecture either way, and I'd rather get torn apart here than in a sales call six months from now.

The problem we kept running into with agentic coding wasn't generation quality. That was fine. It was that nothing in the loop could tell us whether the output was actually correct before a human looked at it. So every change still queued behind a reviewer, and the reviewer was now reading more code than before. Net throughput barely moved. Faster typing, same gates.

What we ended up building is five stages, with sixteen phases distributed across them:

  1. Understand — reads the repo, its conventions, and the task intent
  2. Plan — decomposes the work into an ordered, safely sequenced plan
  3. Execute — writes the code and the tests against that plan
  4. Validate — scores the result against acceptance criteria; failures loop back into a fix cycle instead of surfacing
  5. Deliver — the verified diff lands as a pull request

Stage 4 is the part I think actually matters. Criteria get scored at every gate rather than once at the end, and a failed gate re-enters the pipeline instead of being handed to a human as "here's my attempt, good luck." The goal isn't to remove the reviewer — it's that the reviewer shouldn't be the one finding the bugs.

Three things that were harder than we expected:

Safely sequencing the plan. Naive decomposition produces steps that are individually valid and collectively broken — each one passes, the composition doesn't. Most of our planning work went into ordering and dependency detection rather than into the decomposition itself.

Cost predictability. Open-ended agent loops are financially unbounded by default. A task that retries its way to correctness can cost ten times what a similar task cost yesterday, which makes the whole thing impossible to budget. Capping spend per task without capping quality took more tuning than anything else we did.

Running with no egress. A lot of our buyers are regulated and their code cannot leave their network, so we support self-hosted and fully air-gapped deployment. Good for those deals, painful for every part of the system that quietly assumed it could make an API call.

The open question I'd actually like opinions on: how much of the review burden do you think can move to automated scoring before you'd stop trusting it? We've landed on "a human still approves the PR, but shouldn't be the first line of defence." I'm not certain that's the right line, and I'd rather hear where you'd draw it.

Happy to go deeper on any of the stages, the scoring model, or the air-gapped setup.

reddit.com
u/LeftMethod1154 — 5 days ago
▲ 10 r/codereview+6 crossposts

QA feedback wanted for a complete open-source production readiness checklist

Hey everyone! I’m building an open-source production readiness checklist with extensive testing and quality coverage.

I would appreciate feedback on test strategy, integration testing, acceptance testing, regression, test environments, test data, exploratory testing and nonfunctional testing.

If anything is missing, unclear or unrealistic, please let me know. Corrections and pull requests are very welcome.

Thanks!

https://github.com/MarinJursic/production-readiness-checklist

u/ClerkBeginning961 — 5 days ago
▲ 5 r/codereview+4 crossposts

Honest and Brutal Reviews on our Product Please .... !

I've been building Pinaka on the side for the past few months — an AI agent that automatically writes root cause analyses when a bug ticket is filed.

The problem it solves: every time a production bug hits, a senior engineer spends 2-3 hours investigating before writing a single line of fix. The actual fix takes 20 minutes. The investigation takes the rest of the day.

Pinaka eliminates that investigation step. Tag a Jira ticket or mention @pinaka-app on a GitHub Issue — it reads your indexed codebase, captures runtime context via an SDK, and posts a structured RCA as a comment. Root cause, exact file, exact line, fix approach. Automatically.

What I've validated so far

Ran it against real open source bugs:

  • BullMQ issue #2487 — code only scored 6.5/10, code + runtime context scored 9.2/10 against the actual merged fix
  • Prisma ORM issue #29480 — open bug, no ground truth, diagnosed from behavioral contrast across test files
  • Pinaka's own Java SDK — found a real ForkJoinPool exhaustion bug, fix shipped same day

All three write-ups are public, including the limitations.

What I'm looking for

couple of engineering teams (2-150 engineers) using GitHub who want to try this on a real production bug. Free, no commitment. I'll personally onboard you and be available for any questions.

Works with Jira and GitHub Issues. No Jira required. One OAuth click to connect. First 5 RCAs free.

Honest state of the product

No paying customers yet. Building in public. The product works — the benchmarks above are real — but I need real teams on real codebases to validate it beyond my own test cases.

If this sounds useful for your team, drop a comment or DM me. Happy to show you a live demo on your repo.

👉 getpinaka.com

u/atharvapanegai — 5 days ago
▲ 1 r/codereview+2 crossposts

How can I improve my github profile?

Hey guys, I have been adding new repos and taking my github seriosly after a long time
Most of time i open private repos to maintain my workflow on game/website projects
for last a few month i have been working on portfolio/resume projects to put on my website.
And i just like every time i see my contributions on panel, it is like idk how to explain but seems helping me to maintain consistency. I also realized there is achievement badge i have got.
So to pick it up, I am asking for ways to make my profile look better and be better. What can you recommend? Pins, repo names, maybe even links?

https://github.com/TheIlkinAlizade
this is my github, you can also see my porfolio website(which is old version, i will update it with new projects i have been working on)
thanks btw for spending your time to read this

u/Solid-Newspaper-3054 — 5 days ago
▲ 1.1k r/codereview+10 crossposts

Do zero-LLM health metrics predict where bugs land? Ran it across 21 repos, here's what held up

Most "AI reviews your PR" tools annoy me. They hallucinate nits, the output changes every run, and you can't tell if a flag means anything. So I wanted to know if you can get a useful review signal the boring way. Static metrics plus git history, without any LLM in the loop, so same input gives the same output every time.

The question I actually wanted answered: do deterministic code-health metrics predict which files get bug-fixed later?

How I set it up :

  • Score every file at a historical commit (T0).
  • Count bug-fixing commits to that file over the next 6 months.
  • Correlate the T0 score with the bug-fixes that came after. No file sees its own future.

Metrics are the usual suspects plus churn signals. McCabe complexity, deep nesting, LCOM4 cohesion, god classes, clone detection, function-level churn, code age, ownership spread, change entropy. 25 of them, combined into a 1-10 per file. I ran it across 21 OSS repos, 9 languages.

What came out:

  • Mean ROC AUC 0.74 at picking the files that go on to get bug-fixes. Up to 0.90 on some repos, weaker on others.
  • It survives controlling for file size (partial Spearman -0.16). So it's not just "big files have more bugs," which was my first worry.
  • Out-ranks churn alone by about +0.10 AUC, and prior-defect history by +0.12.
  • Held on an external dataset I never touched (PROMISE/jEdit), AUC ~0.77.

I ran it head to head against CodeScene too, same 2,770 files, same commit, same labels, since it's the closest established tool. Discrimination was close (0.73 vs 0.71). The gap showed up on effort-aware ranking. Under a "you only have time to review 20% of the changed lines" budget, the deterministic score surfaced more of the real defects (recall 0.17 vs 0.07, Popt 0.61 vs 0.46). Similar at telling risky from safe, better at ordering what to look at first.

Where it falls down, because 0.74 is not magic:

  • It tells you where bugs cluster, not what the bug is.
  • Ranks files, doesn't read them so not a review replacement.
  • Defect labels come from bug-fix commits, which is a noisy heuristic. Some "fixes" aren't, some bugs never got a clean fix commit. That ceiling is baked in.
  • The 6-month window is a choice. Move it and the numbers move.
  • The CodeScene edge is specifically ranking under a budget. On raw "is this file risky, yes or no," they're close.

Where I landed: the signal is real and it's reproducible, which is the part I care about for PR gating but it won't catch a logic bug in a 4-line diff.

It's good at "this PR touches a file that's been a problem for a year with one owner, slow down." Different job than line-by-line review, and I think it's the job the deterministic approach is actually good at.

I have also added agent provenance to the same which allows it to determine if an AI PR is worse than human

If anyone wants to take a look at the repo: https://github.com/repowise-dev/repowise

Also, if you have any feedback on the metrics itself or if I can try some new metric, would love to try that

u/Obvious_Gap_5768 — 8 days ago