r/devsecops

Building an evaluation shortlist and trying to separate current reality from reviews written when the AI coding tool landscape was different. Most SAST tools were designed for codebases written entirely by humans. That assumption is no longer valid for most orgs and I want to know which tools have adapted versus which ones added AI language support and called it done.

The questions I keep coming back to are real-time scanning inside AI-assisted editors versus post-commit only, whether the correlation layer handles mixed human and AI code patterns, and how false positive rates hold up when a significant portion of commits come from AI suggestions.

Been evaluating AI coding assistants for rollout across a 200-person engineering org and repo graph drift is the problem that keeps disqualifying tools that look good in demos.

The pattern is consistent, the tool indexes at setup, looks great for the first few weeks, then the codebase evolves and the index doesn't keep up. Service A gets renamed. An internal SDK gets a breaking change. A shared library gets deprecated. The tool keeps reasoning from the old state and the suggestions get quietly wrong in ways that don't surface until integration or review.

Vendors use the term codebase intelligence without defining it and that vagueness is where the drift problem hides. Some mean file-level autocomplete. Some mean a one-time index of your project directory. Very few mean what I'd actually call codebase intelligence: a continuously updated understanding of your full repository graph, cross-service dependencies, and shared library state that reflects what's actually deployed right now.

For a DevOps context the third definition is the only one that matters. I'm documenting our evaluation criteria here because I suspect others have hit the same gap between vendor claims and operational reality.

we scan all our running containers weekly. most come back clean, no critical CVEs.

but some of those images haven't been rebuilt in six months. the scanner does catch new CVEs in old images  Trivy and Grype pull from a continuously updated vulnerability database, so a CVE published last week will show up against a six-month-old image in this week's scan. detection isn't the problem.

the problem is remediation. scanner fires, ticket opens, nobody rebuilds because the service hasn't changed and devs don't want to touch something stable. so the CVE sits open, aging, with no clear owner pushing the rebuild.

security wants a policy: "no image older than 30 days in prod." devs say that's impossible because some services genuinely don't change that often and a forced rebuild just to rotate the base image breaks their release process.

what we need is a way to rebase on a patched base image without triggering a full app rebuild and re-test cycle every month. right now those are coupled and neither side wants to own the cost.

how are you enforcing image freshness without forcing useless rebuilds? 

Hot take: agentic workflows are basically SAST/DAST, just with a reasoning loop on top.

We’ve been experimenting with systems that don’t just run static or dynamic checks once and stop there, but continuously loop : checking code, exploring runtime behavior, revisiting assumptions, pivoting when something interesting shows up…...

And honestly, once the system starts understanding context instead of just matching signatures, things get interesting fast.

Especially around:

• logic flaws
• weird edge cases
• multi-step exploitation paths
• “this technically works but absolutely should not” type bugs

That said, current models still hallucinate, lose context, and do pretty dumb things pretty often, so this definitely doesn’t feel like “AI replaces AppSec engineers” territory at all.

But it does feel like security testing workflows are starting to shift in a meaningful way.

Curious if other people are seeing the same thing or if this still feels like AI hype from your side.

We’ll be digging into this more in a live session soon if anyone wants to join, challenge the takes, or just nerd out about where AppSec tooling is heading.

Every morning its the same cycle:

Locked out again.

Can you install this app.

PN is not working.

How do I access the shared drive.

We support around 200 remote employees and the repetitive tickets are eating up the entire day. The frustrating part is most of these are simple fixes, but they still need someone from our team to jump in manually.

I am working on AgentDiff, an open-source tool that records which AI agent wrote which line ranges in a repo.

The current flow:

1. `agentdiff configure` installs hooks for AI coding agents.

2. `agentdiff init` enables tracking inside a repo.

3. Agent sessions write to `.git/agentdiff/session.jsonl`.

4. On commit, traces are finalized into `.git/agentdiff/traces/{branch}.jsonl`.

5. On push, traces are uploaded to `refs/agentdiff/traces/{branch}`.

6. A GitHub App reads those refs during PR events and posts a check-run.

The reason I chose git refs instead of an external database:

- repo-native

- branch-aware

- works with normal GitHub APIs

- branch protection does not block the custom ref namespace

- traces can be consolidated into repo metadata later

The demo is live here at the dashboard:

https://agentdiff.site/

The os repo is here:

https://github.com/codeprakhar25/agentdiff

I would love feedback from people who maintain CI/platform workflows and how useful these can be in maintaining code security!

We are investigating unauthorized access to GitHub’s internal repositories. While we currently have no evidence of impact to customer information stored outside of GitHub’s internal repositories (such as our customers’ enterprises, organizations, and repositories), we are closely monitoring our infrastructure for follow-on activity.

- Github 5:18 AM · May 20, 2026

Update !

Dark Web Informer says "GitHub source code allegedly offered for sale: Internal orgs and private repositories claimed

A threat actor using the alias TeamPCP claims to be selling GitHub source code and internal organization data.

The actor claims the dataset includes around 4,000 private repositories and says samples can be provided to interested buyers to verify authenticity.

━━━━━━━━━━━━━━━━━━━━
Target: GitHub
Country: United States
Sector: Technology / Software Development / Source Code
Incident Type: Alleged Source Code Sale
Claimed Exposure: Around 4,000 private repositories
Actor: TeamPCP
Price: Offers over $50,000
━━━━━━━━━━━━━━━━━━━━"

TeamPCP hit them via VSCode extension:

https://www.aikido.dev/blog/github-breached-vs-code-extension

https://www.linkedin.com/posts/mccartypaul_its-confirmed-github-microsoft-has-been-activity-7462661577641627648-1owT?utm_source=share&utm_medium=member_ios&rcm=ACoAABRHazYBpN_HRZdThaEksdSCSVg-EfF1CcU

Thanks rowrowrobot for Info!

Our leadership keeps bringing up AI agents and autonomous support tools in every meeting because apparently every company is trying to reduce ticket volume this year. The problem is most of the tools I have tested either hallucinate answers, cant actually take action, or create more work for the IT team instead of less. I am curious if anyone here is actually using AI successfully for internal IT support and automation or if we are all still in the testing phase.

Moved from embedded dev to platform engineering, became the cyber champion on our team, and have been loving the work. The most fulfilling thing I've done so far is building a pipeline that automatically rectifies CVEs across our microservices, and runs tests to validate that nothing regressed.

The only issue is the 24/7 on-call rotations. I know the quality of on-call depends on the company or team, but from what I've heard, it seems like platform and sre teams typically have an especially high load.

I'm trying to figure out a good role to pivot to that overlaps with platform engineering, but with less chance of having 24/7 on-call, and devsecops seems like it might be the one.

hi.. we have a few services in prod and logs keep piling up, but making sense of odd behavior is still hard. we tried Grafana with Loki. it works for basic logs, but not great for following request paths or understanding errors across services. looking for something that helps make sense of what the code is actually doing in production, especially when things dont behave as expected. anyone running setups that help track code paths or anomalies without a heavy setup? whats actually working for you?

We've been generating SBOMs for a year. They sit in a repo nobody opens. Our compliance guy asked about them once during an audit, I showed him a JSON file, he nodded, that was it.

Under CRA that changes completely. SBOMs become legally required documentation and should be machine-readable, and continuously updated. Covering at least top-level dependencies for every product you ship to EU customers. They're not a nice artifact you attach to a release note anymore. They're basically evidence at this point.

If your SBOM pipeline is we'll generate it when someone asks or the CI job does it but nobody checks if it's complete, now is the time to fix it. September 2026 is four months away and incomplete SBOMs are the kind of thing that looks fine until a regulator asks.

Just thought you should know : )

Here's a real example of how development secrets make it to production. Auditing an open source project — mid-size repo, actively maintained, real company behind it. The docker-compose.yml ships with 10 hardcoded credentials across 5 services:

• Admin password: secret
• Session secret: random
• S3 access key: admin / secret: secretsecret
• PostgreSQL password: secret
• MinIO root password: secretsecret

The README documents secret as the default admin password under "Getting Started." No warning to change it before deployment. No .env.example. No SECURITY.md. NODE_ENV: dev set in the same file. Intended for production use. The credentials alone get you in. A second finding compounds it: user-controlled input reaches a raw HTML render without sanitization in the audit log component — stored XSS as a second vector. Two findings, one deploy, full chain. The pattern is consistent across projects. Credentials written for local dev, never rotated, shipped as-is. Everyone assumes someone else caught it before it went live.

---

How does your team handle secrets before docker-compose goes anywhere near a server?

---

passed SOC 2 Type 2 last year. same controls, same environment, no changes since.

this year we're getting hammered not by our auditor but by enterprise procurement teams doing their own security reviews before signing contracts. they're asking for things our SOC 2 report doesn't cover: full SLSA Level 3 provenance for all container images, cosign signatures on every artifact plus SBOM attestations in a format they can ingest 

the SOC 2 trust services criteria don't require any of this, our auditor isn't asking for it. but three separate enterprise deals this year have stalled in security review because we can't produce signed image provenance or a machine-readable SBOM.

we tried adding cosign to our main pipelines but we have 40+ repos and the engineering time to retrofit all of them plus set up a transparency log and key management is significant. security wants it done before Q3.

anyone navigating this gap. SOC 2 passes your auditor but doesn't satisfy enterprise buyer security reviews? how are you prioritizing which images to sign first and what's the minimum viable provenance story that unblocks deals

Every security vendor in my inbox this month is using the word agentic. Agentic detection, agentic scanning, agentic remediation. I’ve got a rough intuition for what it means when applied to AI code assistants but no clear sense of what it means when applied to the tool securing that code.

As best I can tell the claim is that rather than running a scan and surfacing results, an agentic security layer is making decisions, taking actions, and adapting based on context without waiting for a human to tell it what to do. But the range of things vendors seem to mean by that is enormous.

What does agentic actually look like in practice?

Last July I tracked 189,600 potential AI API key matches in public GitHub code search.

The latest snapshot is 435,608.

Important caveat: these are potential matches, not confirmed active keys. They can include examples, revoked keys, test strings, and false positives. No secrets or repository contents are stored.

Still, the trend seems worth discussing: as AI agents connect to email, databases, MCP servers, and production workflows, leaked provider keys become more than a billing problem.

Curious how teams here are handling this in practice: pre-commit scanning, GitHub secret scanning, CI gates, key rotation, developer training, something else?

Trying to do the right thing here and integrate CSPM scanning into our deployment pipeline. Security team wants every deploy scanned for misconfigs before it hits production. Devs are ready to riot and honestly i get where theyre coming from.

The scans take 12-15 minutes per run. When youre shipping 20 times a day, thats hours of cumulative waiting. On top of that false positives keep blocking non-critical changes, so devs are losing trust in the tool.

We tried the scan everything approach. It lasted about 3 days before someone found a way to skip the pipeline entirely. Now security is mad about shadow deploys and devs are mad about being treated like children. Nobody won.

Current compromise is scanning only production-bound deployments and letting staging and feature branches through with just linting. But i can already hear the security team sharpening their pitchforks about reduced coverage.

For those whove actually made CSPM in CI/CD work, what did your setup end up looking like? Scanning everything or just prod? And how did you handle the false positive problem without security losing visibility?

been spending time on red teaming our internal LLM setup and starting to feel like we’re stuck at the basics.

rn it’s mostly prompt injection and jailbreak-style tests. ignore instructions, roleplay tricks, that kind of thing. useful early on, but it doesn’t feel close to how these systems actually fail in production.

we’ve got a few internal models that need proper testing before they’re used in anything sensitive. the current approach works for demos, but it doesn’t give much confidence once real usage starts. trying to go beyond that, but it’s not very clear what a more complete setup should look like. things like data exfil, model inversion, or abuse through tool integrations seem more realistic, but harder to test in a structured way.

the tools we tried so far either generate generic prompt lists or don’t give much insight into why something worked.

what teams are actually testing once they move past basic injections?

Saw Cal.diy trending on Hacker News, looked cool, asked Claude to compile and run it locally. Took maybe 5 minutes. Worked flawlessly. Felt like magic.

Then i did the thing you should probably do before the cool part. Ran Grype.

Three container images. calcom, postgres, redis. 1,673 vulnerabilities total. 31 critical. 159 high.

None of them in the app code btw. Every single one came pre-installed in the base images. I hadnt touched a line and was already buried in CVE debt.

AI can build software faster than we can check if its safe and were all just... shipping it. Anyone else had this wakeup call?