u/Lance_Saul_85

We turned on Copilot chargeback in Q3 with seats mapped to cost centres off the Entra groups and each team sees its own copilot license cost on a monthly report.

Engineering looked at theirs, saw 56% utilisation and handed back 41 licences inside the first month which is exactly what the report was built for. Finance was pleased right up until the next invoice came back identical to the one before it, to the dollar.

I did not know this and probably should have. Microsoft bills the quantity you bought and not the quantity you assigned. Pulling a licence off a user in Active users sends it back to the pool where it sits unassigned and still billed. The number that changes the invoice lives in Billing then Your products. Both counts show on the same screen in the admin centre and that is how you end up believing you have done something.

Then the second half of it, under new commerce terms the licence quantity can only be reduced in the 7 days after you buy or renew or add seats and we were 4 months past that window when the reclaim happened. The reduction now lands at renewal, 11 months after the behaviour we were trying to change.

The data was right and the team acted on it correctly, which is why this one still nags. The only wrong thing in it was me telling them 41 returned seats would show on their line this quarter.

The fixed one we cannot move while the variable half arrives switched on by default for new Business licences next month at $10 a user. I have no chargeback story for that one yet.

Just last week, I pulled the pager and spent more of it clearing Jira tickets than doing anything on-call is meant for. Security stood up a nightly scan a while back that opens a ticket for every high and critical. Most of our services have an empty owner field meaning they land on whoever is holding the pager. That was me.

2-3 days later there were something like 30 waiting, all kinda the same with CVE number, a package name and due date. But no note on whether it is even loaded, which service drags it in, nothing I can act on without going and digging it out myself.

I came across one yaml parser flagged critical which turns out only a build tool pulls it in and it never lands in a running image. I note that down, close it but theres like a dozen more behind it all with the same story.

I get wanting a clock on the real ones because right now the one on-call gets to decide what is real off a ticket that tells them nothing, that was never the deal. How are you lot stopping the stuff that never runs from becoming a ticket in the first place?

Our Cloudability renewal is up in a few weeks and I have to call it, renew or switch. What forced it is finance. Agent spend was a rounding error a year ago and it's about a quarter of the cloud bill now and when our CFO asked me to split it by team I couldn't. It all lands in one "other" row cloudability won't break down.

I’ve done the rounds on cloudzero, vantage and pointfive. Cloudzero and Vantage are good, more engineer-friendly than Cloudability even, but the coding agent spend just shows up as a provider total, no per-team split. Pointfive broke our Claude Code and Cursor spend out by team on our own data without me building a tagging pipeline first, and it runs the fix after an approval. It's newer though, and I keep seeing the line that it's not battle tested at big spend, which is fair.

What I'd want from those live on pointfive is whether the waste it flags is stuff cloudability already nagged me about or spend we didn't know was running.

We closed an acquisition last month, roughly 20 sites and a few hundred people and the integration plan lands all of it on me with a deadline that assumes this is easy. It is not.

But their network has different firewall vendor at every site, an MPLS contract with two years left, their own VPN for remote people and a security stack chosen by someone who is no longer there. Ours is different in every one of those ways. The plan says connect the two, apply our policies and cut over before the transitional service agreement runs out.

Every path I map is ugly. Site to site tunnels between two different firewall platforms that no one wants to own. Rebuilding their access rules by hand. Waiting on their MPLS to expire while paying for it. And doing it without breaking the business we just paid a lot of money for.

What I want is a way to bring their sites and users onto one platform quickly, apply one set of policies and not inherit more boxes I have to babysit. Ideally without ripping everything out because the business has to keep running through the cutover.

Those who’ve merged two networks after such an acquisition, what worked?

I do not know how other teams put up with this. Our security scan runs on every PR and re-reads the whole repo each time, it is always the slowest step by a mile, slower than the build and the tests put together. During an outage last month it sat on a one line fix long enough that a lead just force merged past it and I did not blame him.

And now that is normal. People skip the scan whenever they are in a rush, which is most of the time then it still shows green on the dashboard so on paper nothing is wrong. A gate everyone routes around is not really a gate, it just looks like one in the audit.

So I am trying to work out how you make it fast enough that skipping it stops being the obvious move. Incremental scans that only look at what changed. Something in the editor you see it while you type instead of waiting on the pipeline. Both maybe. What is working for you, and did you give up coverage to get the speed.

I think we have been staring at the wrong number when it comes to measuring container security. Total CVE count is what everyone tracks. It is a snapshot, a stock. It tells you what a scanner found today and nothing about tomorrow. Two images with the same count can behave completely differently over 6 months one stays quiet, the other throws a new critical every week.

What we started tracking is frequency of how much time passes on average between new CVEs appearing in a given image. A base dragging in a huge pile of packages gets a new vuln constantly because there is so much surface for a disclosure to land on. A minimal one goes quiet for weeks because there is just less in it that can be found vulnerable.

And that resulted into far better prediction on triage load. The noisy images had the biggest number on day one and they were the ones with the most surface accumulating new findings over time.

Have you tracked frequency like this or changed how you measure container risk?

Spent a while assuming spend was climbing because usage was climbing. It wasnt, request volume was flat, cost per request was the thing that moved.

Two things once we instrumented it properly. There was a retry on timeout that in certain failure modes fired three times on one request and every one of those was a full price call. Separately, our context had grown because people kept appending to the system prompt over months and nothing ever removed, so it was up near 4k tokens of accumulated instructions with some of it contradicting other parts of it.

The retry was a bug and thats fixed. The prompt is more of a people problem, everyone who added a line had a reason at the time and nobody wants to be the one who deletes someone else's guardrail.

Anyone got a sane process for stopping a shared system prompt turning into that. We've talked about ownership and reviews but it feels like it needs to be more boring and automatic than a review.

Been chewing on this for a while. We run a SAST tool on our Go services and I have realized none of us can say how accurate it is. It throws findings, we work through some, we ignore a lot and we have never once measured how many were real versus noise. We just vibe it.

What set me off was two scanners disagreeing hard on the same repo. One flagged 30 criticals while the other found 6. They cannot both be right and I have no way to tell which one is lying to me. Everyone sells accuracy now, especially with all the AI code we are pushing but no one hands you a way to check the claim.

How are you all measuring whether your scanner is right, not just whether it is loud.

Postgres, Redis, nginx. We moved everything onto continuously rebuilt minimal images about 6 weeks ago to cut down the Debian CVE noise we were drowning in and it helped.

Every bump still goes through the test suite. No shell makes debugging painful though.

Anyone else do this and still feel like it was worth it?