u/iamjessew

This week I found a community-maintained KitOps package on conda-forge.

A contributor packaged the Kit CLI, wrote the recipe, and now anyone in the conda ecosystem can run:

conda install -c conda-forge kitops

Best of all, they are. So far the package has over 1.7k installs! And, I'm honestly wondering how I missed it(!?)

Conda is where data scientists live, which is really exciting. We typically see KitOps adoption starting with engineers, devops, and platform teams who are used to using containers and familiar with Docker workflows. Seeing it growing on a data scientist community is awesome.

Open governance means the community can carry the project into places we'd never prioritize on our own roadmap. That's how standards form.

To the contributor who built the feedstock, thank you.

This is what an open AI supply chain looks like in practice. If you're packaging models, agent skills, or MCP servers and want them versioned like everything else in your registry, KitOps is one conda install away.

I find it interesting that when I speak with companies implementing AI that there is an assumption that the more sophisticated they are (Eg, self-hosting agents vs using a foundation model desktop app) is somehow more secure. I think this is from a false sense of control. "It's on our instance, so we control it." when in reality, there's very little control at all.

Hey all, I'm a project lead on KitOps, a CNCF project. I've been talking with an ML team (Arlequin AI, they do topological deep learning for market intelligence) about how they moved from ad-hoc model packaging to standardizing on KitOps. The part that surprised me wasn't the tooling. It was who adopted it fast and who didn't.

Their before state will look familiar: weights pulled straight from Hugging Face or baked into Docker images, datasets in LakeFS, experiments in MLflow, configs somewhere in the ether. Nothing tied a model version to the dataset it was trained on and the config it needed to run.

They already used Docker for their apps, so they went looking for the equivalent for ML and landed on packaging everything as ModelKits. Model card, dataset references, MLflow pointer, configs, all in one versioned immutable artifact in the container registry they already ran. Promotion to staging or prod became a registry operation. Shadow traffic and A/B tests run against a specific artifact, so there's never a question about which version, trained on which data, is serving which slice of traffic.

Here's where it got interesting. Engineers were already fluent in Docker and OCI, so it felt native and they moved fast. Researchers needed education on what OCI even is and why it exists. I keep seeing this same split on other teams. The tools we take for granted on the infrastructure side are genuinely foreign one desk over, and rollout plans almost never account for it.

If you've standardized packaging (with KitOps or anything else), how did you get your research side on board? And if you tried and it stalled, I'd honestly like to hear that too.

I've been hearing a lot of talk about cost optimization/token usage lately. The solution is always the same, route specific tasks away from generalized foundation models to self-hosted open source models that are trained to be task-specific. This (like all things) is easier said than done. Creating the routing policy is a massive task/challenge by itself before you even factor in training the task-specific models then getting them into production.

So my question is if anyone (outside of a FAANG resourced tech company) actually doing this?

Disclosure up front: I'm one of the co-founders of Jozu ... buy am curious about a pattern we keep running into.

I consistently see two gaps in our prospects AI patters.

First (and this shouldn't be a surprise to anyone) teams are pulling models off HuggingFace, agents and MCP servers off GitHub and public registries, and then wiring them straight into production.

The problem is that when you ask which of those are signed, which got scanned, and which are a black box, no one knows. This is crazy because the npm Shai-Hulud worm (which everyone claims to be aware of) showed how fast a poisoned dependency spreads.

Second, most teams can tell you if an agent is authorized, but very few can tell you which tools it's allowed to invoke, what data it can reach, or what should trigger a human approval before it acts. Obviously this is an issue.

The interesting part though, is that these teams put themselves a stage further along on the sophistication scale than they actually are, yet when you push on it they backtrack and lean on 'still figuring it out'.

Genuinely curious how people here are handling this, and specifically handling governance of the runtime side. Are you gating tool access? Logging agent actions in a way you'd trust in an audit? Or is it mostly still trust-and-hope right now?

** If useful, I'm considering having our team pull together some content on this, walking through both gaps and a quick diagnostic to help teams evaluate their program honestly. LMK if that's of interest as well.

hey all, I'm a project lead for CNCF KitOps and one of the founders of Jozu.

Last week I was on a call with a community member who was exploring our commercial offering. We were talking about runtime policy enforcement and took a sidebar over to our supply chain security offering. We offer a Huggingface import feature in KitOps that automatically packages it into a ModelKit, it's then pushed to Jozu Hub where it's scanned for a bunch of stuff like tserialization attacks, data poisoning, backdoored weights, adversarial susceptibility, prompt injection, and license/dependency issues, etc

If something is found, we suggested quarantining the ModelKit. Where to my surprise, the individual responded negatively to that suggestion, making it clear that they didn't want to unnecessarily block the dev process.

I understand this point, and have felt the pain of human in the loop blockers. In a perfect world, you could have an agent in the loop to run this process, which would be faster, but personally wouldn't trust one for this purpose.

So I'm curious, is this an outlier? I find it hard to believe that most orgs would be okay with a compromised model floating around in their registry?