r/deeplearning

We retrained Wolf Defender.

The main reason was not that attack detection was bad. The bigger issue was false positives.

The previous models were already good at detecting prompt injections, but especially on short benign inputs, security-related text, code snippets or ordinary conversations they could still be too aggressive. We also got a few reports from users that made this pretty obvious.

One example was just:

“Who are you?”

Wolf Defender Small previously classified this as a prompt injection with around 94% confidence.

For v2 we therefore changed the training setup quite a bit. Both Wolf Defender and Wolf Defender Small were retrained from fresh mmBERT checkpoints, with a much stronger focus on hard negatives.

That includes short conversations, emails, documentation about prompt injections, benign policy and system language, code and configuration snippets and generally inputs that contain words or structures which look suspicious without actually trying to manipulate a model.

We also added more counterfactual samples, multilingual examples, adversarial obfuscations and long-context injections at different positions in a document. Training combines short 256-token samples with full 2,048-token windows and uses supervised contrastive regularization, FreeLB adversarial training and Smooth-Max aggregation for long documents.

The main change can be seen in the benign benchmarks:

Model	Benchmark	v1	v2
Wolf Defender	Hard benign specificity	81.57%	96.23%
Wolf Defender	Real-world benign specificity	66.85%	96.63%
Wolf Defender Small	Hard benign specificity	82.12%	96.67%
Wolf Defender Small	Real-world benign specificity	73.60%	94.38%

At the same time, attack detection stayed roughly where we wanted it:

Model	Qualifire F1	Jayavibhav F1
Wolf Defender	95.14%	97.84%
Wolf Defender Small	95.21%	97.68%

There is also a tradeoff here. Some of the very high scores on our cleaner validation distributions went down slightly.

For us that is fine.

A security classifier with near-perfect benchmark scores is not very useful if normal traffic gets blocked all the time. We would rather lose a small amount on an easier validation set and get substantially better behavior on actual benign inputs.

The “Who are you?” example now gets classified as benign by Wolf Defender Small v2 with 98.55% confidence. A real instruction-override attempt is still detected as an injection with 99.99%.

We also updated the deployment variants. Both models are available as regular Transformers checkpoints and as ONNX exports in FP32, FP16, mixed INT8/FP16 and INT8 with INT4 embeddings.

The smallest Wolf Defender Small artifact is now 96 MB.

More details, benchmarks and model files are here:

https://huggingface.co/patronus-studio/wolf-defender-prompt-injection

https://huggingface.co/patronus-studio/wolf-defender-prompt-injection-small

If anyone is running prompt-injection classifiers on real traffic, I’d also be interested in which benign inputs still cause the most false positives for you.

Hi everyone,

I'd like to share a research project I've been working on for the past few months.

BCMT (Blockwise Causal Memory Transformer) explores an alternative approach to long-context language modeling.

Instead of propagating long-range information through dense global self-attention, BCMT keeps dense causal self-attention within local blocks and propagates contextual information through a compact exponential causal memory built from adaptive block summaries.

The goal is to investigate whether long-range dependencies can be modeled efficiently while reducing the computational and memory costs associated with global attention.

The paper presents:

• the complete BCMT architecture,
• the mathematical formulation,
• an open-source PyTorch implementation,
• an initial experimental evaluation on WikiText language modeling.

The initial experiments show validation perplexities close to a dense Transformer baseline while achieving higher training throughput and lower GPU memory usage.

I'd greatly appreciate technical feedback on the architecture, the experimental methodology, or potential limitations. Any technical feedback, suggestions, or criticism would be greatly appreciated.

Code: https://github.com/rachidlabs/BCMT

Paper (DOI): https://doi.org/10.20944/preprints202607.0333.v1

"Built my own game engine from scratch using just JavaScript! It was a great project for learning how engines work under the hood. Check out the repo here if you're interested: [Link]. Any feedback is welcome!"

https://github.com/programer321321/Game-Engine

I've never really been a reader. Books usually lost me a few chapters in.
My first attempt at learning machine learning was the usual route — one YouTube playlist after another. It felt like watching something, not learning it. Nothing really stuck.
So I picked up Hands-On Machine Learning by Aurélien Géron. And somehow, I ended up reading a 1000+ page book . Every chapter, I ran the code myself, broke it on purpose, and debugged it until I understood why it worked — alongside college lectures, assignments, and exams.
Somewhere along the way, something shifted in how I learn.

I stopped reaching for the fastest explanation and started reaching for the actual source — documentation, research papers, and technical writing I would've previously skipped for a quicker video.
In the middle of learning the ML pipeline basics, I built a GoogLeNet-style CNN with a custom DepthPool layer, and many more things at low level.

That's when it stopped feeling like an exercise and started feeling like something I could actually own — chasing shape mismatches, tracing silent preprocessing bugs, and retraining models more times than I'd like to admit.
From there, I kept rebuilding things: RNNs, attention mechanisms, transformers, autoencoders, GANs, diffusion models, RL. Each one broke in a different way, and each one taught me something different when I had to figure out why.
I'm still going deeper into Computer Vision and NLP from here. Those are the areas I keep getting pulled toward.
I still think YouTube has its place.
But this book is what made me a reader in the first place — and now research papers and documentation are where I actually go to learn.
Still early in this. Still building. Just glad I stuck with it.
hashtag#MachineLearning hashtag#DeepLearning hashtag#ComputerVision hashtag#TensorFlow hashtag#Keras hashtag#LearningInPublic

I got this paragraph from Claude, could someone please explain this and verify if it's a real thing or hallucination:

Overparameterization isn't just about final capacity, it's about the optimization process itself. A wide, overparameterized network gives gradient descent a much friendlier loss landscape — more paths downhill, fewer bad local minima, room to explore before committing. The "core" only emerges as a byproduct of that search happening in a much bigger space than it needs to end up in. Strip the space down first and you've removed the thing that let the search work.

Conversation: https://claude.ai/share/8813a637-c327-4d0c-b120-def27e5203d5

Hey everyone,

I launched SlideSieve on Product Hunt today. Honestly I've had the launch ready for almost a month but kept putting it off because I was nervous about putting my work out there. Finally hit the button today.

I take a lot of online courses on DLAI, Coursera, Udemy, YouTube, etc. Whenever I'm watching a lecture, I want to save the slides for later. The problem is that manually screenshotting kills the flow: you're constantly pausing, snapping a screenshot, trying to avoid catching the video player UI. And you still end up with a messy folder of images, or miss a slide and have to scrub back to find it.

That's the problem SlideSieve is meant to solve. It's a browser extension that runs in the background and captures slides automatically while the video plays, so you don't have to pause.

A few things I built into the capture logic to make it actually usable:

• Skips frames where the instructor's face fills the screen
• Ignores code, terminal windows, and dark transition screens, so it only grabs actual slides
• Exports everything as a clean, ordered PDF once you're done

Product Hunt has a lot of big platforms promising to change everything. This isn't that. It's not an AI "second brain," it won't take your tests for you. It's a small, focused tool to make studying a bit less annoying. There's a free tier to try it, and a one-time purchase for unlimited captures — no subscription, no cloud processing, everything runs locally in your browser.

Link if you want to check it out: SlideSieve on Product Hunt

Built this to fix my own studying mess. Nervous but glad to finally share it.

I've been curious whether you can kill an Claude's text watermark just by editing (not "rewriting/rephrasing") what it wrote. And so I built a Claude/OpenAI/Gemini text-watermark generator plus a detector and threw a bunch of attacks at gpt-oss-20b and Qwen outputs to find out. The technique relies on the famous Tournament Sampling built upon standard Gumbel-max sampling.

It turns out almost everything people assume works, doesn't. Swapping em-dashes for hyphens, stripping markdown, converting AmE to BrE spellings... none of it moved the needle. Across nearly 300 test runs only one attack crossed the detection threshold, and that was deleting 40% of every word, which just wrecks the text.

After rigorous benchmarking, the only method that consistently beat 10/10 times was inserting invisible Unicode variation selectors (the same characters used for emoji and CJK rendering) throughout the text. Performed that to about 30% of characters and the watermark score dropped from 45 down to under 1. And unlike every other invisible character trick I tried, this one survives normalization, because these are real meaningful codepoints that a normalizer can't safely strip.

Interesting finding: Code is barely watermarked to begin with. Watermark strength tracks how uncertain the model is about the next token, and code is low entropy, so some code samples come out basically unwatermarked with zero attack at all.

Not the first repo doing this kind of attack, I know, but I wanted to actually spend the weekend testing it properly across a few different open models instead of rushing something half-baked out.

Repo with all the code and results: https://github.com/aloshdenny/claude-awm

Interactive demo where you can try the attack yourself: https://aloshdenny.com/claude-awm/

Check it out and let me know what you think!

since jan 2026 i've been building Forge, a deep learning framework written entirely from scratch in C++ - no PyTorch, no TensorFlow underneath.

Eigen handles most of the math backend. btw i wrote some custom AVX2 SIMD kernels (element-wise ops) too, and OpenBLAS-backed GEMM for the heavy matrix ops.

what's implemented so far:--

- A custom tensor engine with its own autodiff engine and memory allocator

- Dense/Linear layers, Optimizers (Adam, AdamW, SGD and SGD with momentum), Self Attention, LayerNorm, Activation Functions (sigmoid, softmax, tanh, GELU[tanh approximation], RELU, leakyRELU), loss functions (Cross Entropy Loss [log softmax fused], Binray Cross Entropy (Sigmoid fused), and Mean Squared Error) and Embeddings.

- A from-scratch BPE tokenizer (GPT-2-style pre-tokenization + merges)

- A reflection-based (reflect-cpp) parameter system - models declare their structure, Forge auto-discovers trainable parameters, no manual registration

- a safetensors-format save/load pipeline

the part I'm actually proud of- I loaded real pretrained GPT-2 small weights into a GPT-2 architecture built entirely on Forge, and under greedy decoding, its output matches HuggingFace's transformers library token-for-token. not similar, but exact. every layer (embeddings, attention, LayerNorm, the final projection) has to be numerically correct for that to hold, since a single wrong transpose or masking bug would have diverged the output within a few tokens.

it's still CPU-only for now (currently limited to float32 and int32 - working through some dtype/SIMD coverage gaps), and slower than i'd like (the only main culprits are the CE loss fn implementation and its gardient function and softmax, which i am on to optimize, it has no KV-cache yet) - a CUDA backend and those perf fixes are next on the list.

Repo: https://github.com/muchlakshay/Forge

Windows/Linux release builds: https://github.com/muchlakshay/Forge/releases/tag/0.1

YT demo link - https://www.youtube.com/watch?v=EO1aYBF5jwU

would love feedback, especially from anyone who's built something similar and much better than me.

thats all. im a 17yo deeply passionate about Deep Learning and system level programming.

I've been working on an open-source project called UnFlow:

https://github.com/UnFlow-Labs/mlunflow

The idea is pretty simple:

Most ML experiment tracking looks like a list of independent runs usually stored in a table:

run_001
run_002
run_003
run_004
...

But in practice, experiments are usually related.

You change the learning rate, then the number of epochs, then the model, then some preprocessing code. Eventually you have hundreds of runs, but it's surprisingly difficult to answer:

• What actually changed between these two experiments?
• Which experiments are essentially the same computation?
• Have I already run this experiment before?
• How did I get from experiment A to experiment B?
• Can I navigate the history of my experiments rather than just search through runs?

Unflow simply detect code changes in a Python function (limitation that for it is just a single function) and arguments that are passed to this function to build a graph where nodes are "states" and edges are transformations "what has changed", a new state is not added to the graph or executed expect if it has a transformation.

The project is still early, so I'm much more interested in feedback than pretending this is a finished product.

I'm particularly curious about three things:

1. Does the "experiments as a graph" abstraction make sense to you?
2. Do you currently run into problems with duplicated/redundant experiments?
3. If you could see the complete lineage of your ML experiments, what would you want to query or visualize?

Repo: https://github.com/UnFlow-Labs/mlunflow

I'd love to hear how other people currently manage experiment lineage and whether this solves a real problem for you.

Demystifying AI Attention Mechanics in under 90 seconds! ⚡

Queries, Keys, and Values don't have to be abstract.