been watching the credit spreads stay tight longer than most ppl expectedd nd started thinking about high yield exposure . the carry is there but so is the risk and i didnt wantt to go through any trad broker ...
what will you suggest.. which product should i consider
Ran into a small release that uses a trick I haven't seen discussed much in this sub and want to know if this is standard practice or unusual.
Setup: two models released together. TwIL-LM2 (1.7B, PEFT LoRA on SmolLM2) and TwIL-LM3 (3B, merged fine-tune of SmolLM3), both from webAI. Both specialized for formal logic translation.
The interesting part is what they do after supervised training. Instead of shipping the fine-tuned weights directly, they use WiSE-FT interpolation back toward the base model. W = (1-λ)·W_base + λ·W_finetuned.
Here's the thing that got my attention:
- TwIL-LM2 uses λ=0.75 (keeps 3/4 of the fine-tune delta) → strong in-domain gains but slight regression on general benchmarks
- TwIL-LM3 uses λ=0.25 (keeps only 1/4 of the delta) → smaller in-domain gain but stayed at or above base on general benchmarks
Same architecture family, same training pipeline, same task. Just a different interpolation dial. And the "keep less of the fine-tune" version is the one that avoided catastrophic forgetting while still meaningfully improving on the target task.
Their theory (as best I can tell from the model card): smaller interpolation ratio = less forgetting, but you still capture most of the domain-specific improvement because the fine-tune direction is preserved even at low λ. They chose λ by constrained optimization: maximize in-domain score subject to minimal degradation on held-out benchmarks.
Is this common practice? WiSE-FT itself isn't new but I don't see the "dial down the interpolation to preserve general capability" approach discussed much for narrow specialists. Feels like it should be more of a default.
Also worth noting: they document a failed post-RL self-distillation attempt (SDFT) that made both tracks worse. Rare to see negative results published in a model card.
like when you havee multiple teams all using multiple models from different providers for a number of use cases . who decides what is allowed. how do you acutally enforce it. and how do you know whatever rules you put in place
seen a few approaches come up. some team are doing it through the gateway layer . some through internal policy docs that nobody reads . some through access controls on api keys. some just hoping for the best
tools that come up in this context. orqai , portkey , azure api management , aws bedrock , langsmith
portkey gives access controls and budget limits are there , policy enforcement feels more at the routing level than the org governance level
orqai has a model allow list, budget controls and role based access across teams, but its newer so audit depth is still an open question if compared to the more established peers
langsmith has vvisibility into what is happening is good , actually policy enforcementt feels limited , more observe than control
azure api management has enterprise policy controls are native here, feels generic tho ,not built specifically for llm usage policy
aws bedrock has model acess control and guardrails exists, works well if you are already in aws, feels restrictive if you are not
has anyone actually built something that works here. or is everyone just doing api key management and calling it a policy
I watched Nightcrawler recently and really liked how the protagonist slowly becomes more and more messed up as the movie goes on. Looking for more movies like that where you actually watch the protagonist gradually turn into the villain rather than it happening all at once.