built PortfolioLab, multiple ai models run a strategy through stages, backtest validation, then paper trading, then read only api output for your own agent or broker to act on. no skipping stages, if it doesn't survive out of sample testing it never sees paper trading let alone real capital.

built it this way because most "ai agent trades for you" stuff either has zero guardrails or yolos straight to live execution off a backtest that was probably curve fit.

curious how others building agents for finance or other high stakes stuff handle trust, do you gate on stages like this or is there a better pattern. also is the read only split, agent plans, something else executes, the right call or just friction.

not selling anything, mostly want to hear how others are architecting this. will answer anything in the comments.

been building PortfolioLab for a while, too close to it at this point, need people who aren't me to poke holes in it.

premise is simple, don't trust a backtest just because the curve looks good. every strategy has to survive out of sample and walk forward testing before it even gets to paper trade real tickers and position sizing, no capital moving yet. only after that does it go read only api, execution stays with you and your broker, platform never touches your money.

curious if that bar actually matters to people or just feels like more hoops. also what would make you trust a backtest at all, since half of algotrading is people finding out later theirs was curve fit.

probably hits hardest for anyone who's had a strategy look amazing on paper then die live. if that's not you still want your take, just say where you're coming from.

not selling anything, want the criticism more than the compliments. link's in the comments.

Every "our agents continually learn" pitch I've read describes the same mechanic. Refit params to recent data on a schedule.

The selective data issue is rampant. Signal to noise is tiny so you eat mostly noise, there's one history so nothing can be checked against a counterfactual, and it's adversarial so anything you find is getting arbed while you find it.

The evidence isn't new either. M4 comp, 100k series, most pure ML entries lost to naive statistical baselines. DeMiguel/Garlappi/Uppal ran 14 optimized portfolio methods against plain 1/N equal weight and 1/N won on estimation error. Goyal and Welch found the classic equity premium predictors fell apart out of sample. Zillow Offers is the corporate version, ML pricing engine kept quoting confident numbers while the market shifted, 500M+ in writedowns. Model never crashed. The prices just stopped meaning anything.

But freezing everything fails too, which is the annoying part. Concept drift is silent. Under distribution shift accuracy degrades and expressed confidence doesn't, so you get precise assured garbage with no warning. Google Flu Trends read too high in 100 of its last 108 weeks and nobody killed it for two years because the output still looked like output. McLean and Pontiff measured 97 published predictors, 58% lower returns post publication, fastest decay in the ones that looked best.

The way out I keep landing on is that adaptivity is a budget, not a dial. Every adaptive component is another param under refit. So which layer gets it?

Beliefs are "what leads to what." Perception is "how unusual is this reading right now." Continuous learning products spend the whole budget on beliefs, which is exactly where every failure above lives. Spend it on perception and leave beliefs frozen. A threshold that was extreme in 2015 might be nothing now.

And risk limits get zero budget ever. A stop that reinterprets itself during the crash isn't a stop, because in a long crisis the context is the crisis.

Anyone actually doing the perception-only version in their own stack? Curious how you handle the anchoring in practice.

Posting references in comments.

Not a rant, just something I've been chewing on.

The whole 65 to 67 thing started back in 1983 and it finally finished. If you were born 1960 or later your number is 67 now. Took 42 years, two months at a time, so there was never really a day where anyone got mad about it. It just moved.

My mom still says 65 like it's a fact. It isn't and hasn't been for a while.

And I don't think 67 is where it stops. There's a CBO option sitting there that takes it to 70 for anyone born 1981 or later. Denmark already did it. 70 by 2040, and they tie it to life expectancy so it just keeps going up on its own.

The "we live longer so work longer" argument always loses me at the averages part. My uncle did concrete for 30 years and his knees were done at 55. Guy I know who sits at a desk could probably go til 75 fine. Same number for both of them, apparently.

Also nobody I know actually got to pick their retirement date. It was a layoff at 57, or a parent who needed care, or a back that gave out. You retire when something decides for you and then you find out if the money works.

Which I guess is the whole point. The age on paper keeps moving and I don't control when I stop working, so the only thing I actually have a say in is what's saved before that.

What number are you guys planning around? 67, 70, or just "whenever it happens"?

first time actually trying to do this properly. simple mean reversion setup on a handful of liquid us equities, daily bars, data pulled with yfinance. i've got about 12 years of it and i'm holding the last 2 back, but no real reason for 2 other than it felt like enough.

not sure if 2 years is just too short to tell me anything, or if the bigger problem is that 2020 ends up sitting in the training half. what do you normally go with

there's an average of 1.7k visitors weekly, with 1.1k posts in the same time frame. This sub needs to do something to make sure they're not just making a dead internet page that's shouting at a brick wall. Can anyone here actually say they're looking to invest time, money, or resources into someone elses content?

ok so this bugs me about basically every writeup here. we get the sharpe, the max DD, the cost assumptions. we never get the graveyard.

went back through my notes and actually counted. 61 configs, ~5 months. kept 2. and like... if I had zero edge and just rolled 61 times, best of 61 still looks fine? so I genuinely can't tell if my two are real or if I just p-hacked myself over a long weekend.

started logging the rejects after that. every dead variant, date on it. then I treat the survivor's sharpe as best-of-61 instead of a real number. did that and one of mine went 1.8 -> basically nothing lol. other one survived but not by a comfortable margin. entire cost was a google sheet and it's the most useful process change I've made in months.

where I'm stuck: what counts as a try. 40 param combos inside one strat, is that 40 or 1? what about ideas I talked myself out of before writing any code, do those count? no clean answer that I can find and I might be overthinking this at retail size.

anyone live long enough to have an actual rule of thumb here

yeah i know what's happening, it's hallucinating, not asking what the phenomenon is called. what gets me is it keeps doing it on stuff where it could just flag that it's stuck instead of guessing.

been having it go through some trading log stuff and summarize what changed week to week, and when it hits a number it can't actually compute clean it doesn't say that, it just puts something plausible in and keeps going like it finished the task normally. caught it today because the number was off from what i expected, otherwise i probably wouldn't have.

using claude for most of this. added stuff to the prompt telling it to flag when it's not confident instead of filling in a guess, cut down on it some, not all the way.

mostly just annoyed i have to spot check everything now instead of trusting the summary. anyone actually gotten this down or is double checking just the tax you pay for using these things

I've been seeing a lot of posts come up about how their project does a super subjective task and eventually goes awry. A lot of you are advertising, or getting sentiment research, and some of you are real hobbyists, and all of that is okay. If you (and your customers) are going to show any sort of interest in an agentic project, it certainly doesn't hurt to avoid turning it into a black box. There are two main ways to handle large projects, and they're very closely related; Hierarchical goals, and parallel pursuits.

Hierarchy is a widely used method of ordering projects due to the digestible layout of tasks within a project. One agent (or model) makes the call at the top, and that decision splits into smaller tasks that flow down the chain until it hits an output. Clean to read, easy to explain to anyone who isn't building it, but the tradeoff is real: if that top level call is wrong, every task under it inherits the mistake, and you usually don't notice until you're three or four layers deep trying to figure out where things went sideways.

Parallel pursuits work differently. Instead of one chain making every call, you run the same question through a few agents (or the same agent with different framing) at once and see where they land. Two agents landing on the same answer independently means a lot more than one agent being confident about it, and when they split you actually know where to go dig instead of guessing which layer of the hierarchy broke. It costs more compute to run things twice, but for anything customer facing that pays for itself the first time it catches something before a user does.

Curious if anyone here is running both at once, hierarchy for the routing and a parallel check on the parts that actually need validation.

Hi everyone, I wanted to go over the less fun part about building agents, since it's especially important in my work, and some of you might dive into similar fields. I will include references in the comments.

Probabilistic Forecasting: AI models at responsible platforms generate probabilistic, forward-looking statistical forecasts conditioned on historical and disclosed data.  This boils down to a spectrum of answers where the LLM will zero in on the answer with a value tailored to the intention of your prompt INCLUDING prior language in the conversation. The biggest danger here is very similar to priming, aka the anchoring effect in psychology. If you ask for a set of data, it is inaccurate and not verified, and then build on top of that information, you will have a permanently biased system that becomes much harder to fix down the line due to weighted data.

Overfitting Prevention: Overfitting occurs when a model is calibrated too closely to historical data, capturing statistical noise rather than genuine patterns. These results look very good in back-testing, but fail to pull real information out of data, causing performance to drop very quickly, if not instantly. Out-of-sample data testing is a very straight forward approach to checking if you are overfitting results by just simply applying tests using data that's just not in the initial training and seeing where your model fails. This one is quite undervalued when it comes to qualitative data like copy, deep analysis, or semantics heavy work, and often the crux of agentic trading systems.

Model Explainability: While LLMs start out as a complete black box for all of us, as we study them, they become more and more transparent. The ability to discern where in the pipeline from call, thru the chain of actions, to outcome that an answer is disfigured, or out of context becomes a debugging tool, and an asset to model reliability. When compliance teams and portfolio managers can trace the inputs, assumptions, and logic that produced a given forecast, they can identify whether model outputs reflect genuine statistical relationships or artifacts of the training data.

Hi guys, I was writing a article about Renaissance's refusal to change strategies during the covid crash, and ended up shifting gears into talking about how when we trade, we should be having a better internal description of how our portfolio will perform more than "dollar go up."

I started a case study since this was a good example of how the fund had plenty of data, and a deep understanding of where their position stood in the middle of a complete outlier event, and a repeating trend of early capitulation meaning realized losses. Whether it be hiring hot shot investors only to see their returns dwindle, while hiring underperformers on average, showed some massive growth swings all showing signs of fighting through drawdowns.

This is something i did for way too long and see constantly on here. you build a strategy, run it on a few years of data, curve looks rough so you tweak one input. rerun, slightly better, tweak again. do that 30-40 times in an afternoon on the same exact window and eventually you land on something with a sharpe around 2 and a nice smooth curve.

technically nothing shady happened, but it's still overfit, just manually. you basically ran a grid search with one test set and no holdout, then kept the best of like 40 tries. that's not an edge, that's just picking the winning lottery ticket after the draw already happened.

tried it on purpose with a dumb ma crossover + vol filter. swept ~50 param combos on the same 5yr window, best one came out sharpe 2.1, 61% win rate, drawdown looked fine. took those exact params and ran them walk forward on the next 18 months the sweep never saw. sharpe fell to 0.1, win rate to 48%, curve basically turned into a coinflip after fees.

params were fine. the process wasn't. if you can see a score and immediately change an input based on that score, you will optimize toward noise in that window every time, doesn't matter how careful you think you're being.

annoying but actual fix: split the data before you touch anything, tune only on the training chunk, and only look at the holdout once, not in a loop. if it falls apart on the untouched part that's not bad luck, that's just the real number showing up.

anyone here actually walling off test data like this or is everyone just iterating on the whole set and hoping paper trading catches it before real money does

tldr: swept 50 params on 5yrs of data, best sharpe was 2.1. same params walk forward on the next 18mo dropped to 0.1. retargeting against the same window is still overfitting even with zero lookahead.

Glassdoor surveyed 1,700+ workers and 57% said that after a layoff they felt pressured to accept lower pay for their next role. Women and older workers were even more likely to say they had to settle on compensation.

Glassdoor’s economist says it takes almost three years for workplace anxiety from a layoff to fade, and older research shows displaced workers can take a 20–25% permanent earnings hit. If you’ve been laid off recently, did you end up taking a worse offer just to get back in?

There’s a hedge fund called Medallion (run by Renaissance Technologies) that’s basically the final boss of “beating the market.” Over a few decades it reportedly averaged around 60–66% a year before fees and still roughly high‑30s after fees, while the S&P sat closer to 10%. One breakdown says $100 in Medallion in 1988 could’ve turned into hundreds of millions by the late 2010s, while $100 in the index over the same stretch barely makes it into the low thousands, which is why people are skeptical on first hearing about them. They’re not doing stock‑picking TED talks either; it’s quants and PhDs using algorithms and huge datasets to find tiny repeatable edges, then spamming short‑term trades with a small advantage on each, which somehow added up to only one losing year on record and positive returns even in 2008 when the S&P was getting wrecked.

In Medallion’s case, keeping the fund small is part of the edge. The strategies rely on lots of short‑term trades in relatively limited opportunities, and those edges don’t scale well if you try to push tens or hundreds of billions through them. That’s why Medallion only runs roughly $10–15 billion of mostly employee money and pays out profits instead of letting assets balloon; if they took on too much outside cash, they’d start moving prices, crowding their own trades, and the secret sauce would just slowly disappear.

Thanks for the read, guys. This was really fun to look into.

If you’ve been around markets for a while, you eventually realize most retail threads are mixing up three different things and calling all of them “risk”. Volatility, goal shortfall, and concentration are related, but they’re not the same animal.

Volatility is just the path the price takes. It’s annoying, it can trigger bad behavior, but by itself it’s neither good nor bad. In a long-horizon, disciplined framework you need some volatility or you’re probably under-earning your goals. The thing that actually matters is the probability of a permanent or long-duration shortfall against a defined objective; that’s real risk. Most people don’t bother to specify the objective, which is why they end up obsessing over daily moves instead.

Second layer is risk tolerance, which is not a quiz score, it’s the intersection of capacity and temperament. Your balance sheet, income stability, and time horizon define how much risk you can take; your behavior in a 30–50 percent drawdown defines how much you will take. The minimum of those two is your true tolerance. Anything above that is just fantasy until the next bear market snaps you back to reality.

Concentration is where experienced investors quietly blow themselves up. A concentrated book in a space you genuinely understand, with aligned horizon and a clear thesis about cash flows and survivability, can be perfectly rational. A concentrated book built out of “things that did well recently” is just disguised fragility. Concentration risk is essentially “too much of your future depends on a small set of correlated outcomes” one sector, one factor, one country, one style, or even one index that’s become extremely top-heavy.

You manage that not by worshipping diversification for its own sake, but by asking a simple question: “What specific economic story has to stay true for my portfolio to work, and how many different ways can that story fail?” If the answer is “my returns depend on a handful of similar mega-cap growth names all continuing to be priced for perfection,” you’re not diversified, you’re concentrated in a narrative. There’s nothing wrong with making that bet consciously, but calling it “low risk because it’s an index” is how people sleepwalk into concentration risk.

So when I talk about risk at this point, I’m rarely talking about standard deviation. I’m talking about goal risk (do you hit the target), behavior risk (do you stay in the game when it matters), and concentration risk (how many things have to go right, in how few places, for your future to look the way you think it will). Volatility is just the weather on top of that.

You’ll see people say “I’m long” all the time, but they often mean very different things. Two big flavors you’ll run into are tactical longs and momentum longs. If you’re new, understanding the difference saves a lot of confusion (and some pain).

Being long just means you’ve bought something because you expect it to go up in price. That’s it. The reason you’re long, and how you plan to manage that position is where tactical vs momentum comes in.

What is a tactical long?

A tactical long is about where and when you want to have exposure, based on the overall environment.

You’re asking: “Is this a good time to be in this asset at all?”

• You use simple rules about market conditions Examples: price above/below a moving average, volatility filters, basic macro/regime signals.
• You’re mainly trying to avoid bad environments The goal is to sidestep big drawdowns or chop, not necessarily nail every trend.
• You’re willing to move to cash or a safer asset When your “risk-off” signal hits, you exit or rotate, even if the trend might continue.

The key idea: the regime matters more than the exact momentum ranking within that regime.

What is a momentum long?

A momentum long is about what you want to own based on relative strength or recent performance.

You’re asking: “Which assets are showing the strongest price strength right now?”

• You rank a list of stocks/ETFs by past performance Example: top performers over the last 3, 6, or 12 months.
• You buy the strongest names (and sometimes short the weakest) You assume strength tends to persist for a while.
• You often stay invested somewhere You rotate between winners rather than fully dumping into cash.

key idea: you care more about relative strength than broader “risk-on / risk-off” calls.

Tactical means "Am I in, or out right now?" while momentum means "who has the strongest positions to continue riding?"

For beginners, the useful part here isn’t learning two new buzzwords, it’s getting clear on what kind of long you actually are. If your brain thinks in terms of regimes, you’re closer to a tactical trader asking “should I be in or out here at all?” If you think in terms of strength, you’re closer to a momentum trader asking “what’s strong enough that I want to ride it?” Once you know which bucket you’re in, it’s way easier to write down rules, test them, and talk about your strategy without hand‑wavy “I’m just long because the chart looks good” logic.

You’ll see people say “I’m long” all the time, but they often mean very different things. Two big flavors you’ll run into are tactical longs and momentum longs. If you’re new, understanding the difference saves a lot of confusion (and some pain).

Being long just means you’ve bought something because you expect it to go up in price. That’s it. The reason you’re long, and how you plan to manage that position is where tactical vs momentum comes in.

What is a tactical long?

A tactical long is about where and when you want to have exposure, based on the overall environment.

You’re asking: “Is this a good time to be in this asset at all?”

• You use simple rules about market conditions Examples: price above/below a moving average, volatility filters, basic macro/regime signals.
• You’re mainly trying to avoid bad environments The goal is to sidestep big drawdowns or chop, not necessarily nail every trend.
• You’re willing to move to cash or a safer asset When your “risk-off” signal hits, you exit or rotate, even if the trend might continue.

The key idea: the regime matters more than the exact momentum ranking within that regime.

What is a momentum long?

A momentum long is about what you want to own based on relative strength or recent performance.

You’re asking: “Which assets are showing the strongest price strength right now?”

• You rank a list of stocks/ETFs by past performance Example: top performers over the last 3, 6, or 12 months.
• You buy the strongest names (and sometimes short the weakest) You assume strength tends to persist for a while.
• You often stay invested somewhere You rotate between winners rather than fully dumping into cash.

key idea: you care more about relative strength than broader “risk-on / risk-off” calls.

Tactical means "Am I in, or out right now?" while momentum means "who has the strongest positions to continue riding?"

For beginners, the useful part here isn’t learning two new buzzwords, it’s getting clear on what kind of long you actually are. If your brain thinks in terms of regimes, you’re closer to a tactical trader asking “should I be in or out here at all?” If you think in terms of strength, you’re closer to a momentum trader asking “what’s strong enough that I want to ride it?” Once you know which bucket you’re in, it’s way easier to write down rules, test them, and talk about your strategy without hand‑wavy “I’m just long because the chart looks good” logic.

I’m curious what matters most to you before trusting a sandbox for strategy testing.

I approach it in two parts:

• Simulated results: back tests where slippage and other assumptions matter, and the output is theoretical.
• Paper portfolio: live deployment with virtual capital, where strategies execute like a real capital flow, but without real money.

What would make either of those feel credible to you? I’m especially interested in data transparency, execution assumptions, and trade-level auditability.

I’m curious what matters most to you before trusting a sandbox for strategy testing.

I approach it in two parts:

• Simulated results: back tests where slippage and other assumptions matter, and the output is theoretical.
• Paper portfolio: live deployment with virtual capital, where strategies execute like a real capital flow, but without real money.

What would make either of those feel credible to you? I’m especially interested in data transparency, execution assumptions, and trade-level auditability.