I have 2 years of backtesting results and one month of forward testing and it all looks good. Would you trust it enough to go live at that point? I’m still a little scared that it’s overfit and the second I go live it stops working but that’s probably just my anxiety speaking
A thread here a few days ago asked how real quant edges get discovered. I left a short comment. This is the longer version, at the horizon most people here actually trade: minutes to a few hours.
I trade FX, crypto, and futures. I start with the payer. I do not start with an indicator.
Maybe forced traders. Information hits one market before another. Overnight inventory has to get out when the real session opens. Market makers get stuffed and fade. Participants underreact for an hour and then finish the move.
Then: what should I see if this mechanism is real?
That expectation becomes a feature. Then I define where the feature is supposed to go. That is the hypothesis.
Forced selling creates unusually high liquidation pressure, which can create a short-term reversal.
That is testable. Put it next to “RSI below 30 is bullish” and the gap is obvious. One names a payer, a thing you can measure, and a horizon. The other cannot die, so it cannot earn.
Before I build a strategy I test the cheapest relationship I can: an event study, buckets, a lead-lag regression. I want to see the effect I am claiming exists. If I cannot see it in a sort, a 47-parameter backtest will not find it for me.
The pipeline:
Thinking → thing to see → a feature → the hypothesis → test the hypothesis → trading production → risk control
“Explain it to me like I’m five” is the creative half. “Now prove me wrong” is the scientific half.
Three walks on that spine, all session-speed.
FX: equity lead into the session
ES, DAX, Nikkei print while half the FX book is asleep. The equity move is information. Spot has not finished digesting it. The lag is the trade: one to twelve hours, pair by pair. Same family as “yields moved, EURUSD has not.” The bond market prices policy first; FX is late.
Who pays: the slower FX book. Discretionary and slower systematic flow that still has to catch up.
A cousin on shock days: rank which pairs digest the print fastest. The laggards stay tradable for hours. That is the same mechanism with a news clock instead of a session clock.
Crypto: liquidation cascade, then classify
Perps publish the leverage. You can watch forced flow in real time. A liquidation is a market order into a falling book, which triggers the next one. 19 May 2021: bitcoin −30% intraday, about $8bn force-closed. The question after the spike is not “oversold.” It is: were positions closed, or flipped.
Who pays: the forced. Margin engines do not have a view. They have a threshold.
Ride the forced flow while its intensity stays above a decay threshold, then stop. Fade the forced component, trend the organic one. Two hypotheses, same feed.
Futures: overnight inventory into the RTH open
Globex is a thinner auction. Size that built overnight is inventory, not a thesis. 9:30 ET is when cash, index arb, and the real book show up. Same observable, two claims, and which one pays is the open itself.
Who pays: overnight positioners who cannot hold the inventory into a full session.
A strategy is MARKET × MECHANISM × INSTRUMENT. ES lead into AUDUSD at the Sydney open, a BTC liquidation fade on the perp, an ES overnight unwind at the cash open: three leaves, one tree.
Most ideas die at the sanity check. That is the pipeline working. I do not start writing entries until the payer has a name.
For those looking to understand what means to have an edge on the markets
This is the first algo I have built, I still feel I can get higher profit factor and bring down average loss trade value.
Hi everyone,
I am new to this and am about two to three months into my algo trading project. I have discovered about 200 Strategies which look promising and I have soon arrived at the conclusion that a fleet is better than a solo strategy. Now how do I best combine strategies? Maybe some have already mastered this and can share a few tipps. My strategies are mostly about indices, gold and some FX pairs. Thanks in advance!
ok so last week i posted that claude helped me build a strategy and bracket setup but honestly i didn’t fully understand what was going on under the hood. got a lot of comments calling that out, fair.
so i went back and actually dug into it. spent the week building out a bucket screener instead of one single strategy. basically i’m tagging every trade with a set of factors (setup type, time of day, ticker, entry trigger, whatever) and grouping trades into buckets based on which factors overlapped. then i can see which combos of factors actually cluster around good win rates instead of just eyeballing one equity curve.
been paper trading this for a bit now and some of the buckets are looking really strong. one bucket with 13 trades is running 85% win rate and +12.94% average per trade. another bigger bucket with 20 trades is at 60% win and +4.80% average, which feels more like the “real” number once sample size grows.
still very much learning as i go and treating this as paper only until i have a lot more trades in the bigger buckets. small sample sizes can lie to you easily (that one bucket with 3 trades at 67% win and a 15+ profit factor is obviously not something to trust yet).
I was tired of hunting for long-tail events; and creating bootstrapped event-generators was a hassle. If I'm focusing on a project, why would I want to maintain a sub-project? So I built a once-and-for-all market simulator with events on demand (completely free & open-source, available to everyone)
It's a fake market data API — realistic OHLCV for any ticker, a tick stream that runs 24/7, and reserved tickers (CRASH, GAP, HALT) that action on command. Point your existing client at it to test code paths that only fire on market edge-cases.
This is a testing tool, to test long-tail events like run-ups, halts, SSE loss, and intraday flash events.
No API Keys needed. Simple curl to test:curl 'https://cuckootrade.com/api/v1/alpaca/v2/stocks/bars?symbols=AAPL,CRASH'
alpaca.com/api -> cuckootrade.com/api)Worth saying: it's for testing, not backtesting. Synthetic data will make almost any strategy look profitable.
It's a portfolio project I'm still polishing, so feedback actually helps me. Tell me what's broken, or what provider or scenario you want next and I'll build it. Or you're welcome to contribute to the repository:
Open source Github: https://github.com/tj-miller-dev/stock_simulator
No paywalls. No fees. No login. Just the data.
Ran the numbers after 4 months live. Strategy was up. Green on the equity curve, win rate matched backtest, nothing broke.Then I actually sat down and added up server costs, data feed subscription, and the spread markup I was eating on every fill. Plus the buffer I keep idle just so a margin call doesn't wreck me mid-trade.Take all that out and the profitable strategy was barely breaking even. On paper it looked like a working system. In my account it looked like a very expensive hobby.
Nobody backtests their fixed costs. But at small size, they're not a rounding error, they're basically another drawdown that never shows up on the equity curve, it just shows up on your statement.I'd spent weeks tightening entries when the bigger leak wasn't in the strategy logic at all.
Anyone else actually done the full accounting on this? At what account size did fixed costs stop mattering for you?
IYKYK
Do you guys have any stocks your Bots just love to buy and regardless of how strong the set up is...you just know its going to trailing loss (or hard stop)... Mine funny enough is Reddit. Im tempted to Blacklist it
I'm making a long only algo for both QQQ and equities, derived from the same base code but then honed in to each for more specific characteristics. My QQQ strategy currently has a 1.703 profit factor but its still in tradingview environment (so take it with a grain of salt?). I am absolutely killing it with preventing downturns in my algo and preventing large drawdowns, however, i think it is handicapping my long as for example I am vastly underperforming a strong environment. Is there any type of criteria you guys use to help loosen sell rules by having the algo identify strong trending markets or something along those lines? Or in general how do you help your algo capture the long term trend better without selling? This is my first algo, I am definitely a trader first, coder second so bear with me
Posting this as a warning and a genuine question, because I've just spent two days learning something the hard way and I want to know whether I was chasing something that exists.
Context, deliberately vague: intraday mean-reversion system on liquid futures, 1-minute bars, ~4.7 years of tick data, one position at a time, realistic fill modelling (entries and stops at real tick prices, exits as resting limits rounded to the actual tick grid).
Raw performance, no filter: ~9,900 trades, 77% win rate, profit factor 1.05. High win rate, tiny edge per trade roughly half a tick on average. The mean-reversion tendency is clearly real, it's just not big enough per trade to cover costs.
What I tried: the obvious move was a regime filter trade only when conditions favour reversion, stand aside when they don't. I screened Kaufman Efficiency Ratio, RSI, ADX, Choppiness Index, Money Flow Index and MACD histogram across multiple time frames.
One combination looked outstanding. Profit factor 1.05 → 1.54. It held out-of-sample on three years the thresholds had never touched. It passed split-half consistency. It passed split quarter consistency beat the baseline in all four quarters independently, monotonically. By every robustness check I knew to apply, it was real.
Then I found the bug. Indicators were being merged onto each bar's opening timestamp instead of its closing timestamp. A 3-minute bar labelled 10:06 spans 10:06–10:09 and isn't knowable until 10:09 but entries at 10:06:30 were being gated on it. Up to three minutes of look-ahead, sitting directly in the entry decision.
That's fatal for a regime indicator specifically. These things measure whether price is trending or ranging. Give one a forward window and you've handed it the answer it was supposed to infer from the past.
Lagged everything to true availability time. Profit factor went straight back to 1.05, statistically indistinguishable from no filter. The entire apparent edge was the leak.
I then re-screened all six oscillators properly lagged ~48 buckets, each scored on both halves independently. Six beat baseline in both halves. Pure chance predicts about twelve. So the result is below random, with no coherent or monotonic structure, and the previously-best bucket now sits under 1.0.
What I'm actually asking:
Not looking for anyone's edge. Mostly I want to know if anyone has a regime filter that's been through this specific audit and lived. Any advice would be greatly appreciated! I know I have something solid as a baseline I just need some ideas on filtering.
143 tickers that disappear from free s&p 500 data, and what actually happened to each one, with a source link on every single row
for anyone who missed the earlier posts, first one measured how many old members you can't download anymore, 20-25%, second one measured what that does to returns, 0.4 to 1pp a year, and found the reused tickers were doing more damage than the survivorship itself, a few of you asked for the corpse list, this is it
each row has what happened, who bought it, the last day it traded, and the source, 38 of them are sec filings, rest are company press releases or exchange notices, nothing scraped off wikipedia, i opened every link myself
the split, 87 bought, 18 taken private, 18 merged, 6 renamed, 10 bankruptcy (6 came back out, 4 didn't), 4 split up, same shape as post 1, most of these names didn't die, they got bought, thats why the us bias direction isn't obvious the way india's is
one thing that looks like it contradicts post 2, i said 4 reanimated tickers there, this file flags 14 as reused, they're different things, 14 is every symbol that later got used by some other company at some point, 4 is how many actually broke my numbers with my dates and my data source, which ones bite you depends on your sample, so check the column against your own data instead of taking my 4
what to do with it, take your universe list, compare it against this one, count how many exits your source quietly dropped, thats your survivorship hole measured instead of guessed, and before you join old prices to new prices on ticker alone, look at the reused column, that splice is invisible in the returns, nothing warns you
sortable and searchable here, csv download on the page: https://financebroski.com/deaths.html
free, no signup, no email
caveats, this is the names that go missing from the free sources i've been checking, not every index change ever, it grows as i check more vintages, and the exit date is the last day the ticker traded, so for a merger that closes mid day thats the day it closed, if a row is wrong or i missed a name just tell me and i'll fix it and credit you, i'd rather it be right than look clean
still owe a few people the method from post 1, thats next
I’m a college student building a soccer prediction model and finally tested it on a season I had kept completely untouched.
The forecasting model itself held up reasonably well, but on ~1,700 matches de-vigged Bet365 probabilities beat every model I tested on log loss/Brier/RPS, and all of my pre-frozen betting strategies ended negative. My main one finished around -12.5% ROI.
Right now I already use things like goals/xG, shots/SOT, home/away splits, Elo/opponent strength, form, rest/congestion, standings, league scoring environment, and promoted-team uncertainty. I also have a decent amount of player, lineup and goalkeeper data, but the historical player/lineup experiments didn’t add much and exact pre-match availability timing was a limitation.
I’m planning to stop optimizing on historical results and run the system prospectively with timestamped odds, injuries and confirmed lineups.
What I’m mainly wondering:
What additional data would you look for at this point?
Manager changes? Transfers? Better injury/suspension history? Player market values? Weather? Travel? Referees? Tactical/style data? More detailed event/possession data?
Is there any dataset you’ve found that actually adds signal beyond what the market already knows?
Or would you stop adding features and focus on prospective validation / finding where the model disagrees with the market for a defensible reason?
Trying to avoid just throwing more variables at it until something backtests well.
Hi,
I've got my reddit post and YouTube transcript engines up looking for momentum plays and signals. I follow a bunch of the usual stuff.
If anyone has recommendations of other subreddits or YouTube channels to process, I'm here for it.
Thanks
This is a dedicated space for open conversation on all things algorithmic and systematic trading. Whether you’re a seasoned quant or just getting started, feel free to join in and contribute to the discussion. Here are a few ideas for what to share or ask about:
Please remember to keep the conversation respectful and supportive. Our community is here to help each other grow, and thoughtful, constructive contributions are always welcome.
I’ve been thinking about adding execution portability to the robustness checks I use before trusting an algo, but I’m not convinced it’s actually a fair requirement.
Basic problem is pretty simple
I can keep the signal logic, sizing and exits completely unchanged and still get a very different system once I change the assumptions around execution.
So instead of only stress testing parameters and market regimes, I’m considering testing the execution layer separately.
Something like this
Baseline
| Stress test A | spread 25% worse | |||
|---|---|---|---|---|
| Stress test B | spread 50% worse | |||
| Stress test C | normal spread | 1 tick adverse slippage on entry and exit | ||
| Stress test D | spread 25% worse | some adverse slippage | small fill delay | a percentage of limit orders left unfilled |
I’d then compare more than just final P/L
The metrics I’m thinking about are
Numbers above are just hypothetical stress parameters. I’m more interested in the framework than those exact thresholds.
Here’s where I’m getting stuck.
Say a strategy has a PF around 1.30 under the baseline execution model. A modest increase in spread brings it close to 1.10. Add a little adverse slippage and it drops below breakeven.
There are two completely different ways I can read that.
The first is that the strategy was fragile from the start. If a small change in trading costs wipes out the edge, I probably shouldn’t trust the backtest much. Second is that this is asking the strategy to survive something it was never designed for. A short-horizon system can have a real edge that only exists with a specific fee structure, spread, order type or execution venue. In that case the execution environment isn’t noise around the strategy. It’s part of the strategy.
That second case is what makes broker portability questionable to me as a robustness test. I wouldn’t expect a market-making system to keep the same economics after changing the fee structure. I also wouldn’t expect a tight scalper to behave the same after doubling its spread assumptions. At some point I’m no longer stress testing the same business logic. I’m changing the conditions that created the edge in the first place.
But giving the execution model zero robustness testing seems just as bad.
Backtest that only works at one exact spread and one exact slippage assumption feels way too brittle for live trading.
So I’m leaning toward measuring an execution degradation curve instead of using a simple pass or fail rule.
Basically I want to know how quickly the edge decays as execution gets worse.
A gradual decay seems a lot healthier to me than a strategy that falls off a cliff after one extra tick.
What I don’t know is where that becomes unacceptable.
For people running systems live, how do you handle this?
Do you expect an algo to remain profitable across moderately worse execution assumptions, or are you fine with an edge being tightly tied to one broker, venue or fee structure as long as those conditions are stable?
And if you stress test execution, what do you actually perturb: spread, commissions, latency, fill probability, slippage distributions, or all of them together?
I built a model to estimate current short interest between official FINRA reports using daily short-volume data.
FINRA short interest is only published twice per month and with a lag, so there is a period where the latest official number can already be fairly stale. The model uses daily FINRA short volume and total trading volume to estimate how short interest may have changed since the last settlement date.
I evaluated it on 660,246 settlement windows covering 6,959 US-listed stocks. On a fully out-of-sample period from 2025 through July 2026, the estimated change achieved a +0.414 Spearman correlation with the subsequent reported change in short interest.
This is not meant to predict returns or identify short squeezes. The goal is simply to provide a nowcast of short interest while waiting for the next official release.
I wrote up the methodology, validation setup, formulas, experiments that did not work, limitations, and released the model weights here:
https://equibles.com/research/does-daily-short-volume-predict-short-interest
Equibles also shows the latest official short-interest figure next to the model's current estimate for covered US stocks. It's free and has no ads.
The model is available at HF:
https://huggingface.co/daniel3303/equibles-short-interest-nowcast
Curious whether people here would find this useful when analyzing stocks with unusually high short interest.
Good morning everyone, I recently wanted to backtest a new ea but when I try to download the data via Tickstory, it won’t let me download it. It comes back as: request timed out or rate limited. I thought it was a problem with TS Program and so I coded one in Python but it came out with the same result. I tried using a VPN and it worked for few seconds, afterwards, same problem… I only have 1 worker for the downloads but still it fails me everytime both on Tickstory and Python… Since I’m getting so much frustrated because I spent the last 24h not sleeping in order to download this mf data, is there a FASTER way to download them? I need the most precise data possible (ticks), since I’m currently working on a multi-currency scalper EA on MT4.
Thanks for your help, last time I did backtest was 2 years ago and I used datas from Dukascopy & Tickstory, I really don’t know why it’s not working now.
I've been backtesting and paper trading an NQ scalping strategy for awhile. I went with IBKR as my broker based on the API and the low fees for API, data and trades. However, IBKR doesn't have discount margin rates on futures, and NQ has skyrocketed to well over 40k per contract. This means tying up quite a bit of money in either cash or t-bills (best I can find as an alternative to at least get some return). No matter how successful my strategy is (or isn't!), that's opportunity cost lost in other investments compared to discount margin. But switching to another broker would be a lot of work, plus potentially higher regular costs. Are others still using brokers such as IBKR for futures?