u/Historical_Ad9654

Been working on this for a while and finally feel good enough about it to share.

It's a stock research dashboard that pulls together macro signals (FRED, EIA, CBOE), insider trading filings (SEC EDGAR Form 4 XML), FINRA short interest data, and 13F institutional positioning — and uses those to produce a per-ticker "confluence score" showing how many independent signals are aligned bullish or bearish for a given stock.

Stack: Python, Streamlit, PostgreSQL/SQLAlchemy, Plotly, pandas, yfinance, Render for hosting.

A few things I'm reasonably proud of:

- The lag scan (cross-correlates each macro signal against forward price returns, scans 1–12 week lags, applies Bonferroni correction for multiple comparisons, and validates out-of-sample)

- Congressional Trade Tracker — parses STOCK Act disclosures and flags cluster activity (3+ members buying the same ticker within 45 days)

- Short Squeeze Radar — combines FINRA short interest % with insider cluster detection and macro confluence score

- Signal Reliability Score — a meta-score on top of the lag scan that tracks whether each signal's historical lead time is holding up

Live at: unstructuredalpha.com — free to browse, no account required for most pages.

Would love any feedback on the approach or the UI. Happy to answer questions about the architecture.

The clustering problem with correlated signals

My system scores ~40 macro signals (Fed funds rate, yield curve, M2, insider buying, short interest, etc.) and generates a composite "confluence score" for a given ticker. The naive approach is to just average the signals. Problem: many signals are correlated — yield curve and credit spreads move together, insider buying and short interest are often inversely related. Averaging them inflates apparent confidence.

Fix I landed on: pairwise Pearson correlation matrix using pandas + numpy on 3 years of weekly signal history. Then scipy.cluster.hierarchy.linkage with single-linkage at a 0.6 threshold groups correlated signals into clusters. Each cluster gets one vote, weighted by the cluster member with the best out-of-sample Sharpe ratio on that ticker's 60-day forward returns.

Streamlit caching gotchas

@st.cache_data is great but has a subtle memory issue: it keeps ALL cached versions until max_entries is hit. For a function that fetches 40 signals with 5 time-period variations, you can end up caching 200+ DataFrames. Added max_entries=1 to the main signals cache — memory dropped from ~1.1GB to ~200MB under concurrent load.

Also: calling ThreadPoolExecutor inside a cached function is fine for pure data fetching. But if the cached function spawns threads that themselves call other cached functions, you can hit Streamlit's session state lock. Solution: only parallelize at the outermost uncached layer.

SEC EDGAR Form 4 XML parsing

EDGAR serves Form 4 filings as XML, but namespace handling is inconsistent across filings. Some have explicit xmlns declarations, some don't. I strip namespaces with a regex before parsing:

xml_str = re.sub(r'\s*xmlns[^"]*"[^"]*"', '', raw_xml)
tree = ET.fromstring(xml_str)

For insider cluster detection (flagging when 2+ insiders buy within 21 days), I group by issuer CIK, filter for transactionCode == 'P' (open-market purchase), then use a rolling window on sorted transaction dates.

SQLAlchemy Core schema

Using SQLAlchemy Core (not ORM) for the main tables: users, signal_snapshots, watchlist_items, alerts. One thing I'm glad I did: a single DATABASE_URL env var that switches between Postgres (prod) and SQLite (local dev). Same schema DDL works for both — keeps the local dev loop fast.

Happy to answer questions on any of the above.

Started this as a personal project to stop manually checking 15 different data sources every morning. Ended up building something I actually want to use daily.

**What it does:** Unstructured Alpha (unstructuredalpha.com) scores 30+ macro and alternative data signals daily — FRED economic indicators, SEC EDGAR insider transactions, FINRA short interest, 13F institutional filings, options flow, Fed liquidity, credit spreads — and maps all of it to individual stock tickers via a correlation-weighted Confluence Score.

**What shipped in v2:**

**Dark UI redesign** — Went from a WSJ-style light theme to a full Bloomberg-dark design system. Every page rewritten with a consistent color token set, skeleton loading states, fade-in animations, and styled empty states. Took longer than expected but the app finally feels like a professional tool rather than a prototype.

**Editorial morning digest** — Built a Seeking Alpha-style email cron that generates an actual article every morning: dynamic headline based on the signal bias ("Macro Machine: Bullish lean — 17 of 30 signals green"), KEY DATA POINTS box, WHAT THIS MEANS analysis, WATCH FOR risk block, and a BOTTOM LINE. Previously it was just a table of numbers.

**Real PDF exports** — The export now uses the real correlation-weighted score engine instead of a naive average. Adds a full Positioning section: insider conviction score, short interest score, 13F institutional score, top-4 insider transactions with officer names and trade values, top-4 13F holders with position sizes.

**Tech stack:** Python, Streamlit, SQLAlchemy (Postgres/SQLite), FRED API, SEC EDGAR Form 4 XML parsing, FINRA short interest feeds, yfinance, Resend, Stripe, deployed on Render.

**What I learned building this:**

- Correlation-weighting signals matters a lot. Naive averaging of 30 signals overweights clustered macro indicators (e.g. all 5 Fed signals move together). The final score only counts ~8-10 effective independent signals.

- Insider transaction data is noisier than expected. Have to filter for cluster patterns (2+ insiders buying within 21 days) to get signal above noise.

- Email deliverability is its own project. SPF + DKIM + DMARC setup took longer than the email template itself.

Happy to answer questions on the build or the architecture.