u/Ok-Difficulty-7607

I run a small AI news aggregator (helucino). When Jev launched on Sept 16, it never reached the top of my feed, even though half of local llama subreddit was talking about it.

Why: my ranking counts how many outlets cover the same story. Jev got one launch article and then a dozen separate community threads: open copies, reproduction trackers, "what even is Jev". Each thread counted as its own small story, so none of them ranked.

Fix 1: I added a trending strip that counts how often a name appears in headlines, compared with its usual rate over the past week.

Fix 2: two days later Jev dropped off that strip. Its own launch weekend had become its "usual rate". Names with no history before that week are now judged on volume instead.

Lesson: "many outlets covered it" and "many people are talking about it" are different signals. How do you catch stories that start in the community before the press picks them up?

I run an AI news reader (https://helucino.com — mine, free, no account needed to read) that pulls 64 feeds every 15 minutes. Reading them was easy. The thing I wanted was harder: rank stories by how many separate outlets are covering them. Five news websites independently publishing the same story is definitely a signal that the news is important — and to get it you must first know that four articles are one story. So dedupe was the problem I picked.

Four layers, cheapest first:

1. Canonical URL hashing — kills syndication and tracking-param variants.
2. Exact normalized-title match — free, catches straight reprints.
3. Title-token similarity — Jaccard + containment, but gated on shared named entities. Without that gate "Meta open-sources X" merges with "OpenAI open-sources X", which is definitely a wrong answer.
4. Embeddings to shortlist, then a language model to decide — asked in both directions and these must agree, because one-directional prompting says yes far too easily.

The output is a +N badge per row, and a sort order. Ranked orders purely by that count; Top applies the same count with an 18-hour half-life so recency still matters. A false merge is much worse than a missed one here — merging two stories hides one of them and inflates the other's count — which is why layer 4 has to agree with itself before it will collapse anything, and why complete linkage is checked both ways before two groups join.

Current reality check: 63 of 64 feeds green. The one that isn't is Hacker News, which is a bit on the nose. Genuinely curious how others handle publishers that block feed readers — mirror, back off, or drop them?

Solo dev here 🙏 Trying to clear the 12-tester / 14-day closed test.

Geography Worlds is a set of 34 geography quiz games — flags, world capitals, country shapes, satellite imagery, city skylines and more. 100% free, no ads, no sign-up needed (play as a guest).

Test for test — drop your links in a comment and I'll join your group + opt into your app in return.

1. Join the group:  https://groups.google.com/g/anshul-test
2. Opt in:          https://play.google.com/apps/testing/com.geographyworlds.app
3. Install:         https://play.google.com/store/apps/details?id=com.geographyworlds.app

⚠️  Use the same Google account for all 3 steps + on your phone. Please keep it installed for the 14 days. Any feedback very welcome — thanks! 🙏