u/No-Guarantee4200

I have scrapped ~250k job tech listings from company career pages & (Greenhouse, Lever, Ashby and so on) over the last 2 months and I think the hype is real! :(

Here's the data I get:

Distribution by seniority:

1. Senior: ~78k listings
2. Mid: ~62k
3. Junior: ~20k
4. Intern: ~5k
5. Manager: ~4k
6. C-level: <1k

So basically that means seion+mid roles combined is over 60% while junior+inter=~10% -- that's bad news for juniors.

Salary disclosure:

1. 45% disclose a real range
2. 55% do not disclose salary range

Happy to gather more insights & information -- lmk if you have any questions.

For 2 months I've been crawling ~250k tech jobs - ~$90 spent on AI for enrichment and writing this post because the resulsts suprised me. Now once everything is setup I'm planning to stress-test whole workflow (find best match jobs -> tailor resume with resume builder on claude -> tracking jobs).

How it works under the hood:

Crawling part

1. crawls company career pages and ATS endpoints directly (Greenhouse, lever, Ashby etc)
2. filters out low signal listings
3. run each job listing through an LLM to clean up description + structure it

Searching part

1. knn(vector search) + bm25 on openseerch (fast indexing + full text search)
2. over 25 filters to find best fit jobs

Tech Stack:

1. crawlee (cheerio + playwright fallback) for scraping
2. BullMQ + redis for the pipeline
3. OpenSearch for hybrid searching
4. Postgres for primary database
5. claude+deepseek for AI enrichment
6. nuxt4 for the front-end

Things that I didn't expect:

1. enriching 250k jobs costs ~90$ -- I expected at least 5x
2. senior + mid roles on AI are most wanted
3. junior roles are basically extinct

And the experiment I want to test the whole workflow and apply to at least 100tech jobs to validate several things

1. if resume tailor works the best and it passes ATS
2. if job board recommends me best jobs based on my resume data

Wish me luck! If you have any advice, I'd love to hear it!

https://preview.redd.it/bxqn4a87tv1h1.png?width=1006&format=png&auto=webp&s=9853c7ca9fbb24fffe42e57b7321ad921541d7b1

just saw a sysadmin job listing demanding expertise in 53 technologies - XD -- AWS, Google Cloud, tons of Linux distros, Docker, Kubernetes, multiple databases, monitoring, scripting, automation, web servers, security frameworks -- I think they looking for an alien.

Do you have similar experience?

I just crunched the numbers on 155 recent tech job listings and something's way off. Out of all of them, only 9 were junior roles. That’s 6.4%. For interns? Just 4 spots — a microscopic 2.9%. Meanwhile, a whopping 35.7% are mid-level and another 32% senior. We talk about gatekeeping in tech, but I didn’t expect the data to be this brutal.

Here's the breakdown:

Experience Level	Count	% of Listings
Mid	50	35.7%
Senior	45	32.1%
Lead	29	20.7%
Junior	9	6.4%
Intern	4	2.9%
Director	3	2.1%

After I crawled over 234K recent tech job listings I found that only 42% include salary range in job listing.

Category	Count	Percentage (%)
Total jobs	234,447	100
Real Salary Listed	99,901	42.6
No Salary Mention	134,546	57.4

So that means that more than half job listing do not add salary range leaving candidates guessing XD.

Do you think that's fair or not? :D

I scraped over 250k tech job listings from company websites and ATS platforms. If you ever searched for a job you know how messy it is -- some list salary some dont, tech stack is buried somewhere in paragraph five, and half the listings are probably ghost jobs anyway.

I have scraped over 250k tech jobs from various sources or companies webpages +ATS platforms. Most of jobs or fake, do not provide salary range or have unclear requirements.

I'm using LLMs right now (chatGPT & deepSeek) to extract key information from job listings -- but before I'll go all In I would like your feedback here:

1. what information is most important to you?

salary range, tech stack, remote/hybrid/onsite, interview process, visa sponsorship, team size? What do you look for and whats hardest to find?

2. what filters would save you the most time?

Most job boards give you like location and job type. If you could filter by anything -- salary transparency, company size, specific tech, red flags -- what would actually be useful?

3. whats the most broken thing about job search rn?

Ghost jobs? Duplicates everywhere? No salary? "Senior" roles that want 2 yrs experience? Just curious what frustrates you the most.

My goal is:

1. turn messy job postings into clean data
2. provide best search engine for tech jobs +rich filters
3. filter out ghost/fake/broken jobs -- save time

any help or feedback would help me a lot --- wish me luck 🤞

I built a job board that scrape tech jobs from popular ATS or company webpages and send them through chatGPT:

to extract:

• extracting salary (estimate salary if not provided)
• experience level
• tech stack
• red flags
• I do some enrichment from the job description.

p.s for each job extraction + enrichment I spend around ~2500 tokens so for 200k jobs I have spent around ~90$. IMO -- that's great price!

Some highlights from 205k jobs:

• top tech stack: Python, SQL, Kubernetes, React, Docker
• most listings are for senior roles, onsite still leads over remote
• backend jobs dominate, AI roles are second
• built-in red flag detection - flags things like mandatory overtime and missing salary info

Would love feedback if anyone wants to check it out.

https://preview.redd.it/s7drcs02fszg1.png?width=2146&format=png&auto=webp&s=e591aab41d59fde5b29d655dd441104efdb0fd35

I built a job board that runs every listing through an AI pipeline - extracting salary, experience level, tech stack, red flags, and scoring quality 0-100. Hit 201k listings, thought the data was worth sharing.

Salary transparency is still terrible

~60% of listings have zero salary info. Only ~15% disclose real numbers. When they do, average range is $153k – $215k.

• Intern: $70k – $110k
• Junior: $80k – $120k
• Mid: $100k – $140k
• Senior: $130k – $180k
• Lead: $140k – $195k
• Director: $170k – $230k
• VP: $200k – $300k

Biggest jump is mid -> senior, roughly +30%.

The market is brutal for juniors

Senior + mid roles = 58% of all listings. Junior + intern = under 15%. If you're a new grad wondering why it's hard - this is literally why.

Remote isn't dead

On-site ~40%, remote ~30%, hybrid ~12%. Despite all the RTO noise, nearly a third are still fully remote.

Most wanted tech

Python (25k), SQL (15k), Kubernetes (12k), React (10k), Docker (9k), Azure (8k), JavaScript (7k), C++ (7k). Infra skills are becoming baseline, not just a DevOps thing.

13% of listings have red flags

26k out of 201k triggered at least one - mandatory overtime language, hidden salary, third-party recruiters. Average listing quality: 80/100, higher than I expected.

I built a job board that runs every listing through an AI pipeline - extracting salary, experience level, tech stack, red flags, and scoring quality 0-100. Hit 201k listings, thought the data was worth sharing.

Salary transparency is bad

~60% of listings have zero salary info. Only ~15% disclose real numbers. When they do, average range is $153k – $215k.

-----

1. Intern: $70k – $110k
2. Junior: $80k – $120k
3. Mid: $100k – $140k
4. Senior: $130k – $180k
5. Lead: $140k – $195k
6. Director: $170k – $230k
7. VP: $200k – $300k

The market is crazy for juniors

Senior + mid roles = 58% of all listings. Junior + intern = under 15%.

----

Remote isn't dead

On-site ~40%, remote ~30%, hybrid ~12%. Nearly a third are still fully remote.

Most wanted tech

1. Python (25k)
2. SQL (15k)
3. Kubernetes (12k)
4. React (10k)
5. Docker (9k)
6. Azure (8k)
7. JavaScript (7k)
8. C++ (7k)

>!13% of listings have red flags!<

26k out of 201k triggered at least one - mandatory overtime language, hidden salary, third-party recruiters. Average listing quality: 80/100, higher than I expected.