Benchmarks compare open models against closed products, not closed models. We might be missing what were actually paying for

So this has been on my mind for a while and it kinda bugs me. Every time someone benchmarks glm-5.2 or deepseek against claude or gpt, the closed one wins on some tasks and people just assume the underlying model is smarter. but thats not really what were measuring.

We dont know what these closed providers actually do behind the api. they might be running rag over their own docs, injecting hidden system prompts based on your query, routing to specialized expert models depending on task type, doing prompt preprocessing we never see, hitting internal tool calls before the model even generates a response. anthropic already hides reasoning traces and doesnt show you the full pipeline. we get the polished output and we assume its just the model.

Meanwhile when you benchmark an open model youre benchmarking raw inference. no scaffolding, no hidden tools, no preprocessing. its like comparing a cars engine on a dyno to another car actually driving on a road with traction control and abs and lane assist. the road one looks better but its not because the engine is stronger.

Which makes me wonder if the actual model quality gap between the frontier closed stuff and something like glm-5.2 is way smaller than benchmarks suggest. What you are paying premium for might be the tooling and the harness wrapped around it, not the raw model. and if thats true this whole industry is heading somewhere weird, because tooling is way easier to replicate than model architecture, and open weights plus open source tooling starts to look really competitive really fast.

There is a broader thing going on too. software engineering hasnt actually changed in principle, its still specs, architecture, tradeoffs, maintainability. what changed is the volume. line by line code review doesnt scale when agents produce diffs at this rate, so review has to move upstream to specs and downstream to tests, metrics, traces, observability. thats where the actual verification happens now, not in the middle where volume already broke it.

So heres what i am stuck on. when we say model X is better than model Y based on benchmarks, are we actually comparing model to model, or are we comparing raw inference against everything the closed provider bolted onto it that we cant see, and does that distinction even matter to anyone anymore.

reddit.com
u/Stir_123 — 15 hours ago

Guys anyone here running multi RoK acc's?

Ive got extra 3 Rise ok Kingdoms account on 3 different devices. But sometimes too lazy to open it, switching between devices all the time is kind of a hassle.. Do you guys a better setup for this kind of situation?? wondering what are you guys else using. Any good EMU or PC setup for running multi accounts at the same time??

reddit.com
u/Stir_123 — 19 days ago

Half the time someone asks "what streaming camera should i buy" the camera isnt the problem

A buddy of mine who does av work for a living pointed this out and i cant unsee it now.

Whenever someone asks what camera they should get for streaming, the instinct is to immediately start rattling off specs. 1080 or 4k, frame rate, autofocus, blah blah. I used to do it too. but he always stops and asks them what theyre actually trying to fix first, and nine times out of ten the camera isnt even the issue.

Most people starting out dont have a camera problem. They have a dark room that makes any cam look bad. or they're overwhelmed trying to get obs to cooperate. or they just hate how washed out they look and assume a new camera fixes that when really its the lighting or their mic dragging the whole thing down.

And when it genuinely is the camera, throwing money at it isnt the move either. the right question is more like, do you want something you just plug in and forget, or are you cool tweaking settings, and whats your lighting actually like right now.

Had this happen with someone i was helping recently. They were about to drop like 300 on a camera somebody told them was what "real streamers use." thing is it needed manual everything and their whole reason for asking was they didnt wanna learn any of that and their room had one sad lamp in it. talked them out of it and they got something cheap that just does auto exposure and handles low light on its own. way happier, way less money.

Point being, when someone asks you for gear advice, push them to actually describe the problem before you name a product. saves them from buying the wrong thing and saves you from being the reason they did.

What's something you bought early on that you wouldnt have if someone had just asked you the right question first?

Edit: A few dms asking what the cheap camera was, it was an emeet pixy. nothing fancy but it does auto exposure and handles low light on its own which was the whole point for them

reddit.com
u/Stir_123 — 21 days ago

what actually works for transcribing messy audio recordings?

been working with a lot of interviews and meeting recordings lately and still haven’t found a transcription tool that feels consistently reliable.

clean audio is usually fine, but once there’s background noise, accents, or people talking over each other, the results start falling apart. then i still end up spending time cleaning the transcript anyway.

curious what people are actually using for real-world recordings.

reddit.com
u/Stir_123 — 1 month ago

I keep seeing red light therapy for skin everywhere but I can’t tell if it actually does anything

Lately I’ve been seeing red light therapy mentioned a lot for skincare, especially for things like acne, redness, and overall skin texture.

Some people swear it helped their skin look clearer or more even over time, but I also see a lot of comments saying they didn’t notice anything even after using it consistently for weeks.

I’m trying to figure out if it’s actually something noticeable in real life or if the results are just too subtle unless you’re really tracking changes closely.

For people who’ve tried it for skincare:
Did you actually see any visible improvements, or was it more of a “maybe it’s working” kind of thing?

How long did it take before you felt confident saying it made a difference (if at all)?

Would you say it’s worth sticking with long term or more of a hit-or-miss thing?

reddit.com
u/Stir_123 — 2 months ago

Vendor went out of stock mid-protocol on BPC - how do you guys handle continuity?

Anyone else get weirdly attached to one vendor and then get burned when they go out of stock right when you need to reorder?

I've been running BPC/TB500 for a tendon thing in my elbow for about 5 weeks. Finally starting to feel real progress, sleeping better too as a bonus. Go to reorder from my usual spot last weekend and BPC is just... gone. No ETA. No "back in two weeks." Just out.

I sat there for a sec actually annoyed at myself for not having a backup plan. Like I know better. This isn't the first time a vendor has ghosted mid protocol. Spent maybe an hour bouncing around peptiprices trying to find someone reputable who actually had it in stock at a reasonable number and not jacked up 40%. Ended up going with a vendor I hadn't used before, reviews looked solid, fingers crossed the quality is comparable.

But it got me thinking, how do you guys handle continuity? Do you stockpile a month or two ahead? Run two vendors in parallel so you always have backup? I feel like every time I get comfortable with one source something happens (out of stock, shipping delays, vendor disappears entirely, lab results come back funky).

Also slightly worried about switching mid cycle for the same peptide. Probably overthinking it since BPC is BPC if it's real, but still. Anyone notice a difference when swapping suppliers partway through a protocol?

reddit.com
u/Stir_123 — 2 months ago

Finally using Virtual UGC to make my AI tool stack pay for itself

As a digital marketer, my monthly overhead for AI tools was starting to get a bit ridiculous. Between my CRM, ChatGPT, Midjourney, and video subs, I am spending before I get any return. Idk if other people deal with this but yeah, thats where I'm at.

I spent weeks trying to create Virtual UGC using Runway Gen-3 and Kling. I wanted a virtual creator holding a product and talking, hence saving me the actual content creator fees which also adds up.

Runway: The visual fidelity is insane great, the way it handles skin texture and high-end cinematic lighting is totaly on point. When it comes to the physics, though, it was just off, often the character moved their hand, the product would warp, which is a total waste of time.

Kling: It is hard to keep the character consistant across different shots.

By the end of it, I actualy went back to Fiverr to look for real creators because I figured, it is either me, or the tech is not quite there yet. I was ready to accept that Id just have to keep paying the $300 shipping and creator fees to promote some products.

A buddy of mine in a Discord group sent me a clip of a character he created holding up a phone. He told me he used a few minutes to create a clip with PixVerse V6. I was honestly annoyed because I didnt want to learn again and to be dissapointed, again.

I signed up just to prove him wrong. My first render was actualy a failure too, I used a basic prompt and the lighting was flat and I just wasnt feeling it.

The click happend when I stopped letting the AI guess what I wanted and started actually utilizing the Character Consistancy and Regional Control tools. I am able to lock in the face and animate the product. It still took me 4 or 5 tries to get a clip that was useable, and I lost track of how many credits I burned through, but I finaly felt like I was getting somewhere.

For the first time, I had 15 seconds of footage where the girl actualy looked like the same human being from start to finish and it looked passible that it seems like it was shot under a ring light.

It is not perfect, regardless I starting show the how to video of this workflow. I even applied for its affiliate program, and those recurring commisions got to $185 this month.

The income is meager at best. But between the money saved on hiring real creators and the affiliate payouts, my subs are finaly subsidized.

For it to monetize, dont dump a tool because the first render is bad. Document your workflow, find the tool that handles the specific technical bottleneck (for me, it was character stability), and let the affiliate commisions help you while you try to get better at it!

reddit.com
u/Stir_123 — 2 months ago

As a future investor, how do I get an expertise in multiple industries and those emerging in-between?

I’ve been thinking about this lately because a lot of startups don’t really fit into one clean category anymore.

Like, healthtech is both healthcare, AI, data, hardware, privacy, insurance, sometimes even consumer apps, and what have you. Same with fintech. it overlaps with cybersecurity, identity, regulation, and big enterprise software. But then, a pretty usual sports businesses were always tied to media, analytics, betting, wearables, fan platforms etc.

So I’m wondering if the old idea of being a super-focused “sector investor” is becoming less useful in some cases. Like, some of the biggest names in the investment world are mostly those. But a younger generation gives us names like Palantir of Peter Thiel, Andreessen Horowitz of Marc Andreessen, Sun Microsystems of Vinod Khosla, NJF Holdings of Nicole Junkermann and other investors who have backed companies across a bunch of diff areas.

For this topic, I'd like to discuss the mindset and practical tools of someone willing to develop horizontally in investments. Am not saying this approach is automatically better. It's just a different topic. Personally, it does feel like the best opportunities now are between industries, not inside one neat box.

reddit.com
u/Stir_123 — 2 months ago

Unpopular opinion but copying what works is how most channels actually grow

Gonna say something that might piss some people off but its the truth.

When you start a new channel, find someone in your niche who's smaller but growing fast and study what they're doing. Look at their thumbnails, their titles, their topics. If you can see their tags use those too. This helps the algorithm figure out what your channel is about and eventually your videos start showing up in their suggestions. Their audience finds you.

Once you have that core audience built up then you can start experimenting and doing your own thing. But early on trying to be completely original is usually a losing game.

I know this sounds grim. Trust me i get it. Nobody wants to hear stop being creative and just copy what works. We all have those passion project ideas we want to make. But if you're trying to actually grow and not just upload into the void sometimes you gotta play the game first.

Now here's where it gets complicated. If you hate making the type of content that performs well you're gonna burn out anyway. Doesn't matter if a video gets 100k views if you dreaded every second of making it. You're not gonna keep doing it. So theres a balance.

Also some people say this approach creates copycat slop with no longevity and honestly there's truth to that too. If all you do is imitate someone else eventually your audience realizes they're watching a watered down version and goes to the original. You need to add something of your own at some point.

I think the move is to use this strategy to get initial traction then pivot into what actually makes you excited once people are watching. Not everyone can afford to be purely creative from day one unless they already have an audience or dont care about growth at all.

My setup is nothing crazy btw. Basic desk, emeet pixy for facecam, decent usb mic, neewer ring light. I spent more time studying what works in my niche than buying gear honestly.

How do you guys balance doing what performs vs doing what you actually want to make?

reddit.com
u/Stir_123 — 2 months ago

I started with the Bestqool BQ60 mainly because I didn’t want to spend too much on red light therapy without knowing if I’d actually stick with it.

I’ve been using it mostly for face sessions and post-workout soreness. It’s pretty basic (660nm and 850nm), nothing fancy, but it’s easy enough to use that I don’t really overthink it.

Later on I started looking into other panels like Hooga and Mito and expected there would be a big difference in results, but that’s not really what stood out to me.

What I noticed more was just practicality and consistency. The BQ60 is smaller so I keep having to reposition it depending on what I’m targeting. With bigger panels, it seems like you just set it once and you’re done.

That’s kind of where the price difference makes sense to me, but I’m still not fully sure if it’s worth upgrading just for that.

I also can’t really tell if the actual effects on recovery are that different yet. Maybe I just haven’t used it long enough.

Curious if anyone here uses red light therapy alongside their usual recovery stack (supplements, sleep, training, etc.) and noticed any real difference, or if it’s mostly just convenience too.

reddit.com
u/Stir_123 — 2 months ago