u/jasmineliumai

I feel like at least a third of my time right now is just getting subsurface data into a usable state before I can do anything real with it. LAS files with inconsistent curve naming, well logs from different vintages that do not line up, SEG-Y that needs preprocessing before any interpretation tool will even open it. And that is before I start trying to cross reference any of it across wells or formations.

I did not expect this to be such a big part of the job honestly. I am wondering whether this gets easier with experience or if people just get faster at dealing with it. Did you learn most of this on the job or is there stuff you actively went and studied? And do bigger companies actually have this figured out or do they just have more people dealing with the same problem?

I am wondering if this is just me but it feels like at least half my week lately has been exporting things out of Petrel, reformatting them, figuring out why a file that worked fine last month is suddenly not reading correctly, tracking down which version is actually current. Like actual housekeeping stuff, not engineering.

The tools are fine when they work but nothing talks to anything else and at some point you end up just manually babysitting data between applications indefinitely. I do not know when that became a normal part of the job.

What I keep coming back to is whether petroleum engineering is actually getting harder or if the geology is roughly the same and we are just quietly drowning in more software that does not integrate. It feels like the second one but I might just be having a bad week.

Is this something people are solving internally or is everyone just dealing with it?

Is anyone actually solving the problem of subsurface data being completely siloed and inaccessible, or has the industry just accepted it?

I have been thinking about how much data exists across well logs, seismic surveys, core samples, pressure tests, some of it going back to the 1940s, and how little of it actually gets reused in any meaningful way. A geologist finishes an interpretation, it gets filed somewhere, and the next team that works the same area basically starts from scratch.

Other fields have built real infrastructure around making historical data queryable and reusable at scale. Subsurface feels like it is a decade behind on this, and I think part of it is format fragmentation (LAS, SEG-Y, RESQML, proprietary workstation exports) but honestly that feels more like a symptom than the actual problem. The data just was never treated as something worth investing in at the infrastructure level.

I'm curious if this is better in certain subsectors or companies or if it is pretty universal?

Running into this at Lium and I'm curious how other people handle it?

The deterministic parts of a harness are easy to test. Retry logic, parsing, routing, all of that you can unit test like normal code. But the second the model has to make a real judgment call how do you even write a test for that?

Do you check for an exact output and accept it'll be brittle since the model phrases things differently every run? Do you use another model as a judge, and if so, who tests the judge? Do you just run it fifty times and eyeball whether it feels right often enough?

I tried golden output diffing first. Failed constantly even when the agent was doing the right thing, just worded differently. Switched to LLM as judge for a bit, which works better but now I've got a non-deterministic test grading a non-deterministic system, which feels like it's just moving the problem one layer up instead of solving it.

Anyone landed on something that actually works here? Is it just accepted that agent testing is fuzzier than normal software testing, or is there a pattern I'm missing?

The pattern I keep running into after building a bunch of these is that the harnesses that actually hold up call the model way less than I expected starting out.

At my company (Lium) we deal with messy terabyte-scale scientific data, so picking the right tool or parser for a file is basically never a judgment call, it's deterministic almost every time.

But I see people routing everything through the model anyway. Tool selection when there's one obvious answer. Retries. Output parsing. Deciding when to stop. None of that needs judgment, it needs code. Do it through the model and you get something slow and hard to debug, since the failure could be hiding anywhere in a chain of probabilistic calls.

My diagnostic now is that if a broken step gets "fixed" by rewording the prompt instead of touching the code, that's a wrapper, not a harness.

Model gets called for genuine ambiguity, competing signals, stuff no rule covers cleanly. Everything else is plumbing, and once you map it out that pile is smaller than you'd think.

How do you all draw that line? Hard rule or more case by case?

Anthropic just accidentally made one of the best ads for Lium.

They built Fable 5, their most powerful model yet, then put so many guardrails around it that biology, chemistry, cybersecurity, and other scientific topics can cause it to back away from the conversation.

That raises a pretty simple question.

What is the point of building a frontier model if it gets uncomfortable when you ask frontier questions?

Lium was made for the science that Fable blocks.

We built it because a lot of the people doing serious work in science, engineering, energy, geospatial analysis, and technical research don't need an AI that is constantly looking for reasons not to help. They need one that can actually engage with difficult problems and large, messy datasets.

Real research is rarely clean. It involves uncertainty, complexity, and questions that don't fit into a polished demo.

While everyone is arguing about how capable Fable 5 is, we're focused on something more practical.

Can it actually help people do their work?

If you've hit a wall with Fable, try the same problem in Lium.

We're live on Product Hunt today: https://www.producthunt.com/products/lium

Come test the science Fable won't touch.

https://preview.redd.it/wdith04iso6h1.png?width=2320&format=png&auto=webp&s=c4073821637915678ee9d0dbfb595d7ab0fb7fab

I’m wondering how much the software and data side of climate tech is going to grow over the next few years compared to the more visible stuff like EVs and solar. Things like environmental data infrastructure, satellite monitoring, grid software, climate risk tools, wildfire detection, optimization systems, and emissions tracking seem like they could become a much bigger part of the space.

I’m wondering what areas people here think are going to matter the most going forward, especially on the technical side.

I've been interested in climate and environmental data science lately and I’m curious what areas people think are growing the fastest right now?

Could be things like:

• satellite data
• wildfire prediction
• energy and grid optimization
• climate risk
• emissions tracking
• weather modeling
• environmental AI

It would also be cool to hear what people here work on and what skills/tools seem most useful in the space.