I think I finally understand why my dialogue was bad and it's not what every craft book told me
For years I read the standard advice on dialogue. Cut the small talk. Every line should reveal character or advance plot. People don't actually say each other's names that often. Avoid "as you know" exposition. Read your dialogue out loud. I followed all of it and my dialogue still felt off, and I couldn't figure out why.
The thing I eventually realized, embarrassingly late, is that my characters were all responding to what the previous character had just said. Every single time. It read like a tennis match. Statement, response, statement, response, perfectly logical, perfectly tracked, completely lifeless.
Real people don't do this. Real people half listen. They respond to the thing the other person said two sentences ago. They're thinking about what they want to say next and they miss the actual question. They answer a different question than the one that was asked, because the one they answered was the one that mattered to them. They interrupt themselves. They go on tangents. They circle back to something from earlier in the conversation that's been bothering them.
Once I started writing dialogue where the characters weren't really listening to each other, the whole book got noticeably more alive. It also got harder to write, because perfectly logical dialogue is much easier to construct than the messy, half overlapping, slightly mistuned thing that real conversation actually is. But the reader can feel the difference immediately, even if they can't articulate why.
I don't see this in any of the craft books I've read. They all talk about what dialogue should do. None of them talk about how it should fail to do those things, which is most of what makes it sound like people instead of character