r/SpecDrivenDevelopment

(d'oh. replace 'crud' with 'clutter.' )

For 15 years, the assorted test directories cluttering the root of my projects have driven me nuts... so I decided to see how fast I could build a fully tested gem using SDD with claude - and finally fixed it.

Oubliette banishes your tests to a single location, available for execution but out of sight until you need them. Now my code and my tests live in two clean trees i can view side by side.

https://github.com/soychicka/oubliette

• Moves test directories, fixtures, seeds, helpers, and reports into one place
  • Configures rspec, cucumber, cucumber_env, jest, and jasmine for you
  • Wires up fixtures, FactoryBot, Capybara, VCR, and SimpleCov at runtime
  • Leaves your executable JavaScript alone, but gives precise wiring instructions for Vitest, Playwright, Cypress, and Karma
• One command to run them all (yes, wrong quest, but you get the point)
• Tracks pass/fail/pending counts over time, so coverage drift shows up as a number that moved, not a surprise six months from now

If you need an escape hatch, rake oubliette:hoggle leads you out of the oubliette and back into the labyrinth, putting everything back where it was and cleaning up after itself.

And the important stuff:

• Bonus 1: a brief history of the oubliette, from the famine, plague, and war of the 14th century that helped free peasants from their feudal masters.
• Bonus 2: gratuitous Labyrinth references.
• Bonus 3: no rewrites to any package.json files, so you're safe from gratuitous David Bowie references.

I've only tested it against Ruby 3, but it's been running happily in an rspec/capybara/cucumber-tested Rails 4 to 8 migration, moving from jQuery/Bootstrap to Stimulus/Tailwind.

Haven't released a proper gem yet: I haven't used VCR, SimpleCov, Vitest, Playwright, Cypress, or Karma in any of my projects - so those need exploration in live implementations.

If you give it a try, let me know what you think.

( and yes, I let Claude write the docs 🥴 )

We are designing an evaluation for a structural code-access layer used by coding agents.

The three configurations are:

1. Native shell, grep, read and edit tools.
2. Native tools plus optional structural queries.
3. Automatically routed structural transactions with native fallback.

We believe the benchmark needs to measure more than task completion:

- Pass rate.
- End-to-end wall-clock time.
- Input and output tokens.
- Tool-call count.
- Bytes returned to the model.
- Repository exploration before the first edit.
- Repeated retrieval of the same context.
- Unexpected files and entities changed.
- Validation and retry count.
- Cold versus warm index performance.
- Unsupported-language fallback correctness.

One lesson so far is that measuring index-query latency alone is misleading. A 7 ms query does not guarantee a faster session if the model performs additional calls or ignores the result.

Our implementation is Sem:
https://github.com/Ataraxy-Labs/sem

What controls or failure cases would you require before accepting claims of token or session-speed improvement?