u/AvmnuSng

rspec-tracer 2.0.0.pre.2 — flaky-test detection restored + Rails-specific fixes from field testing
▲ 3 r/rails+1 crossposts

rspec-tracer 2.0.0.pre.2 — flaky-test detection restored + Rails-specific fixes from field testing

Follow-up to the pre.1 announcement on r/ruby a few weeks back. After field-testing pre.1 against several Rails apps, 15 issues surfaced; two more turned up during fix-verification. All 17 closed before the pre.2 tag this week.

What rspec-tracer is, briefly: test-skip caching for RSpec. Tracks which inputs each example depends on; second run skips examples whose inputs didn't change. Cold runs full; warm runs skip everything that wasn't affected by your edit.

Headline — flaky-test detection restored (#194). A top-line README feature present in 1.x since v1.0.0; silently dropped in the 2.0 rewrite. The :flaky registry status, the filter reason, the snapshot field, and the HTML reporter's Flaky tab were all retained, but no code path transitioned an example into :flaky. Caught by an attentive field tester noticing "no flakes reported across N CI runs where I know flakes exist." Would have shipped in 2.0.0 final without it.

Rails-specific fixes:

  • track_ar_schema_notifications now actually fires under the canonical README setup order (RSpecTracer.start BEFORE require_relative '../config/environment'). Pre-fix: defined?(::Rails::VERSION) was false at engine.setup time, so the entire Rails-observer install path short-circuited — sql.active_record subscriber never attached, AND the documented use_transactional_fixtures widening warn never fired. Two silent failures on one code path. Fix is a before(:suite) late-bind. (#192)
  • README per-example-precision section now covers Rails engines — an engine's own lib/ is required at gem-load time and lands in the boot set regardless of eager_load. COOKBOOK gains a transitive_load_tracking false opt-out recipe. (#189)
  • Engine + dummy-app two-rspec-summary explainer in COOKBOOK. (#190)
  • parallel_tests cache_hit_reason counts no longer inflated by worker count (was sum-merging identical per-worker tallies). (#193)

Identity-hash stability (long-standing bugs since v1.0.0):

  • example_id stable across runs when multiple files share a describe name (the load-order-dependent RSpec::ExampleGroups::Name_N disambiguator suffix is no longer in the digest). (#196)
  • example_id stable across line-shift edits for it { ... } / specify { ... } / example { ... } unnamed one-liners. Pervasive in shoulda-matchers model specs. (#210)
  • Companion fix to the duplicate-detection redesign: NPE in RSpec.world.example_count for suites with intermediate describe groups. (#218)

Cache + CLI:

  • storage_backend :json, serializer: :msgpack no longer crashes on Time values. (#182)
  • bin/rspec-tracer cache:info + explain compose with storage_backend :sqlite. (#183)
  • bin/rspec-tracer doctor no longer false-reports SimpleCov/Rails: not loaded. (#184)
  • cache:clear --force / -f synonym for --yes. (#191)
  • remote_cache success now emits visible INFO lines on download/upload/prune_all. (#188)
  • New coverage_modes config DSL for standalone Coverage runs. (#195)

Behavior changes worth knowing:

  • Cache schema_version bumps 3 → 5 cumulatively (one cold-run on upgrade).
  • Duplicate-example-identity detection now drops the colliders and runs the rest of the suite instead of aborting. fail_on_duplicates becomes purely an exit-code lever.

Both covered in UPGRADING.md.

Install:

# Gemfile
gem 'rspec-tracer', '= 2.0.0.pre.2', group: :test, require: false

Release notes: https://github.com/avmnu-sng/rspec-tracer/releases/tag/v2.0.0.pre.2 CI recipes (CircleCI / GitLab CI / Buildkite / Heroku CI): https://github.com/avmnu-sng/rspec-tracer/blob/main/docs/CI_RECIPES.md COOKBOOK (13 recipes): https://github.com/avmnu-sng/rspec-tracer/blob/v2.0.0.pre.2/COOKBOOK.md Discussion: https://github.com/avmnu-sng/rspec-tracer/discussions/180

2.0.0 rc.1 in ~1-2 weeks if pre.2 is clean. Feedback welcome — 2.0-feedback label on issues feeds straight into the rc.1 triage.

u/AvmnuSng — 6 days ago
▲ 9 r/ruby

I shipped rspec-tracer 1.0 in 2021 and the project sat untouched for 4 years. I came back to it last month, shipped 11 maintenance releases on the 1.x line absorbing crash + correctness fixes that had piled up in forks, then cut a 2.0 pre-release this week.

What rspec-tracer does, for context: tracks which source files each RSpec example depends on, so the second run of your suite skips examples whose inputs didn't change. Cold runs full; warm runs skip everything that wasn't affected by your edit.

What's new in 2.0 vs 1.x:

  • Pluggable storage: JSON (default, preserves the 1.x layout) or SQLite (single-file DB; MRI-only).
  • Pluggable remote cache: S3, Local-FS, or Redis (with optional per-key TTL + PR-branches sidecar).
  • Per-example tracks: DSL for declared deps the tracker can't auto-observe (tracks: { files: 'app/policies/**/*.rb', env: 'ROLE_CONFIG' }).
  • track_env(*names) config-level DSL with single-wildcard patterns ('RAILS_*', '*_TOKEN', '*').
  • Rails preset: track_rails_defaults auto-attaches views, locales, fixtures, factories, helpers, config, schema.
  • bin/rspec-tracer CLI: doctor, cache:info, cache:clear, report:open, explain <example_id>.
  • HTML + JSON reporters alongside terminal output.
  • SimpleCov branch coverage now works alongside (1.x had it disabled).

Pre-release framing — the gem is on RubyGems with --pre opt-in. 2.0.0 final is targeted in ~2 weeks. Looking for testers + feedback before final.

# Gemfile
gem 'rspec-tracer', '= 2.0.0.pre.1', group: :test, require: false

Release notes: https://github.com/avmnu-sng/rspec-tracer/releases/tag/v2.0.0.pre.1

u/AvmnuSng — 15 days ago