Condense tests and especially e2e tests to reduce task times #87
Labels
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Grandiras/Ponente#87
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Tasks often take very long and that mostly is because of tests and e2e tests. Maybe we can trim those down to more reasonable cases and overall a smaller amount of tests.
🤖 On it — follow along in the action run; the reply will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the task runs.
Pushed to
claude/issue-87.What I found first
The premise needed checking before cutting anything, so I measured both suites:
dotnet test(xunit)workers: 1The unit suite isn't the problem — a quarter of a second, and the 13 s that
dotnet testtakes is almost entirely the build. I left it completely alone: there's nothing to win and coverage to lose. All the work went into e2e.The actual cost model
In e2e, the price of a test is almost never its assertions. It's the browser context, the
/dev/loginround trip, the prerender, the circuit connect, and whatever cards or tags it mints before it can look at anything. A lot of the suite was paying all of that to ask one more question of a screen the previous test already had open:createTag(two page loads) to measure one property of the same column-filter flyoutAccept-Languagenegotiation<link>tags out of the same<head>So I grouped assertions onto whichever test already builds the fixture they need, rather than deleting coverage.
174 → 123 tests (−29%), and −33 sign-in call sites. Only one assertion was actually dropped, and deliberately:
comment count shows on card after adding commenthad its single expectation behindif (count > 0), so it passed on an empty board — the badge is pinned for real by the test that creates a card, comments on it and measures where the badge lands.Three changes that aren't merges:
board-entrance's delay sweep,[0, 250, 700, 1500]→[0, 700]. It samples one boundary — whether the circuit's rebuild lands before or after the prerendered cascade would have started — and the two dropped points sit on the same two sides of it, for 6.75 s of sleep and two board loads. Its safety-reveal test also waited 5.5 s twice around a circuit delayed to 6.5 s; the second is now 2 s, still a full second past the circuit./login. They read computed styles off detached scraps of markup, so all they ever needed wasapp.css— not a sign-in and a board render, twice.I also wrote the rule into
CLAUDE.md("a test is a fixture, not an assertion"), because without it the suite regrows a test per assertion. It records both corollaries: grouping stops where the fixture stops being shared (a test that mutates a card still creates its own, since teardown can delete rows but not undo a write), and the timeout is why.What I verified — and what I could not
Verified: every spec and helper parses (
node --check); Playwright enumerates all 123 tests across 16 files with no duplicate titles; no helper function was left unused by the merges;dotnet buildsucceeds and 194/194 unit tests pass.I could not run the e2e suite. This CI container has no container engine (no
podman/docker), so the compose stack the suite needs can't be started — the merged tests are statically checked but have not been executed. That's the one thing worth a run on a machine with the stack up before merging. For the same reason I have no wall-clock number for you, only the structural counts above.One thing I deliberately did not touch:
workers: 1. It's the single biggest remaining lever, butCLAUDE.mddocuments it as a considered choice (BoardNotifier re-render cross-talk between parallel workers), and it's exactly the kind of change I can't validate without being able to run the suite. Worth revisiting separately if the wall clock is still the complaint after this.⏱ 17m 49s · 86 turns · tokens: 236.8k in / 78.1k out (+11465.2k cache-read)
📜 Full session transcript
→ Opened PR: #89