docs(agents): correct the testing guidance to match the shipped CI
The concurrency notes described `sequence.concurrent: true`, but that was rolled back — it starved timing-sensitive render tests on the 4-core CI runner. Rewrite to reflect what actually ships: - File-level parallelism (fileParallelism: true), tests within a file serial; explain WHY in-file concurrency is deliberately avoided (the local-vs-CI core-count trap) and that PTY needs pool: forks. - *.sequential.test.* files group process-global-state tests (fake timers, listener/yoga-node counts). - New rule: tests must not implicitly depend on host env. CI=true flips interactive mode off, so both vitest configs force CI:false; inject env behavior explicitly and reproduce CI with `CI=true vp run ci` on a fresh checkout. - FORCE_COLOR must also be set in spawned child envs, not just vitest config.
This commit is contained in:
@@ -6,12 +6,11 @@
|
||||
- Vue SFCs must use `<script setup>` unless there's an explicit reason not to.
|
||||
- When code must deviate from normal/idiomatic style because the situation genuinely requires it (e.g. a control-character regex in a terminal parser, a deliberate string code-point spread, a lint rule suppressed for a justified reason), add a comment explaining _why_ it has to be written that way. Don't silence a linter or write surprising code without a note — the next reader should not have to guess whether it's intentional.
|
||||
- Bug fixes must follow test-first: write a failing test that reproduces the bug, then fix the code and verify the test passes.
|
||||
- Tests must simulate real user conditions. Non-TTY environments disable chalk colors — use `FORCE_COLOR` env var so ANSI output is always exercised. A bug invisible in tests but visible in a real terminal is a testing gap, not a minor issue.
|
||||
- Tests must simulate real user conditions. Non-TTY environments disable chalk colors — use `FORCE_COLOR` env var so ANSI output is always exercised. A bug invisible in tests but visible in a real terminal is a testing gap, not a minor issue. Each spawned subprocess is a fresh Node process with its own chalk, so PTY/child helpers must set `FORCE_COLOR` in the child env too, not just the vitest config.
|
||||
- When debugging color/ANSI output in non-TTY environments (e.g. Claude Code shell), use `FORCE_COLOR=3` env var to force chalk to output ANSI codes.
|
||||
- **Tests run concurrently by default** (`sequence.concurrent: true` in both the main `vite.config.ts` and `vitest.pty.config.ts`). Write each test self-contained — own subprocess/app, no shared mutable state. Never assert on wall-clock-dependent behavior (e.g. exact render/commit counts that rely on the renderer's ~32ms throttle): trigger the event so it renders synchronously and assert immediately after, or it flakes under CPU contention. `it.sequential` does NOT fix cross-fork CPU contention — fix the timing dependency instead.
|
||||
- A few patterns are **incompatible with concurrent execution**. Handle by cause:
|
||||
- **Inline/file snapshots** (`toMatchInlineSnapshot`): the module-level `expect` loses snapshot test context under concurrency. _Fix in place_ — use the context-local `expect`: `test("name", async ({ expect }) => { expect(x).toMatchInlineSnapshot() })`. These stay concurrent.
|
||||
- **Process-global state** — fake timers (`vi.useFakeTimers`, which mutates global `setTimeout`/`performance`), or assertions on shared globals (e.g. `process.listenerCount`, live yoga-node counts). A concurrent sibling clobbers the shared state mid-test. _Not fixable_ in place — put these in a `*.sequential.test.*` file and mark them `it.sequential` / `describe.sequential`, with a header comment saying which global forces it.
|
||||
- **Test files run in parallel (`fileParallelism: true`), but tests WITHIN a file run serially.** We deliberately do NOT set `sequence.concurrent`: many render tests assert timing-sensitive counts driven by the renderer's ~32ms commit throttle, and in-file concurrency starves them of wall-clock on a 4-core CI runner (it passes on higher-core dev machines — the classic local-vs-CI trap). PTY tests need `pool: "forks"` (node-pty requires `child_process.fork`, not worker threads).
|
||||
- Tests that depend on **process-global state** — fake timers (`vi.useFakeTimers`, which mutates global `setTimeout`/`performance`) or assertions on shared globals (`process.listenerCount`, live yoga-node counts) — live in `*.sequential.test.*` files. Even file-level parallelism can perturb them, and grouping them by name documents the constraint. Add a header comment saying which global forces it.
|
||||
- Tests must not implicitly depend on the host environment. The CI runner sets `CI=true`, which flips `interactive = !isInCi && isTTY` off (disabling the resize listener, cursor, ANSI erases) — so both vitest configs force `env: { CI: "false" }`, and PTY child helpers set `CI: "false"` per-spawn. If a test needs a specific CI/TTY/color behavior, inject it explicitly (mount option, child env) rather than relying on the ambient value. Reproduce CI locally with `CI=true vp run ci` on a fresh checkout (`rm -rf packages/*/dist`).
|
||||
- After completing any task, run `vp run ready` (or `vpr ready`) to verify: lint, type-check, test all packages, and build.
|
||||
- Never commit anything under `docs/`. That directory is for local working notes and specs — it must stay out of git.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user