mirror of
https://github.com/pestphp/pest.git
synced 2026-09-05 14:23:34 +02:00
wip
This commit is contained in:
nuno maduro
+1
-1
@@ -1,7 +1,7 @@
|
||||
.idea/*
|
||||
.idea/codeStyleSettings.xml
|
||||
.temp/*
|
||||
composer.lock
|
||||
/composer.lock
|
||||
/vendor/
|
||||
coverage.xml
|
||||
.phpunit.result.cache
|
||||
|
||||
@@ -0,0 +1,342 @@
|
||||
# TIA write-tier conformance — plan
|
||||
|
||||
Baseline sweep: **147 cases, 144 PASS / 2 FAIL / 1 SKIP** against `dev-fix/tia-filtered` (`db70017`)
|
||||
on `playground/laravel` with pcov enabled.
|
||||
|
||||
This file records (1) what is left to fix, and (2) the full case matrix with the target
|
||||
outcome for every row, so the sweep can be re-run as a conformance check after the fixes land.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — What's left
|
||||
|
||||
### 1. `--parallel` silently under-records `edges` (G12, I5) — **high**
|
||||
|
||||
Any `--tia --parallel` complete run writes `files=4` where sequential writes `files=25`. Only
|
||||
`app/**` and blade views survive; every `config/`, `routes/`, `bootstrap/` edge and every test
|
||||
self-edge is lost. A `routes/web.php` edit that sequential TIA catches, parallel TIA reports as
|
||||
`No affected tests found`.
|
||||
|
||||
Root cause — `src/Restarters/PcovRestarter.php:33`:
|
||||
|
||||
```php
|
||||
if (! Tia::isEnabledForRun($arguments)) {
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
`Tia::isEnabledForRun()` (`src/Plugins/Tia.php:275-288`) returns true only for `--tia` in argv or
|
||||
the `PEST_TIA` env flag. A paratest worker's argv carries neither, so workers never re-exec with
|
||||
`-d pcov.directory=<projectRoot>` and record under pcov's empty default scope.
|
||||
|
||||
Proven both directions:
|
||||
- `PEST_PCOV_RESTARTER_RESTARTED=1 pest --tia` (sequential, restart suppressed) reproduces
|
||||
`files=4` with edge sets identical to the parallel graph.
|
||||
- `PEST_TIA=1 pest --parallel` restores the full `files=25` graph. **Working workaround today.**
|
||||
|
||||
Coverage config is irrelevant: widening `phpunit.xml` `<include>` to `app|config|routes|bootstrap|tests`
|
||||
still yields `files=4`.
|
||||
|
||||
Sticky, not just per-run: `pest --tia --filtered --parallel` re-records only the affected test
|
||||
through a worker and strips that entry's edges from an otherwise-healthy sequential graph, so one
|
||||
parallel run degrades a good graph incrementally.
|
||||
|
||||
**Fix direction:** propagate the TIA-enabled decision into workers (e.g. set `PEST_TIA` in the
|
||||
worker env, or have the parent stamp a recording global the restarter also consults) so
|
||||
`PcovRestarter` restarts them. Then assert `edges` equivalence between sequential and parallel.
|
||||
|
||||
### 2. Deleted test file with a cached failure wedges `--filtered` forever (I9) — **high**
|
||||
|
||||
Expected a WARN and a full-suite fallback. Actual: `--filtered` selects the phantom file, runs
|
||||
nothing, exits **0 green**, on every subsequent invocation.
|
||||
|
||||
Root cause — `src/Plugins/Tia/Graph.php:1523-1525`:
|
||||
|
||||
```php
|
||||
if ($real === false) {
|
||||
$real = $path;
|
||||
}
|
||||
```
|
||||
|
||||
`relative()` falls back to the raw path when `realpath()` fails, so a deleted-but-in-project test
|
||||
file is never "unlocated", `Graph::hasUnlocatedTestsToRerun()` never fires, and the WARN branch at
|
||||
`src/Plugins/Tia.php:982-986` is unreachable for the deletion case. `pruneMissingTests()` only runs
|
||||
under `--fresh`, so nothing clears it.
|
||||
|
||||
**Fix direction:** distinguish "path could not be resolved" from "path resolved outside the root" in
|
||||
`relative()`, or have `hasUnlocatedTestsToRerun()` stat the file directly.
|
||||
|
||||
### 3. `--coverage-text` does not disable filtered mode (I8) — **low**
|
||||
|
||||
`coverageReportActive()` (`src/Plugins/Tia.php:1705-1710`) reads only Pest's own `--coverage`, so
|
||||
raw PHPUnit coverage flags (`--coverage-text`, `--coverage-html`, `--coverage-clover`, …) leave
|
||||
filtered mode on. `Tia::COVERAGE_FLAGS` at `:113-115` already enumerates them — that list is not
|
||||
consulted here.
|
||||
|
||||
### 4. `--coverage` narrows edges like parallel does — **medium, needs triage**
|
||||
|
||||
`pest --tia --coverage` prints `fresh graph (recording coverage baseline)` even when a graph
|
||||
exists, and writes `Feature/ExampleTest` with 2 files instead of 16, dropping self-edges. Same
|
||||
observable shape as G12, likely a different mechanism (piggyback collector scoped by
|
||||
`phpunit.xml <include>` rather than the pcov-restarted recorder). Theme: **edges silently narrow to
|
||||
`app/**` whenever recording does not go through the pcov-restarted TIA recorder.** Confirm whether
|
||||
these are one fix or two.
|
||||
|
||||
### 5. Smaller items
|
||||
|
||||
| Item | Where | Note |
|
||||
|---|---|---|
|
||||
| Warning/deprecation recorded as `status=0` | write path | Codes `6` and `4` appear unreachable; risky/skipped/incomplete record faithfully |
|
||||
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
|
||||
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
|
||||
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
|
||||
| Replay clobbers cached `time` | write path | `0.058` → `0.001`; timing data degrades toward zero across runs |
|
||||
| `pest --parallel` without `--tia` writes nothing | G3, G6 | Sequential `pest --filter=…` does record. Parallel CI contributes nothing to the cached-failure replay path |
|
||||
| `--tia --parallel --filter`/`--shard` record nothing | G2, G8 | More conservative than the results-only contract requires |
|
||||
| `--min=50` without `--coverage` is a silent no-op | — | — |
|
||||
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
|
||||
|
||||
### 6. Test-harness gaps to close before re-running
|
||||
|
||||
- The playground has no `UsesClass`, `->note()`, `->flaky()`, `->issue()`, `->pr()`, `->ticket()` or
|
||||
`->assignee()` annotations, so **C19, C23–C28, C37–C39 matched zero tests** and their
|
||||
graph-invariant assertions proved nothing. Add annotated fixtures to make those rows load-bearing.
|
||||
- Use **sentinel patching** (rewrite every cached entry to `time=9.999 assertions=42`, then see which
|
||||
entries get overwritten) as the discriminator. It is the only way to tell "wrote identical values"
|
||||
from "wrote nothing". A canary test absent from `edges` is unreliable under `--filter` because it
|
||||
never matches the filter and so never runs.
|
||||
- `--tia` on a clean green tree can never cache a failure (unchanged tests replay rather than
|
||||
execute); seeding one requires `--fresh` or an env-driven flaky fixture.
|
||||
- Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix (target outcomes)
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Status column: `PASS` = conforming today · `FIX` = must pass once the item above lands ·
|
||||
`SKIP` = case not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | PASS |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | PASS |
|
||||
| A4 | `git status` after reset ritual | clean | PASS |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | PASS |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | PASS |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | PASS |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | PASS |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS |
|
||||
| B10 | `pest --tia` twice | `time` differs; statuses stable | PASS |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key + new result appear | PASS |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written | PASS |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS |
|
||||
| D11 | `pest --tia --bail` | RO | PASS |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran | PASS |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | PASS |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | PASS |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written | PASS |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | PASS (but see G12 — it strips edges) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | PASS |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | PASS |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** (`files=25` both) | **FIX (item 1)** |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | PASS |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | PASS |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **FIX (item 1)** |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report | **FIX (item 3)** |
|
||||
| I9 | cached failure whose test file was deleted | WARN `could not be located on disk`; falls back to the full suite with replay | **FIX (item 2)** |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild (and the drift reason should be printed — item 5) | PASS |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE | PASS |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (signal must go to the re-exec'd child — item 5) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS |
|
||||
|
||||
---
|
||||
|
||||
## Re-run procedure
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json"
|
||||
|
||||
# per case
|
||||
git checkout -q . && git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
Every pest invocation must be prefixed with `PAO_DISABLE=1` (the app has `laravel/pao`, which emits
|
||||
JSON when it detects an agent). The shell is zsh — build commands with arrays or `eval`; unquoted
|
||||
`$args` does not word-split.
|
||||
@@ -0,0 +1,455 @@
|
||||
# TIA default-branch fallback — phase three
|
||||
|
||||
## Your task
|
||||
|
||||
Fix [pestphp/pest#1823](https://github.com/pestphp/pest/issues/1823) — TIA's cached results never
|
||||
hit for repos whose default branch is not literally `main` — then re-run the **whole** conformance
|
||||
matrix against the playground app, plus the new section **L** that covers the fix.
|
||||
|
||||
Work in this order, and **stop where the plan says stop**:
|
||||
|
||||
1. **Part 1** — read the diagnosis. It is already measured; do not re-derive it, but do re-confirm
|
||||
the two reproductions in Part 1.3 take ~2 minutes and prove your environment is sane.
|
||||
2. **Part 2** — implement the fix in the `pestphp/pest` repo. **Do not commit. Do not touch the
|
||||
playground's `vendor/`.**
|
||||
3. **HARD STOP → Part 3.** Report the diff to Nuno and wait. He validates, commits, and syncs it
|
||||
into the playground himself. You must not proceed until he confirms.
|
||||
4. **Part 4** — verify the sync landed, then run section **L** (new) and the full phase-two matrix
|
||||
(**A–K**, all 156 rows) against the playground.
|
||||
5. **Part 5** — report in the given format.
|
||||
|
||||
Per `CLAUDE.md`: **do not write new `pestphp/pest` unit tests and do not run `composer test`.** Make
|
||||
the change, report it, and ask whether repo tests should be added. The section-L rows are *playground
|
||||
invocations*, not repo tests — those are the deliverable and are always in scope.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — The diagnosis (already measured; commit `bfd5b756`)
|
||||
|
||||
### 1.1 Root cause — two independent hardcoded `'main'` literals
|
||||
|
||||
**(a) The read fallback.** `src/Plugins/Tia/Graph.php` — seven methods default the fallback to the
|
||||
literal `'main'`:
|
||||
|
||||
| line | method |
|
||||
|---|---|
|
||||
| 579 | `recordedAtSha(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 614 | `getAssertions(…, string $fallbackBranch = 'main')` |
|
||||
| 625 | `getTime(…, string $fallbackBranch = 'main')` |
|
||||
| 636 | `getResult(…, string $fallbackBranch = 'main')` |
|
||||
| 663 | `testFilesToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 700 | `hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 811 | `lastRunTree(string $branch, string $fallbackBranch = 'main')` |
|
||||
|
||||
They all funnel into `Graph::baselineFor()` (line 819), which *does* implement a real cross-branch
|
||||
fallback:
|
||||
|
||||
```php
|
||||
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
|
||||
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
|
||||
return ['sha' => null, 'tree' => [], 'results' => []];
|
||||
```
|
||||
|
||||
The mechanism is deliberate. The problem is that **no caller ever passes `$fallbackBranch`** — all
|
||||
nine call sites in `src/Plugins/Tia.php` (lines 419, 429, 432, 785, 857, 1026, 1031, 1048, 1056) pass
|
||||
only `$this->branch`. So on a `master`-named repo the second branch of `baselineFor()` can never
|
||||
fire, and the whole mechanism is dead code.
|
||||
|
||||
`grep -rniE 'defaultBranch|symbolic-ref|init\.defaultBranch|origin/HEAD' src/` returns **nothing** —
|
||||
no default-branch resolution exists anywhere.
|
||||
|
||||
**(b) The detached-HEAD default.** `src/Plugins/Tia.php:206` declares `private string $branch = 'main';`
|
||||
and `ChangedFiles::currentBranch()` (`ChangedFiles.php:208`) returns `null` for detached HEAD. So on
|
||||
detached HEAD `$this->branch` stays the literal `'main'` and is used for **both reads and writes** —
|
||||
minting a baseline key for a branch that does not exist. This is a *write*-side bug and a separate fix
|
||||
from (a).
|
||||
|
||||
### 1.2 Not a regression
|
||||
|
||||
`git log -S"fallbackBranch = 'main'"` bottoms out at `c7e32f5d feat(tia): continues to work on poc`.
|
||||
This is original PoC code, untouched by phase one. Phase one's change 1 modified
|
||||
`hasUnlocatedTestsToRerun()`'s file-existence check — one of the seven methods — without going near
|
||||
the fallback. Do not report it as a phase-one regression.
|
||||
|
||||
No test anywhere exercises the mechanism: `tests/Unit/Plugins/Tia/Graph.php` uses `'main'` as the
|
||||
*actual* branch name, so those assertions pass whether or not the fallback exists. A branch-name
|
||||
mismatch is never tested.
|
||||
|
||||
### 1.3 The two reproductions (re-confirm these before you start)
|
||||
|
||||
Both on the playground, `master`-named default branch, zero local changes:
|
||||
|
||||
```
|
||||
default = master default = main
|
||||
1. record on default → full run (cold) → full run (cold)
|
||||
2. 1st run on feature-x → 25 UNCACHED → 25 replayed ← (a)
|
||||
3. 2nd run on feature-x → 25 replayed → 25 replayed
|
||||
4. back on default → 25 replayed → 25 replayed
|
||||
5. 1st run on feature-y → 25 UNCACHED → 25 replayed
|
||||
```
|
||||
|
||||
```
|
||||
master-only graph, then `git checkout --detach`, then `pest --tia`:
|
||||
→ 25 uncached, and keys become [master,main] ← (b) spurious key
|
||||
```
|
||||
|
||||
The cost is **one full run per new branch, forever**, with no output explaining why — the only clue
|
||||
is the `N uncached` count; the headline is just `─ Experimental TIA mode enabled.`
|
||||
|
||||
### 1.4 Correction to phase two
|
||||
|
||||
Phase two reported **H6 and H7 as passing. Both were false passes.** They ran after H5, which had
|
||||
renamed `master`→`main` and left a `main` key in the graph, so the hardcoded fallback resolved by
|
||||
accident. Re-measured against a clean `master`-only graph, both fail. Section L replaces them as the
|
||||
load-bearing rows; H6/H7 must be re-run **from a cold graph** this time (see Part 4.2).
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — The fix to implement
|
||||
|
||||
### 2.1 Recommended design
|
||||
|
||||
**Step 1 — add a non-throwing resolver** to `src/Plugins/Tia/ChangedFiles.php`, next to
|
||||
`currentBranch()`:
|
||||
|
||||
```php
|
||||
public function defaultBranch(): ?string
|
||||
```
|
||||
|
||||
Resolution order, each step failing soft to the next:
|
||||
|
||||
1. `git symbolic-ref --short refs/remotes/origin/HEAD` → strip a leading `origin/`
|
||||
2. `git config --get init.defaultBranch`
|
||||
3. `null`
|
||||
|
||||
Unlike `currentBranch()`, this must **never throw** `MissingDependency` — it is advisory. Return
|
||||
`null` on any non-zero exit or empty output.
|
||||
|
||||
**Step 2 — add a config surface.** `src/Plugins/Tia/Configuration.php` already exposes `always()`,
|
||||
`locally()`, `filtered()`, `baselined()`, `watch()`. Add:
|
||||
|
||||
```php
|
||||
public function defaultBranch(string $branch): self
|
||||
```
|
||||
|
||||
so `pest()->tia()->defaultBranch('master')` works in `tests/Pest.php`. Explicit config **always
|
||||
wins** over autodetection — that is the escape hatch when `origin/HEAD` is unset.
|
||||
|
||||
**Step 3 — resolve once, in `Tia.php`.** The read path is hot (`getResult()` is called per test at
|
||||
line 419), so resolution must not shell out per call. Resolve alongside `$this->branch` at
|
||||
`Tia.php:1910`, under the existing `$branchResolved` guard:
|
||||
|
||||
```php
|
||||
$this->fallbackBranch = $configuredDefaultBranch
|
||||
?? $changedFiles->defaultBranch()
|
||||
?? 'main';
|
||||
```
|
||||
|
||||
**Step 4 — thread it into `Graph`.** Prefer a `Graph`-level property over editing nine call sites:
|
||||
add `Graph::setFallbackBranch(string $branch)`, change the seven signatures to
|
||||
`?string $fallbackBranch = null`, and resolve inside each with
|
||||
`$fallbackBranch ??= $this->fallbackBranch;`. `baselineFor()` itself needs no change. This keeps the
|
||||
public signatures backward-compatible and minimises blast radius.
|
||||
|
||||
**Step 5 — fix the detached-HEAD write.** `Tia.php:206`'s `= 'main'` default must become the
|
||||
resolved default branch, so detached HEAD stops minting a phantom key.
|
||||
|
||||
### 2.2 Invariants the fix must not break
|
||||
|
||||
These are all covered by existing matrix rows — the fix is wrong if any of them moves:
|
||||
|
||||
- **Read-only.** The fallback must affect *reads* only. Writes go through `ensureBaseline($branch)`
|
||||
and must keep using the real current branch. Otherwise H1–H4/H8 ("no baseline key other than the
|
||||
real branch") break.
|
||||
- **H9** — a non-git dir with `--tia` must still raise
|
||||
`MissingDependency: The feature "Tia mode" requires "git".` Adding a soft resolver must not
|
||||
swallow that.
|
||||
- **H10** — plain `pest` in a non-git dir must still run and create no baseline dir.
|
||||
- **A2/A3** — cold-graph recording unchanged.
|
||||
- **I1/E3** — clean+green `--tia --filtered` must still be a true zero-delta run.
|
||||
- Filtered mode reads `testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so the fallback must
|
||||
reach those two as well, not just `getResult()`.
|
||||
|
||||
### 2.3 Decisions for Nuno (raise these at the Part 3 stop)
|
||||
|
||||
- **D1** — Consult `origin/HEAD` at all? It requires `git remote set-head` and is absent in many CI
|
||||
checkouts and all remote-less repos. Config + `init.defaultBranch` only is simpler but helps fewer
|
||||
people out of the box. *Recommendation: keep it, first in the chain, since it fails soft.*
|
||||
- **D2** — Single-key heuristic: if the graph holds exactly one baseline key, use it as the fallback?
|
||||
Fixes the issue with zero git calls, but is implicit and surprising when several keys exist.
|
||||
*Recommendation: no.*
|
||||
- **D3** — Should detached HEAD write a baseline at all, or be read-only? Current behaviour mints a
|
||||
key. *Recommendation: read-only.*
|
||||
- **D4** — Should `pest()->tia()->defaultBranch()` validate that the branch exists, or accept any
|
||||
string? *Recommendation: accept any string; a nonexistent name degrades to a full run, which is
|
||||
safe.*
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — HARD STOP
|
||||
|
||||
When the code is written:
|
||||
|
||||
1. Show Nuno the diff (`git -C /Users/nunomaduro/Work/projects/pestphp/pest diff`) and a one-paragraph
|
||||
summary of each file's change.
|
||||
2. Answer/raise the D1–D4 decisions.
|
||||
3. State explicitly that you have **not** committed and have **not** synced `vendor/`.
|
||||
4. Ask whether repo unit tests should be added (per `CLAUDE.md`), describing the tests you have in
|
||||
mind — do not write them yet.
|
||||
5. **Wait.** Nuno commits and applies the change to the playground.
|
||||
|
||||
**Never sync the playground's `vendor/` yourself.** `vendor/pestphp/pest` there is a dist copy, not a
|
||||
symlink (composer installed `dev-fix/tia-filtered as 5.2.0`), so pest-repo edits do not reach it. Say
|
||||
what is stale and wait. This applies to before/after contrasts too.
|
||||
|
||||
Once he confirms, verify the sync actually landed before measuring anything:
|
||||
|
||||
```bash
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
grep -c 'defaultBranch' "$V/src/Plugins/Tia/ChangedFiles.php" # must be ≥ 1
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php src/Plugins/Tia/ChangedFiles.php \
|
||||
src/Plugins/Tia/Configuration.php; do
|
||||
diff -q "/Users/nunomaduro/Work/projects/pestphp/pest/$f" "$V/$f" >/dev/null \
|
||||
&& echo "SAME $f" || echo "STALE $f"
|
||||
done
|
||||
```
|
||||
|
||||
If anything reports `STALE`, stop and tell him. State in your final report which pest commit produced
|
||||
the playground numbers.
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Measurement
|
||||
|
||||
### 4.1 Environment and traps
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel` (branch `master`).
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON when
|
||||
it detects an agent.
|
||||
2. **Pin the interpreter to `php85`.** The playground requires PHP `>= 8.4.1` *and* pcov. Locally only
|
||||
`php85` (8.5.8) has both — `php84` (8.4.23) has no pcov, and the bare Herd `php` shim has been
|
||||
observed drifting to 8.3.32 mid-session, which kills every run in
|
||||
`vendor/composer/platform_check.php`. Also put a `php` → `php85` symlink first on `PATH`: the
|
||||
`--shard` list-tests probe spawns a subprocess via bare `php`, not `PHP_BINARY`.
|
||||
3. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`. Scope
|
||||
resets to `git checkout -- tests app` / `git clean -fd tests app`. For `phpunit.xml` and
|
||||
`composer.lock`, copy aside and copy back, verifying with `shasum`.
|
||||
4. **zsh does not word-split unquoted parameters.** A `$PEST` string containing a space becomes one
|
||||
command name. Route every invocation through `eval` (the `pest()` helper below does this).
|
||||
5. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
6. **The sentinel technique must not falsify `assertions` on zero-assertion tests.** A
|
||||
risky/skipped/incomplete status is *derived* from "performed no assertions", so patching those to
|
||||
`42` rewrites the status on replay and destroys the discriminator — it shows up as a phantom
|
||||
`status 5→0` defect. The `sentinel.php` below only patches `assertions` where it is already
|
||||
non-zero. With that, a full replay gives a clean `rewritten=0`.
|
||||
7. Restore branch state after every L row. Several rows rename or detach; leaving a stray `main` key
|
||||
in the graph is exactly what produced phase two's false H6/H7 passes.
|
||||
|
||||
### 4.2 Reference numbers
|
||||
|
||||
The playground already carries the phase-two fixtures at commit `fb2e77e` — **do not rebuild them.**
|
||||
A healthy sequential `--tia` graph is:
|
||||
|
||||
> **`files=27`, 10 edge keys (one per test file), self-edge on all 10, `n=25` results.**
|
||||
|
||||
`sha` will differ once Nuno commits the vendor sync — re-derive it once and use it throughout. The
|
||||
`files`/`edges`/`n` numbers hold as long as no test fixture changes. Suite shape: 10 test files, 25
|
||||
tests, including six deliberate status fixtures (skipped, todo, incomplete, risky, warning,
|
||||
deprecation), a 3-row dataset, a `smoke` group, an env-driven flaky test (green unless
|
||||
`FLAKY_FAIL=1`), and the annotation set (`covers`/`note`/`flaky`/`issue`/`pr`/`ticket`/`assignee`).
|
||||
|
||||
**Re-run H6 and H7 from a cold graph** (`rm -rf` the graph dir, record on `master` only, *then*
|
||||
branch/detach). Their phase-two results are void.
|
||||
|
||||
### 4.3 Harness
|
||||
|
||||
Write these to your scratchpad. `$SP` is your own scratchpad dir.
|
||||
|
||||
<details>
|
||||
<summary><code>lib.sh</code></summary>
|
||||
|
||||
```bash
|
||||
#!/bin/zsh
|
||||
export PAO_DISABLE=1
|
||||
PG=/Users/nunomaduro/Work/projects/playground/laravel
|
||||
SP="<your scratchpad>"
|
||||
PHPBIN="php85"
|
||||
PEST="$PHPBIN $PG/vendor/bin/pest"
|
||||
cd "$PG" || exit 1
|
||||
pest() { eval "$PEST $*"; } # zsh: no word-splitting, must eval
|
||||
GRAPHDIR="$(pest --baseline)"
|
||||
GRAPH="$GRAPHDIR/graph.json"
|
||||
mkdir -p "$SP/bin" && ln -sf "$(command -v php85)" "$SP/bin/php"
|
||||
export PATH="$SP/bin:$PATH" # --shard spawns bare `php`
|
||||
reset_tree() { git checkout -- tests app 2>/dev/null; git clean -qfd tests app 2>/dev/null; }
|
||||
seed() { rm -rf "$GRAPHDIR"; pest --tia >/dev/null 2>&1; $PHPBIN "$SP/sentinel.php" "$GRAPH" >/dev/null; cp "$GRAPH" "$SP/before.json"; }
|
||||
snap() { cp "$GRAPH" "$SP/before.json"; }
|
||||
delta() { $PHPBIN "$SP/cmp.php" "$SP/before.json" "$GRAPH"; }
|
||||
keys() { $PHPBIN -r '$g=json_decode(file_get_contents($argv[1]),true);echo "[".implode(",",array_keys($g["baselines"]??[]))."]";' "$GRAPH"; }
|
||||
tally() { sed -E $'s/\x1b\\[[0-9;]*[a-zA-Z]//g' <"$SP/out.txt" | grep -E 'Tests:' | sed -E 's/^ +//;s/Tests: +//'; }
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>sentinel.php</code> — the write discriminator</summary>
|
||||
|
||||
```php
|
||||
<?php // sentinel.php <graph.json>
|
||||
$p = $argv[1];
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR);
|
||||
$n = 0;
|
||||
foreach ($g['baselines'] ?? [] as $br => $b) {
|
||||
foreach (array_keys($b['results'] ?? []) as $id) {
|
||||
$g['baselines'][$br]['results'][$id]['time'] = 9.999;
|
||||
// Only falsify a non-zero assertion count: risky/skipped/incomplete are
|
||||
// DERIVED from "performed no assertions", so patching those to 42 would
|
||||
// rewrite the status on replay and destroy the discriminator.
|
||||
if ((int) ($b['results'][$id]['assertions'] ?? 0) > 0) {
|
||||
$g['baselines'][$br]['results'][$id]['assertions'] = 42;
|
||||
}
|
||||
$n++;
|
||||
}
|
||||
}
|
||||
file_put_contents($p, json_encode($g, JSON_THROW_ON_ERROR));
|
||||
echo "sentinelled $n results\n";
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>oneline.php</code> — one compact tier verdict per row</summary>
|
||||
|
||||
```php
|
||||
<?php // oneline.php <before.json> <after.json>
|
||||
function load(string $p): ?array {
|
||||
return is_file($p) ? json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR) : null;
|
||||
}
|
||||
$a = load($argv[1]); $b = load($argv[2]);
|
||||
if ($a === null || $b === null) { echo 'GRAPH '.($b === null ? 'DELETED' : 'CREATED'); exit; }
|
||||
function edgeSets(array $g): array {
|
||||
$out = [];
|
||||
foreach ($g['edges'] ?? [] as $t => $ids) { $s = array_map(fn($i) => $g['files'][$i] ?? "?$i", (array) $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$moved = [];
|
||||
if (edgeSets($a) !== edgeSets($b)) $moved[] = 'edges';
|
||||
if (($a['files'] ?? []) !== ($b['files'] ?? [])) $moved[] = 'files';
|
||||
if (($a['fingerprint'] ?? null) !== ($b['fingerprint'] ?? null)) $moved[] = 'fingerprint';
|
||||
$brA = array_keys($a['baselines'] ?? []); $brB = array_keys($b['baselines'] ?? []);
|
||||
if ($brA !== $brB) $moved[] = 'branchkeys('.implode('|', $brA).'->'.implode('|', $brB).')';
|
||||
$add = $rem = $wr = 0; $shaMoved = $treeMoved = false;
|
||||
foreach ($brB as $br) {
|
||||
$ra = $a['baselines'][$br]['results'] ?? []; $rb = $b['baselines'][$br]['results'] ?? [];
|
||||
if (($a['baselines'][$br]['sha'] ?? null) !== ($b['baselines'][$br]['sha'] ?? null)) $shaMoved = true;
|
||||
if (($a['baselines'][$br]['tree'] ?? null) !== ($b['baselines'][$br]['tree'] ?? null)) $treeMoved = true;
|
||||
$add += count(array_diff(array_keys($rb), array_keys($ra)));
|
||||
$rem += count(array_diff(array_keys($ra), array_keys($rb)));
|
||||
foreach ($ra as $id => $x) {
|
||||
if (! isset($rb[$id])) continue;
|
||||
foreach (['status','time','assertions','message'] as $f) {
|
||||
if (($x[$f] ?? null) !== ($rb[$id][$f] ?? null)) { $wr++; break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
$n = 0; foreach ($brB as $br) $n = max($n, count($b['baselines'][$br]['results'] ?? []));
|
||||
printf('n=%d w=%-2d +%d -%d %s%s%s', $n, $wr, $add, $rem,
|
||||
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
|
||||
$shaMoved ? ' sha:CHANGED' : '', $treeMoved ? ' tree:chg' : '');
|
||||
```
|
||||
</details>
|
||||
|
||||
`cmp.php` (verbose per-entry version of the same) and `summarise.php` / `edgediff.php` are in
|
||||
`PLAN_PHASE_TWO.md` §"Graph summariser" — reuse them for drill-downs. Reading the verdict:
|
||||
|
||||
- `w=` — entries actually **written**. Under sentinel patching this is the only reliable way to tell
|
||||
"wrote identical values" from "wrote nothing". A full replay must give `w=0`.
|
||||
- `struct:ok` + `+0 -0` — no prune, no edges/files/fingerprint movement. Required by RESULTS-ONLY.
|
||||
- `STRUCT:branchkeys(...)` — a new baseline key appeared. For section L this is the headline signal.
|
||||
|
||||
### 4.4 Section L — new rows for this fix
|
||||
|
||||
Tiers, unchanged from phase two: **COMPLETE** may change everything · **RESULTS-ONLY (RO)** may
|
||||
change only `baselines[<branch>].results` for tests that ran, and must never remove an entry, add a
|
||||
result for a test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Every L row starts from a **cold graph recorded on the named default branch only** — verify
|
||||
`keys=[<default>]` before branching. Restore branch state afterwards.
|
||||
|
||||
| # | Case | Target outcome |
|
||||
|---|---|---|
|
||||
| L1 | default `master`, record, `git switch -c feature-x`, `pest --tia`, zero changes | **all 25 replayed** (`w=0`), not `25 uncached`. The headline fix. |
|
||||
| L2 | as L1 but default `main` | still all replayed — regression guard, this already worked |
|
||||
| L3 | default `trunk`, then `develop` | replayed for both; the fix must not special-case two names |
|
||||
| L4 | L1, then a *second* new branch `feature-y` | replayed too — the toll must not return per branch |
|
||||
| L5 | L1 then edit `app/Services/Calculator.php` on `feature-x` | narrows to the 2 affected files (`CalculatorTest` + `AnnotationsTest`, which `covers` it); the other 17 replay |
|
||||
| L6 | L1 with `--tia --filtered` | filtered mode reads the fallback too (`testFilesToRerun`, `hasUnlocatedTestsToRerun`) → `No affected tests found`, zero delta |
|
||||
| L7 | L1 with `--tia --parallel` | fallback works in workers as well as the parent |
|
||||
| L8 | L1, then `pest --tia` twice on `feature-x` | idempotent; second run also `w=0` |
|
||||
| L9 | detached HEAD on a `master`-only graph | replays, and **no `main` key minted** — `keys` stays `[master]`. Bug (b). |
|
||||
| L10 | `pest()->tia()->defaultBranch('master')` in `tests/Pest.php`, repo default renamed away | config wins over autodetect |
|
||||
| L11 | config set to a nonexistent branch (`defaultBranch('nope')`) | degrades to a full run; no crash, no phantom key |
|
||||
| L12 | no remote at all (`git remote remove origin` if present) | still resolves (via `init.defaultBranch`) or degrades safely — must not throw |
|
||||
| L13 | branch name with a slash (`feature/x/y`) | replayed; no key-splitting bugs |
|
||||
| L14 | L1, then confirm writes | `feature-x` gets its **own** key; the `master` key is **not** written to (fallback is read-only) |
|
||||
| L15 | git worktree on a new branch (the issue's scenario) | replays from the default-branch baseline |
|
||||
| L16 | non-git dir, `pest --tia` | still `MissingDependency: The feature "Tia mode" requires "git".` — the soft resolver must not swallow it |
|
||||
| L17 | non-git dir, plain `pest` | runs normally; no baseline dir created |
|
||||
| L18 | count `git` subprocesses during one `--tia` run | default-branch resolution is cached, not one call per test. Probe by shimming `git` on `PATH` to a logging wrapper. |
|
||||
|
||||
L10–L11 need a `tests/Pest.php` edit — that file is tracked and **outside** the `tests app` reset
|
||||
scope in practice (it lives in `tests/`, so `git checkout -- tests` does restore it; verify with
|
||||
`git status` after).
|
||||
|
||||
For L16/L17, build a throwaway non-git project — and note the trap that burned phase two: a
|
||||
**symlinked** `vendor` makes Pest resolve the project root back to the playground (identical baseline
|
||||
hash), silently invalidating the test. Use a hardlinked copy:
|
||||
|
||||
```bash
|
||||
NG="$SP/nogit"; rm -rf "$NG"; mkdir -p "$NG"
|
||||
cp -R composer.json composer.lock phpunit.xml artisan tests app bootstrap config routes resources storage "$NG/"
|
||||
[ -f .env ] && cp .env "$NG/"
|
||||
cp -Rl vendor "$NG/vendor" || cp -R vendor "$NG/vendor"
|
||||
```
|
||||
|
||||
Confirm the baseline path differs (`nogit-<hash>`, not `laravel-4a455a95622ac0ec`), and delete both
|
||||
the temp project and its `~/.pest/tia/nogit-*` dir afterwards.
|
||||
|
||||
### 4.5 Re-run the phase-two matrix (A–K, 156 rows)
|
||||
|
||||
Re-run every row of `PLAN_PHASE_TWO.md` Part 2 against the fixed build. No row's status is trusted
|
||||
until re-measured — the fix touches `Graph`'s read path, which nearly every row exercises. Sections
|
||||
**A, B, H, I** are the load-bearing ones here (H is branch-key resolution; I is filtered mode; both
|
||||
consume the changed methods directly). **H6 and H7 must be re-derived from a cold graph** (Part 1.4).
|
||||
|
||||
Most rows batch cheaply — phase two ran C1–C20 in one call at roughly one line of output each. Use
|
||||
`oneline.php` for the sweep and `cmp.php` only to drill into anomalies.
|
||||
|
||||
### 4.6 Known pre-existing failures — do not report as regressions
|
||||
|
||||
| item | status |
|
||||
|---|---|
|
||||
| **G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests. `mergeWorkerReplayPartials()` takes `$result['time']` verbatim at `Tia.php:1451`, never routing through `resultTime()` as the sequential sites (1681, 1762) do. Assertions survive because workers replay those themselves. | Pre-existing (pre-fix had no preservation at all), **out of scope for phase three**. Report it as still-present; do not fix it unless Nuno asks. |
|
||||
| **C19** — `--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
|
||||
| **J11** — `--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
|
||||
| **J10** — `--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Reporting
|
||||
|
||||
Per row: **tier respected (yes/no)**, the **graph delta** under sentinel patching, and for any failure
|
||||
the **pre-fix contrast** so a regression is told apart from a pre-existing defect. For a pre-fix
|
||||
contrast you need `db70017c` (or `bfd5b756` for "before phase three") files swapped into the
|
||||
playground's `vendor/` — that is a sync, so **ask Nuno first** and always restore afterwards.
|
||||
|
||||
State which pest commit produced the numbers. Close with:
|
||||
|
||||
1. Whether L1–L18 all pass (the fix works).
|
||||
2. Whether A–K regressed anywhere relative to phase two's 154/156.
|
||||
3. The D1–D4 decisions as implemented.
|
||||
4. Anything still open — including G4, which will still be failing.
|
||||
|
||||
Leave the playground on `master` with only the four user-modified files dirty, no stray branches, and
|
||||
no leftover `~/.pest/tia/*` dirs beyond `laravel-4a455a95622ac0ec`.
|
||||
@@ -0,0 +1,466 @@
|
||||
# TIA write-tier conformance — phase two
|
||||
|
||||
## Your task
|
||||
|
||||
**Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app** at
|
||||
`/Users/nunomaduro/Work/projects/playground/laravel`. Every row, including the ones already marked
|
||||
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
|
||||
re-measured against the current code. The `Phase 1` column is prior evidence and a hint at what to
|
||||
watch, never a reason to skip a row.
|
||||
|
||||
These are **not** the `pestphp/pest` repo's own tests (`composer test`) — do not run those. Each row is
|
||||
a `pest` invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
|
||||
|
||||
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any
|
||||
failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do **not**
|
||||
stop at reading this file or summarising it; the deliverable is executed results.
|
||||
|
||||
Work in this order: **Part 1 (environment traps) → Part 1b (build the fixtures — the playground has
|
||||
none of them) → Part 2 (the matrix) → Part 3 (priorities)**. The matrix is too large for one context;
|
||||
take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are
|
||||
the load-bearing ones — do those first if you run short.
|
||||
|
||||
Phase one implemented `PLAN.md` Part 1 items 1–4 plus three §5 items. This file turns the matrix into
|
||||
a conformance check rather than a bug list.
|
||||
|
||||
Code under test: `pestphp/pest` at `/Users/nunomaduro/Work/projects/pestphp/pest`, branch
|
||||
`fix/tia-filtered`, commit **`bfd5b756`** or later. Verify with
|
||||
`grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php` → at least 3 hits. Pre-fix
|
||||
baseline for every contrast is **`db70017c`**.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — What phase one changed
|
||||
|
||||
| # | Change | Files |
|
||||
|---|---|---|
|
||||
| 1 | `hasUnlocatedTestsToRerun()` stats the file, so a deleted test file is "unlocated" | `src/Plugins/Tia/Graph.php` |
|
||||
| 2 | `enterReplayMode()` uses `activateLinkTracking()` under piggyback coverage | `src/Plugins/Tia.php` |
|
||||
| 3 | `enterReplayMode()` stamps `TIA_PIGGYBACK_COVERAGE` for workers | `src/Plugins/Tia.php` |
|
||||
| 4 | `replaceEdges(…, keepExisting:)` — piggyback edges seed empty sets, never overwrite populated ones | `Graph.php`, `Tia.php` |
|
||||
| 5 | `renderFreshGraph()` stops claiming "fresh graph" when the graph is kept; reason reworded to `recording a coverage baseline` | `Tia.php` |
|
||||
| 6 | `COVERAGE_REPORT_FLAGS` + `coverageReportActive()` union over `originalArguments`; new `pestCoverageActive()` keeps the coverage-cache marker/hijack on Pest's own `--coverage` | `Tia.php` |
|
||||
| 7 | `Tia::recordsEdgesInWorkers()` + `WrapperRunner::handleTia()` inject `-d pcov.directory=<root>` into worker argv | `Tia.php`, `src/Plugins/Parallel/Paratest/WrapperRunner.php` |
|
||||
| 8 | Sequential record runs announce structural drift via `renderFreshGraph()` | `Tia.php` |
|
||||
| 9 | `Graph::getTime()` + `cachedTimeByTestId` + `resultTime()` preserve replayed durations; edge-less write guard is now `$recordsEdges = $complete && ($markKnownTestFiles \|\| $this->recordingActive)` | `Graph.php`, `Tia.php` |
|
||||
|
||||
Deliberately **not** done: `PLAN.md` §5 SIGINT propagation, §5 warning/deprecation `status=0`
|
||||
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
|
||||
|
||||
### Target-outcome changes this forces
|
||||
|
||||
Two rows in the original matrix asserted the **old**, buggy behaviour. Their targets are updated
|
||||
below — do not report them as regressions:
|
||||
|
||||
- **B10** was "`time` differs; statuses stable". Change 9 means replayed entries now **keep** their
|
||||
recorded `time`. New target: `time` differs only for tests that actually executed.
|
||||
- **B12** gains an edge-preservation assertion it never had (see K1).
|
||||
|
||||
### One known-failing repo test
|
||||
|
||||
`tests/Unit/Plugins/Tia/Graph.php:69-76` asserts `hasUnlocatedTestsToRerun('main')` is `false` for
|
||||
`tests/Feature/FooTest.php` under `new Graph(sys_get_temp_dir())` — a path that does not exist. That
|
||||
assertion encodes the I9 bug and **will fail** under change 1. It needs re-pointing at a root/file
|
||||
that exists (e.g. `dirname(__DIR__, 4)` + `'tests/Unit/Plugins/Tia/Graph.php'`). Left untouched by
|
||||
request; it is a repo-test matter, not a playground one.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Environment, and the traps in it
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel`
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
|
||||
```
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON
|
||||
when it detects an agent.
|
||||
2. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock` —
|
||||
that a blanket reset would destroy. `PLAN.md`'s "reset ritual" is unsafe as written. Scope resets
|
||||
to what you touched: `git checkout -- tests app` / `git clean -fd tests app`, and for
|
||||
`composer.lock` (needed for the drift rows) **copy it aside and copy it back**, verifying with
|
||||
`shasum`.
|
||||
3. **`vendor/pestphp/pest` is a dist copy, not a symlink.** Composer installed
|
||||
`dev-fix/tia-filtered as 5.2.0`, so edits in the pest repo do **not** reach the playground.
|
||||
|
||||
**Tell Nuno whenever a sync is needed to move forward — do not sync silently.** Say what is stale
|
||||
and what the sync would be, then wait. This includes temporarily swapping in `db70017c` files for a
|
||||
before/after contrast. When you report any playground result, state which commit produced it.
|
||||
The sync itself, once he agrees:
|
||||
```bash
|
||||
PEST=/Users/nunomaduro/Work/projects/pestphp/pest
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \
|
||||
src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; done
|
||||
```
|
||||
Verify with `grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"` → `1`. **Always restore the
|
||||
current version before continuing** after a pre-fix contrast.
|
||||
4. **Coverage driver:** pcov only, no xdebug. `ini_get('pcov.directory')` is `''` by default — that
|
||||
emptiness is the entire mechanism behind G12.
|
||||
5. **Suite shape as found:** 7 tests in 5 files — `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,
|
||||
TrioTest}.php`, `tests/Feature/ExampleTest.php`. A healthy sequential graph is **`files=22`,
|
||||
5 edge keys, self-edge on every test** (`PLAN.md`'s `files=25` is stale). Pre-fix parallel gives
|
||||
`files=4` with zero self-edges. **These numbers shift the moment you add the Part 1b fixtures** —
|
||||
re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants
|
||||
that do *not* shift: sequential and parallel must agree, and every test must have a self-edge.
|
||||
6. **Sentinel patching is the only reliable discriminator** between "wrote identical values" and
|
||||
"wrote nothing": rewrite every cached result to `time=9.999 assertions=42`, snapshot, run the
|
||||
case, diff. A canary test absent from `edges` is unreliable under `--filter` because it never
|
||||
matches the filter and so never runs.
|
||||
7. Seeding a cached failure needs `--fresh` (or an env-driven flaky fixture): `--tia` on a clean
|
||||
green tree replays rather than executes, so it can never cache a failure. Working recipe — break
|
||||
an assertion, `pest --tia --fresh`, then restore the source.
|
||||
8. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
9. The shell is zsh — build commands with arrays or `eval`; unquoted `$args` does not word-split.
|
||||
|
||||
### Graph summariser
|
||||
|
||||
Write this to a scratch path and use it for every diff.
|
||||
|
||||
```php
|
||||
<?php // summarise.php <graph.json> [label]
|
||||
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
|
||||
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
|
||||
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
|
||||
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
|
||||
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
|
||||
ksort($edges);
|
||||
foreach ($edges as $test => $ids) {
|
||||
$self = 'no';
|
||||
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
|
||||
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
|
||||
}
|
||||
foreach ($g['baselines'] ?? [] as $branch => $b) {
|
||||
$r = $b['results'] ?? [];
|
||||
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
|
||||
foreach ($r as $id => $x) {
|
||||
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
|
||||
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Normalised edge-set equality (for G12 / I5 / K1):
|
||||
|
||||
```php
|
||||
<?php // edgediff.php <a.json> <b.json>
|
||||
function edges(string $p): array {
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
|
||||
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$a = edges($argv[1]); $b = edges($argv[2]);
|
||||
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
|
||||
foreach ($a as $t => $s) {
|
||||
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
|
||||
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
|
||||
}
|
||||
```
|
||||
|
||||
### Per-case loop
|
||||
|
||||
```bash
|
||||
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 1b — Fixtures you must build first
|
||||
|
||||
**The playground has none of the fixtures the matrix depends on.** Verified inventory: the only test
|
||||
files are `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php` and
|
||||
`tests/Feature/ExampleTest.php` — 7 plain tests, zero occurrences of `->group()`, `->only()`,
|
||||
`->skip()`, `->todo()`, `->note()`, `->flaky()`, `->covers()`, `->uses()`, `->issue()`, `->pr()`,
|
||||
`->ticket()`, `->assignee()`, datasets, or any `FLAKY_OK`-style env hook. `phpunit.xml` defines only
|
||||
the `Unit` and `Feature` testsuites.
|
||||
|
||||
So roughly 30 rows below cannot run as written until you create the fixtures. Build them **all up
|
||||
front, in one commit**, then re-derive the baseline graph shape once and use those numbers for the
|
||||
whole sweep — every fixture you add changes `files`, the `edges` key count and `n`, so adding them
|
||||
piecemeal invalidates earlier rows.
|
||||
|
||||
| Fixture to create | Rows that need it |
|
||||
|---|---|
|
||||
| a test with `->group('smoke')` | C7, C8, C9, C10 |
|
||||
| an env-driven flaky test (passes iff `FLAKY_OK=1`) | E1, E2, E3, E4 |
|
||||
| a `->skip()`ed test | E9, D8 |
|
||||
| a `->todo()` test | E10, C23, C39 |
|
||||
| a risky test (no assertions; pair with `--disallow-test-output`) | E11, D7 |
|
||||
| a test that triggers a PHPUnit warning | D6 |
|
||||
| a test calling `markTestIncomplete()` | D9 |
|
||||
| a test that triggers a deprecation | D10 |
|
||||
| a dataset test with ≥3 rows | E13, E14 |
|
||||
| `->only()` — added and removed per case, not left in | C31, C32, G7 |
|
||||
| `->covers(App\Services\Calculator::class)` | C18 |
|
||||
| `->uses(...)` / `UsesClass` annotation | C19 |
|
||||
| `->note(...)` | C24 |
|
||||
| `->flaky()` **annotation** (distinct from the env-driven flaky test above) | C25 |
|
||||
| `->issue(123)`, `->pr(1)`, `->ticket('X')`, `->assignee('X')` | C26, C27, C28, C37, C38 |
|
||||
|
||||
This is `PLAN.md` §6's "test-harness gaps to close before re-running", now itemised: without these,
|
||||
the listed rows' graph-invariant assertions match zero tests and **prove nothing** — they pass
|
||||
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous *only because* its fixture is
|
||||
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
|
||||
|
||||
Rows needing an *action* rather than a fixture — breaking `trio one` for the D rows, an uncommitted
|
||||
test edit for `--dirty`, branch renames and a non-git dir for H5–H10, `--mutate` against
|
||||
`app/Services` for the F rows — are fine as written; `pestphp/pest-plugin-mutate` is installed.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Phase-1 column: **VERIFIED** = re-run against the fixed code in the phase-one session, pre-fix
|
||||
contrast captured · **PASS (sweep)** = passed in the original `db70017` sweep and *not* re-checked
|
||||
since the changes — these are the bulk of phase two's work · **SKIP** = not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS (sweep) |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | VERIFIED |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | VERIFIED |
|
||||
| A4 | `git status` after reset ritual | clean *except the four user-modified files* (see trap 2) | VERIFIED |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | VERIFIED |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS (sweep) |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS (sweep) |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | VERIFIED |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | VERIFIED |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS (sweep) — recheck under change 9 |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS (sweep) — recheck under change 9 |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | VERIFIED |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS (sweep) |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS (sweep) |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS (sweep) |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS (sweep) |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS (sweep) |
|
||||
| B10 | `pest --tia` twice | **UPDATED TARGET:** `time` differs only for tests that executed; replayed entries keep their recorded `time`; statuses stable | VERIFIED |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key **and** new result appear in the **same** run | VERIFIED (regression guard for change 9) |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written; coverage report prints | VERIFIED |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS (sweep) |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS (sweep) |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS (sweep) |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS (sweep) |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS (sweep) |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS (sweep) |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS (sweep) |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS (sweep) |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS (sweep) |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS (sweep) |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS (sweep) |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS (sweep) |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS (sweep) |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS (sweep) |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS (sweep) |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS (sweep) |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS (sweep) |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS (sweep) |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS (sweep) |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS (sweep) |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS (sweep) |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS (sweep) |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS (sweep) |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS (sweep) |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS (sweep) |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS (sweep) |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS (sweep) |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS (sweep) |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS (sweep) |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS (sweep) |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS (sweep) |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS (sweep) |
|
||||
| D11 | `pest --tia --bail` | RO | PASS (sweep) |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS (sweep) |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS (sweep) |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS (sweep) |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS (sweep) |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS (sweep) |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran (it executed, so change 9 does not apply) | PASS (sweep) |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS (sweep) |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS (sweep) |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS (sweep) |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS (sweep) |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS (sweep) |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS (sweep) |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS (sweep) |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS (sweep) |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | VERIFIED |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | VERIFIED |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS (sweep) |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS (sweep) |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS (sweep) |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS (sweep) |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS (sweep) |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written; all 7 results present | VERIFIED |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS (sweep) |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS (sweep) |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS (sweep) |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | VERIFIED (see I5) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | VERIFIED |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS (sweep) |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS (sweep) |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS (sweep) |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS (sweep) |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | VERIFIED |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** — `files=22`, self-edge on all 5 tests, both | **VERIFIED (fixed)** — pre-fix: `files=4`, 0 self-edges |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | VERIFIED |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS (sweep) |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS (sweep) |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS (sweep) |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS (sweep) |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS (sweep) |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS (sweep) |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | VERIFIED |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS (sweep) |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **VERIFIED (fixed)** — 1 affected test, edges identical |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS (sweep) |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS (sweep) |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report → full suite runs | **VERIFIED (fixed)** — pre-fix: `No affected tests found` |
|
||||
| I9 | cached failure whose test file was deleted | WARN `Some cached tests due a re-run could not be located on disk` + `Running the full suite with replay instead of a filtered run` | **VERIFIED (fixed)** — pre-fix: `No tests found`, exit 0 green |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS (sweep) |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild, **and the reason is printed sequentially**: `fresh graph (composer.lock changed)` | **VERIFIED (fixed)** — pre-fix: bare `Running in TIA mode.` |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS (sweep) |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS (sweep) |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS (sweep) |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS (sweep) |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS (sweep) |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS (sweep) |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS (sweep) |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS (sweep) |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS (sweep) |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS (sweep) |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE (silent no-op without `--coverage`) | PASS (sweep) |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS (sweep) |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS (sweep) |
|
||||
|
||||
### K — New rows for the phase-one fixes
|
||||
|
||||
These assert behaviour no original row covered. All were verified in phase one; re-run them as
|
||||
regression guards.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| K1 | healthy `pest --tia` graph, then `pest --tia --coverage` | `edges` **byte-identical** — piggyback data may seed empty sets, never narrow populated ones; `files=22` stays `22` | **VERIFIED (fixed)** — pre-fix: `ExampleTest` 16→2 edges, self-edges lost |
|
||||
| K2 | same run's headline | `Experimental TIA mode enabled / recording a coverage baseline.` — no false `fresh graph` | **VERIFIED (fixed)** |
|
||||
| K3 | `pest --tia` ×3 on a clean tree | replayed entries keep their recorded `time` across all three | **VERIFIED (fixed)** — pre-fix: `0.053 → 0.001 → 0.001 → 0.001` |
|
||||
| K4 | add a test file, one `pest --tia` (graph exists, fingerprint matches → replay+refresh) | result **and** edges appear in that same run; `n` grows by 1 | **VERIFIED** — regression guard for change 9 |
|
||||
| K5 | add a test file, plain `pest --no-tia` | no result written for it; `n` unchanged; no edges | **VERIFIED (fixed)** — pre-fix: edge-less result inflated `n` |
|
||||
| K6 | `pest --tia --parallel` worker argv | carries `-d pcov.directory=<projectRoot>`; `PEST_TIA` unset; not re-exec'd | **VERIFIED** — probe `bin/worker.php` |
|
||||
| K7 | second `pest --tia --coverage` (cache primed → replay) | recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — **new work** |
|
||||
| K8 | `pest --tia --parallel --coverage` | workers read `TIA_PIGGYBACK_COVERAGE`; no widened pcov scope; report intact | Not yet measured — **new work** |
|
||||
| K9 | `pest --tia --filtered --coverage-html=<dir>` / `--coverage-clover=<file>` | filtered mode off, same as I8, for every flag in `COVERAGE_REPORT_FLAGS` | Not yet measured — **new work** |
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Priorities for phase two
|
||||
|
||||
0. **Part 1b fixtures** — nothing in C, D6–D10, or E9–E14 means anything until they exist.
|
||||
1. **K7, K8, K9** — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
|
||||
reasoned about but not observed; K9 covers the seven `COVERAGE_REPORT_FLAGS` beyond
|
||||
`--coverage-text`.
|
||||
2. **B2, B3, F4** and all of **D** and **E** — change 9 touched the shared result-write path, and
|
||||
these are the rows that exercise it hardest. `$recordsEdges` is the thing to falsify: it must be
|
||||
false for every partial and every non-recording run.
|
||||
3. **F3–F8** — change 7 injects a `-d` into worker argv; `--mutate --parallel` is the one place that
|
||||
both spawns workers and must write nothing.
|
||||
4. **H1–H10** — untouched by these changes; cheapest bulk confirmation.
|
||||
5. Raw PHPUnit coverage flags print **no report at all** in Pest, with or without TIA
|
||||
(`--no-tia --coverage-text` is equally silent). Pre-existing, unrelated to change 6 — do not
|
||||
chase it as a regression, but it means I8/K9 can only assert the selection half.
|
||||
|
||||
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
|
||||
failure the pre-fix contrast (write `git show db70017c:<file>` into vendor, re-run, restore) so a
|
||||
regression is told apart from a pre-existing defect.
|
||||
@@ -48,6 +48,7 @@
|
||||
"Tests\\Fixtures\\Covers\\": "tests/Fixtures/Covers",
|
||||
"Tests\\Fixtures\\Inheritance\\": "tests/Fixtures/Inheritance",
|
||||
"Tests\\Fixtures\\Arch\\": "tests/Fixtures/Arch",
|
||||
"Tests\\Fixtures\\Tia\\": "tests/Fixtures/Tia",
|
||||
"Tests\\": "tests/PHPUnit/"
|
||||
},
|
||||
"classmap": [
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
<exclude>./tests/.snapshots</exclude>
|
||||
<exclude>./tests/Fixtures/Inheritance</exclude>
|
||||
<exclude>./tests/Fixtures/Suites</exclude>
|
||||
<exclude>./tests/Fixtures/Tia</exclude>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
<source>
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Exceptions;
|
||||
|
||||
use NunoMaduro\Collision\Contracts\RenderlessEditor;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessTrace;
|
||||
use Pest\Contracts\Panicable;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Console\Exception\ExceptionInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class TiaRequiresRemote extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
|
||||
{
|
||||
public function __construct()
|
||||
{
|
||||
parent::__construct(
|
||||
'Tia mode requires a repository with a remote, so the default branch every other branch falls back to reading can be resolved.',
|
||||
);
|
||||
}
|
||||
|
||||
public function render(OutputInterface $output): void
|
||||
{
|
||||
$output->writeln([
|
||||
'',
|
||||
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires a repository with a remote.',
|
||||
'',
|
||||
' Without one there is no way to tell which branch is the default — the branch',
|
||||
' whose baseline every other branch falls back to reading — so the first run on',
|
||||
' each new branch would silently re-run the whole suite.',
|
||||
'',
|
||||
' Add a remote, or name the branch yourself in <fg=yellow>tests/Pest.php</>:',
|
||||
'',
|
||||
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
|
||||
'',
|
||||
]);
|
||||
}
|
||||
|
||||
public function exitCode(): int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
+51
-7
@@ -9,8 +9,10 @@ use Pest\Contracts\Plugins\AddsOutput;
|
||||
use Pest\Contracts\Plugins\HandlesArguments;
|
||||
use Pest\Contracts\Plugins\HandlesOriginalArguments;
|
||||
use Pest\Contracts\Plugins\Terminable;
|
||||
use Pest\Exceptions\InvalidOption;
|
||||
use Pest\Exceptions\MissingDependency;
|
||||
use Pest\Exceptions\NoAffectedTestsFound;
|
||||
use Pest\Exceptions\TiaRequiresRemote;
|
||||
use Pest\Exceptions\TiaRequiresRepositoryRoot;
|
||||
use Pest\Panic;
|
||||
use Pest\Plugins\Concerns\HandleArguments;
|
||||
@@ -183,6 +185,19 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
|
||||
];
|
||||
|
||||
/**
|
||||
* Options that cannot be combined with Tia mode.
|
||||
*
|
||||
* `--covers` and `--uses` select on coverage metadata Tia does not model,
|
||||
* so they resolve to no tests at all rather than to the ones the user meant.
|
||||
* `--random-order-seed` exits non-zero on its own, with or without Tia.
|
||||
* Either way the run cannot honour both things it was asked for, so it says
|
||||
* so instead of silently dropping Tia and running something else.
|
||||
*/
|
||||
private const array UNSUPPORTED_OPTIONS = [
|
||||
'--covers', '--uses', '--random-order-seed',
|
||||
];
|
||||
|
||||
private bool $graphWritten = false;
|
||||
|
||||
private bool $replayRan = false;
|
||||
@@ -506,6 +521,10 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$cliEnabled = $this->hasArgument(self::OPTION, $arguments) || self::envFlagEnabled(self::ENV_TIA);
|
||||
$alwaysEnabled = $watchPatterns->isEnabled()
|
||||
&& (! $watchPatterns->isLocally() || Environment::name() === Environment::LOCAL);
|
||||
if (! $isWorker && ! $disabled && ($cliEnabled || $alwaysEnabled)) {
|
||||
$this->guardUnsupportedOptions($arguments);
|
||||
}
|
||||
|
||||
$hasExplicitPath = $this->hasExplicitPathArgument($arguments);
|
||||
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
|
||||
$disabled = $disabled || $partial;
|
||||
@@ -869,6 +888,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
$this->resolveBranch($projectRoot);
|
||||
|
||||
// After resolveBranch(), so a directory that is no repository at all
|
||||
// still reports the missing git dependency rather than a missing remote.
|
||||
// Skipped once the default branch is configured by hand: there is then
|
||||
// nothing left for a remote to answer.
|
||||
if ($this->watchPatterns->defaultBranch() === null && ! new ChangedFiles($projectRoot)->hasRemote()) {
|
||||
Panic::with(new TiaRequiresRemote);
|
||||
}
|
||||
|
||||
$fingerprint = Fingerprint::compute($projectRoot);
|
||||
$this->startFingerprint = $fingerprint;
|
||||
|
||||
@@ -1870,13 +1897,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return true;
|
||||
}
|
||||
|
||||
foreach (self::COVERAGE_REPORT_FLAGS as $flag) {
|
||||
if ($this->hasArgument($flag, $this->originalArguments)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
return array_any(self::COVERAGE_REPORT_FLAGS, fn (string $flag): bool => $this->hasArgument($flag, $this->originalArguments));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1891,6 +1912,29 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $coverage->coverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* Panics when the run asks for Tia alongside an option Tia cannot honour.
|
||||
*
|
||||
* Checked against the original argv as well, because `bin/pest` consumes
|
||||
* some of these itself before PHPUnit ever sees them.
|
||||
*
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function guardUnsupportedOptions(array $arguments): void
|
||||
{
|
||||
foreach (self::UNSUPPORTED_OPTIONS as $option) {
|
||||
if (! $this->hasArgument($option, $arguments) && ! $this->hasArgument($option, $this->originalArguments)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
Panic::with(new InvalidOption(sprintf(
|
||||
'The [%s] option cannot be combined with [%s].',
|
||||
$option,
|
||||
self::OPTION,
|
||||
)));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a selection-narrowing flag was given, either among the arguments
|
||||
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
|
||||
|
||||
@@ -244,6 +244,17 @@ final readonly class ChangedFiles
|
||||
return $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the repository has any remote configured.
|
||||
*
|
||||
* Advisory like {@see self::defaultBranch()} — a `git` that cannot answer
|
||||
* is reported as "no remote", and the caller decides what that means.
|
||||
*/
|
||||
public function hasRemote(): bool
|
||||
{
|
||||
return $this->gitOutput(['git', 'remote']) !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $command
|
||||
*/
|
||||
|
||||
@@ -1568,6 +1568,34 @@
|
||||
PASS Tests\Features\Tia
|
||||
✓ it does not run user hooks when replaying cached skipped and incomplete results
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchReplay
|
||||
✓ replays the default branch baseline on a new branch
|
||||
✓ replays whatever the default branch is called with ('main')
|
||||
✓ replays whatever the default branch is called with ('master')
|
||||
✓ replays whatever the default branch is called with ('trunk')
|
||||
✓ replays whatever the default branch is called with ('develop')
|
||||
✓ replays on a second new branch too
|
||||
✓ writes nothing on a second run on the same branch
|
||||
✓ replays on a branch whose name contains slashes
|
||||
✓ replays inside a worktree on a new branch
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchResolution
|
||||
✓ a declared default branch beats autodetection
|
||||
✓ a declared default branch that does not exist degrades to a full run
|
||||
✓ a repository with no remote is refused rather than silently re-run
|
||||
✓ a declared default branch stands in for a missing remote
|
||||
✓ tia still requires git
|
||||
✓ a plain run outside a repository creates no baseline
|
||||
✓ the default branch is resolved once per run, not once per test
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchWriteTier
|
||||
✓ narrows to the affected tests on a new branch
|
||||
✓ filtered mode reads the fallback too
|
||||
✓ filtered mode finds nothing to do on a clean green feature branch
|
||||
✓ a detached HEAD replays without minting a branch key
|
||||
✓ the branch that ran gets its own key and the default branch keeps its baseline
|
||||
✓ the fallback reaches parallel workers
|
||||
|
||||
PASS Tests\Features\Ticket
|
||||
✓ it may be associated with an ticket #1, #2
|
||||
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
|
||||
@@ -2197,4 +2225,4 @@
|
||||
✓ pass with dataset with ('my-datas-set-value')
|
||||
✓ within describe → pass with dataset with ('my-datas-set-value')
|
||||
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1549 passed (3389 assertions)
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1571 passed (3446 assertions)
|
||||
@@ -0,0 +1,108 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* Reading a baseline recorded on another branch.
|
||||
*
|
||||
* Without the default-branch fallback, the first `--tia` run on every new branch
|
||||
* re-runs a whole suite whose results the default branch already holds — once
|
||||
* per branch, forever, on any repository not named `main`.
|
||||
*
|
||||
* @see https://github.com/pestphp/pest/issues/1823
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('replays the default branch baseline on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($result->exitCode)->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays whatever the default branch is called', function (string $defaultBranch): void {
|
||||
$project = Project::make($defaultBranch);
|
||||
$project->seed($defaultBranch);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->with(['main', 'master', 'trunk', 'develop'])->skipOnWindows();
|
||||
|
||||
test('replays on a second new branch too', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->switchTo('feature-y', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The toll is one full run per new branch. It must not come back for the
|
||||
// second branch either.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('writes nothing on a second run on the same branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->snapshot();
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays on a branch whose name contains slashes', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature/x/y', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toContain('feature/x/y');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays inside a worktree on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$worktree = $project->worktree('feature-worktree');
|
||||
|
||||
// Seeded against the worktree rather than the main checkout: a worktree's
|
||||
// `.git` is a file, so `Storage::originIdentity()` cannot read the remote
|
||||
// from it and the worktree resolves a storage key of its own. That gap is
|
||||
// separate from the branch fallback, and it is the fallback this row is
|
||||
// about — the worktree is checked out on a branch the baseline does not
|
||||
// name, which is the scenario from the issue.
|
||||
$project->seedFor($worktree, 'master');
|
||||
|
||||
$result = $project->pestIn($worktree, '--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,131 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Symfony\Component\Process\ExecutableFinder;
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* How the default branch gets named: declared in `tests/Pest.php`, autodetected
|
||||
* from the repository, or not answerable at all.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a declared default branch beats autodetection', function (): void {
|
||||
// The repository autodetects `develop`, which holds no baseline. Only the
|
||||
// declaration in `tests/Pest.php` can reach the `master` one.
|
||||
$project = Project::make('develop', overlay: 'configured-default-branch');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a declared default branch that does not exist degrades to a full run', function (): void {
|
||||
$project = Project::make('master', overlay: 'unknown-default-branch');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing to fall back to, so everything runs — and no baseline is minted
|
||||
// under the name that resolved to nothing.
|
||||
expect($result->uncached())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'feature-x']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a repository with no remote is refused rather than silently re-run', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing can name the default branch, so every new branch would re-run the
|
||||
// whole suite with no explanation. Saying so beats doing that quietly.
|
||||
expect($result->output)->toContain('Tia mode requires a repository with a remote.')
|
||||
->and($result->exitCode)->toBe(1, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a declared default branch stands in for a missing remote', function (): void {
|
||||
// The escape hatch the refusal above points at: with the branch named by
|
||||
// hand there is nothing left for a remote to answer.
|
||||
$project = Project::make('master', overlay: 'configured-default-branch');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('tia still requires git', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The soft default-branch resolver runs before the branch is named, and it
|
||||
// must not swallow this.
|
||||
expect($result->output)->toContain('The feature "Tia mode" requires "git".')
|
||||
->and($result->exitCode)->not->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a plain run outside a repository creates no baseline', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
|
||||
$result = $project->pest();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($project->graphExists())->toBeFalse()
|
||||
->and($project->graphDir())->not->toBeDirectory();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the default branch is resolved once per run, not once per test', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$git = (new ExecutableFinder)->find('git');
|
||||
|
||||
expect($git)->not->toBeNull();
|
||||
|
||||
// A `git` first on `PATH` that records what it was asked before handing over
|
||||
// to the real one.
|
||||
$log = $project->path('git-calls.log');
|
||||
$project->write('shim/git', implode("\n", [
|
||||
'#!/bin/sh',
|
||||
'echo "$@" >> '.escapeshellarg($log),
|
||||
'exec '.escapeshellarg((string) $git).' "$@"',
|
||||
'',
|
||||
]));
|
||||
chmod($project->path('shim/git'), 0755);
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), [
|
||||
'PATH' => $project->path('shim').':'.getenv('PATH'),
|
||||
], '--tia');
|
||||
|
||||
$calls = file_exists($log) ? explode("\n", trim((string) file_get_contents($log))) : [];
|
||||
$resolutions = array_filter($calls, fn (string $call): bool => str_contains($call, 'symbolic-ref')
|
||||
|| str_contains($call, 'init.defaultBranch'));
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($calls)->not->toBeEmpty('the git shim was never reached')
|
||||
->and($resolutions)->toHaveCount(1, implode("\n", $calls));
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,133 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* What the fallback is allowed to touch.
|
||||
*
|
||||
* Reading another branch's baseline must stay a read: writes belong to the
|
||||
* branch that ran, and a branch that has no name of its own must not mint one.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('narrows to the affected tests on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
// Semantic, not cosmetic: PHP is hashed at the AST level, so a comment
|
||||
// would not register as a change at all.
|
||||
$project->write('app/Calculator.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Calculator
|
||||
{
|
||||
public function add(int $a, int $b): int
|
||||
{
|
||||
return $a + $b;
|
||||
}
|
||||
|
||||
public function subtract(int $a, int $b): int
|
||||
{
|
||||
return $a - $b;
|
||||
}
|
||||
|
||||
public function multiply(int $a, int $b): int
|
||||
{
|
||||
return $a * $b;
|
||||
}
|
||||
}
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Two of the three test files cover `Calculator`; the third replays from the
|
||||
// default branch's baseline.
|
||||
expect($result->affected())->toBe(4, $result->describe())
|
||||
->and($result->replayed())->toBe(2, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode reads the fallback too', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
// A failure cached on the default branch. Filtered mode asks the graph which
|
||||
// test files are due a re-run, and that read has to reach the fallback as
|
||||
// well: without it a new branch sees nothing to do, reports green, and never
|
||||
// re-runs the failure — on every subsequent invocation.
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
// Selection is by file, so the failure's two siblings in `CalculatorTest`
|
||||
// come along; the other two test files stay out of the run entirely.
|
||||
expect($result->output)->toContain('from 1 previously unsuccessful test')
|
||||
->and($result->output)->not->toContain('No affected tests found')
|
||||
->and($result->tally())->toContain('2 passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode finds nothing to do on a clean green feature branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('No affected tests found')
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a detached HEAD replays without minting a branch key', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// A detached HEAD has no branch of its own. The default branch is the
|
||||
// honest key, and no phantom one appears beside it.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the branch that ran gets its own key and the default branch keeps its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'feature-x'])
|
||||
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the fallback reaches parallel workers', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--parallel', '--processes=2');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,170 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A git repository the scenario tests drive.
|
||||
*
|
||||
* Hermetic by construction: every invocation neutralises the machine's global
|
||||
* and system config and carries its own committer identity. Without that, an
|
||||
* ambient `init.defaultBranch = main` would answer questions the scenario means
|
||||
* to leave unanswered, and rows like "resolves to nothing, degrades safely"
|
||||
* would pass by accident.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GitRepo
|
||||
{
|
||||
/**
|
||||
* @var array<string, string>
|
||||
*/
|
||||
public const array ENV = [
|
||||
'GIT_CONFIG_GLOBAL' => '/dev/null',
|
||||
'GIT_CONFIG_SYSTEM' => '/dev/null',
|
||||
// `GIT_CONFIG_SYSTEM` does not cover every system-level file git reads:
|
||||
// Apple's git also loads one from inside Xcode, and it sets
|
||||
// `init.defaultBranch`. Only this suppresses all of them.
|
||||
'GIT_CONFIG_NOSYSTEM' => '1',
|
||||
'GIT_AUTHOR_NAME' => 'Pest Fixture',
|
||||
'GIT_AUTHOR_EMAIL' => 'fixture@pestphp.io',
|
||||
'GIT_COMMITTER_NAME' => 'Pest Fixture',
|
||||
'GIT_COMMITTER_EMAIL' => 'fixture@pestphp.io',
|
||||
];
|
||||
|
||||
public function __construct(public string $path) {}
|
||||
|
||||
/**
|
||||
* Initialises the repository on `$branch` and commits everything in it.
|
||||
*/
|
||||
public function init(string $branch = 'master'): void
|
||||
{
|
||||
$this->run(['init', '--quiet']);
|
||||
$this->run(['checkout', '--quiet', '-b', $branch]);
|
||||
$this->commit('Initial commit');
|
||||
}
|
||||
|
||||
public function commit(string $message): void
|
||||
{
|
||||
$this->run(['add', '-A']);
|
||||
$this->run(['commit', '--quiet', '--allow-empty', '-m', $message]);
|
||||
}
|
||||
|
||||
public function switchTo(string $branch, bool $new = false): void
|
||||
{
|
||||
$this->run($new ? ['checkout', '--quiet', '-b', $branch] : ['checkout', '--quiet', $branch]);
|
||||
}
|
||||
|
||||
public function rename(string $from, string $to): void
|
||||
{
|
||||
$this->run(['branch', '-m', $from, $to]);
|
||||
}
|
||||
|
||||
public function detach(): void
|
||||
{
|
||||
$this->run(['checkout', '--quiet', '--detach']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers an `origin`, which also decides the graph's storage key:
|
||||
* {@see Storage::projectKey()} prefers the remote's
|
||||
* identity over the path, so two checkouts of one repository — a worktree,
|
||||
* say — share a single graph.
|
||||
*/
|
||||
public function addOrigin(string $url = 'git@github.com:pestphp/tia-fixture.git'): void
|
||||
{
|
||||
$this->run(['remote', 'add', 'origin', $url]);
|
||||
}
|
||||
|
||||
public function removeOrigin(): void
|
||||
{
|
||||
$this->run(['remote', 'remove', 'origin']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Points `refs/remotes/origin/HEAD` at a local branch — what
|
||||
* `git remote set-head` would write, without a remote to talk to.
|
||||
*/
|
||||
public function setOriginHead(string $branch): void
|
||||
{
|
||||
$this->run(['update-ref', 'refs/remotes/origin/'.$branch, 'HEAD']);
|
||||
$this->run(['symbolic-ref', 'refs/remotes/origin/HEAD', 'refs/remotes/origin/'.$branch]);
|
||||
}
|
||||
|
||||
public function config(string $key, string $value): void
|
||||
{
|
||||
$this->run(['config', '--local', $key, $value]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch and returns its path.
|
||||
*/
|
||||
public function worktree(string $path, string $branch): string
|
||||
{
|
||||
$this->run(['worktree', 'add', '--quiet', '-b', $branch, $path]);
|
||||
|
||||
return $path;
|
||||
}
|
||||
|
||||
public function sha(): string
|
||||
{
|
||||
return $this->output(['rev-parse', 'HEAD']);
|
||||
}
|
||||
|
||||
public function currentBranch(): string
|
||||
{
|
||||
return $this->output(['rev-parse', '--abbrev-ref', 'HEAD']);
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchNames(): array
|
||||
{
|
||||
$names = explode("\n", $this->output(['for-each-ref', '--format=%(refname:short)', 'refs/heads']));
|
||||
|
||||
return array_values(array_filter($names, fn (string $name): bool => $name !== ''));
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function output(array $arguments): string
|
||||
{
|
||||
return trim($this->process($arguments, mustSucceed: true)->getOutput());
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function run(array $arguments): void
|
||||
{
|
||||
$this->process($arguments, mustSucceed: true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function process(array $arguments, bool $mustSucceed): Process
|
||||
{
|
||||
$process = new Process(['git', ...$arguments], $this->path, self::ENV);
|
||||
$process->setTimeout(30.0);
|
||||
$process->run();
|
||||
|
||||
if ($mustSucceed && ! $process->isSuccessful()) {
|
||||
throw new RuntimeException(sprintf(
|
||||
"git %s failed in [%s]:\n%s",
|
||||
implode(' ', $arguments),
|
||||
$this->path,
|
||||
$process->getErrorOutput().$process->getOutput(),
|
||||
));
|
||||
}
|
||||
|
||||
return $process;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,339 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* What one run did to the graph.
|
||||
*
|
||||
* The three tiers a run may respect, from the conformance matrix:
|
||||
*
|
||||
* - COMPLETE — may change everything.
|
||||
* - RESULTS-ONLY — may change only `baselines[<branch>].results` for tests that
|
||||
* actually ran. It may never remove an entry, nor move `sha`, `tree`,
|
||||
* `edges`, `files` or `fingerprint`.
|
||||
* - HARD-SUPPRESSED — may change nothing at all.
|
||||
*
|
||||
* {@see self::writtenCount()} is the load-bearing measurement, and the reason
|
||||
* {@see Project::sentinel()} exists: without falsified cached values there is no
|
||||
* way to tell "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GraphDelta
|
||||
{
|
||||
/**
|
||||
* @param array<string, mixed>|null $before
|
||||
* @param array<string, mixed>|null $after
|
||||
*/
|
||||
public function __construct(
|
||||
private ?array $before,
|
||||
private ?array $after,
|
||||
) {}
|
||||
|
||||
public function graphWasCreated(): bool
|
||||
{
|
||||
return $this->before === null && $this->after !== null;
|
||||
}
|
||||
|
||||
public function graphWasDeleted(): bool
|
||||
{
|
||||
return $this->before !== null && $this->after === null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries whose stored values actually moved.
|
||||
*/
|
||||
public function writtenCount(): int
|
||||
{
|
||||
$written = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$before = $this->results($this->before, $branch);
|
||||
$after = $this->results($this->after, $branch);
|
||||
|
||||
foreach ($before as $testId => $entry) {
|
||||
if (! isset($after[$testId])) {
|
||||
continue;
|
||||
}
|
||||
|
||||
foreach (['status', 'time', 'assertions', 'message'] as $field) {
|
||||
if (($entry[$field] ?? null) !== ($after[$testId][$field] ?? null)) {
|
||||
$written++;
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return $written;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that appeared.
|
||||
*/
|
||||
public function added(): int
|
||||
{
|
||||
$added = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$added += count(array_diff(
|
||||
array_keys($this->results($this->after, $branch)),
|
||||
array_keys($this->results($this->before, $branch)),
|
||||
));
|
||||
}
|
||||
|
||||
return $added;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that were pruned.
|
||||
*/
|
||||
public function removed(): int
|
||||
{
|
||||
$removed = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$removed += count(array_diff(
|
||||
array_keys($this->results($this->before, $branch)),
|
||||
array_keys($this->results($this->after, $branch)),
|
||||
));
|
||||
}
|
||||
|
||||
return $removed;
|
||||
}
|
||||
|
||||
/**
|
||||
* The baseline keys after the run — the headline signal for the
|
||||
* default-branch rows, where a phantom key is the defect.
|
||||
*
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeys(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->after));
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeysBefore(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->before));
|
||||
}
|
||||
|
||||
public function branchKeysMoved(): bool
|
||||
{
|
||||
return $this->branchKeysBefore() !== $this->branchKeys();
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the named branch's baseline is byte-identical — how a row proves
|
||||
* the fallback is read-only.
|
||||
*/
|
||||
public function baselineUntouched(string $branch): bool
|
||||
{
|
||||
return ($this->baselines($this->before)[$branch] ?? null)
|
||||
=== ($this->baselines($this->after)[$branch] ?? null);
|
||||
}
|
||||
|
||||
public function shaMoved(): bool
|
||||
{
|
||||
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'sha', $this->before) !== $this->baselineField($branch, 'sha', $this->after));
|
||||
}
|
||||
|
||||
public function treeMoved(): bool
|
||||
{
|
||||
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'tree', $this->before) !== $this->baselineField($branch, 'tree', $this->after));
|
||||
}
|
||||
|
||||
/**
|
||||
* Compares edges by the file paths they resolve to, not by file id: ids are
|
||||
* an implementation detail that shifts whenever `files` is rebuilt in a
|
||||
* different order.
|
||||
*/
|
||||
public function edgesMoved(): bool
|
||||
{
|
||||
return $this->edgeSets($this->before) !== $this->edgeSets($this->after);
|
||||
}
|
||||
|
||||
public function filesMoved(): bool
|
||||
{
|
||||
return $this->section($this->before, 'files') !== $this->section($this->after, 'files');
|
||||
}
|
||||
|
||||
public function fingerprintMoved(): bool
|
||||
{
|
||||
return $this->section($this->before, 'fingerprint') !== $this->section($this->after, 'fingerprint');
|
||||
}
|
||||
|
||||
public function structureMoved(): bool
|
||||
{
|
||||
if ($this->edgesMoved()) {
|
||||
return true;
|
||||
}
|
||||
if ($this->filesMoved()) {
|
||||
return true;
|
||||
}
|
||||
if ($this->fingerprintMoved()) {
|
||||
return true;
|
||||
}
|
||||
|
||||
return $this->branchKeysMoved();
|
||||
}
|
||||
|
||||
/**
|
||||
* Nothing moved at all.
|
||||
*/
|
||||
public function isHardSuppressed(): bool
|
||||
{
|
||||
return $this->before === $this->after;
|
||||
}
|
||||
|
||||
/**
|
||||
* Results may have moved for tests that ran; nothing structural did.
|
||||
*/
|
||||
public function isResultsOnly(): bool
|
||||
{
|
||||
return ! $this->graphWasCreated()
|
||||
&& ! $this->graphWasDeleted()
|
||||
&& ! $this->structureMoved()
|
||||
&& ! $this->shaMoved()
|
||||
&& ! $this->treeMoved()
|
||||
&& $this->removed() === 0
|
||||
&& $this->added() === 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* A one-line verdict, for failure messages.
|
||||
*/
|
||||
public function summary(): string
|
||||
{
|
||||
if ($this->graphWasCreated()) {
|
||||
return 'graph created';
|
||||
}
|
||||
|
||||
if ($this->graphWasDeleted()) {
|
||||
return 'graph deleted';
|
||||
}
|
||||
|
||||
$moved = [];
|
||||
|
||||
foreach (['edges', 'files', 'fingerprint'] as $section) {
|
||||
if ($this->{$section.'Moved'}()) {
|
||||
$moved[] = $section;
|
||||
}
|
||||
}
|
||||
|
||||
if ($this->branchKeysMoved()) {
|
||||
$moved[] = sprintf(
|
||||
'branchkeys(%s->%s)',
|
||||
implode('|', $this->branchKeysBefore()),
|
||||
implode('|', $this->branchKeys()),
|
||||
);
|
||||
}
|
||||
|
||||
return sprintf(
|
||||
'w=%d +%d -%d %s%s%s',
|
||||
$this->writtenCount(),
|
||||
$this->added(),
|
||||
$this->removed(),
|
||||
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
|
||||
$this->shaMoved() ? ' sha:changed' : '',
|
||||
$this->treeMoved() ? ' tree:changed' : '',
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, array<int, string>>
|
||||
*/
|
||||
private function edgeSets(?array $graph): array
|
||||
{
|
||||
$files = $this->section($graph, 'files');
|
||||
$sets = [];
|
||||
|
||||
foreach ($this->section($graph, 'edges') as $test => $ids) {
|
||||
if (! is_string($test)) {
|
||||
continue;
|
||||
}
|
||||
if (! is_array($ids)) {
|
||||
continue;
|
||||
}
|
||||
$paths = array_map(
|
||||
fn (mixed $id): string => is_int($id) && isset($files[$id]) && is_string($files[$id])
|
||||
? $files[$id]
|
||||
: '?'.json_encode($id),
|
||||
$ids,
|
||||
);
|
||||
|
||||
sort($paths);
|
||||
$sets[$test] = $paths;
|
||||
}
|
||||
|
||||
ksort($sets);
|
||||
|
||||
return $sets;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<mixed>
|
||||
*/
|
||||
private function section(?array $graph, string $key): array
|
||||
{
|
||||
$section = $graph[$key] ?? null;
|
||||
|
||||
return is_array($section) ? $section : [];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function baselines(?array $graph): array
|
||||
{
|
||||
$baselines = [];
|
||||
|
||||
foreach ($this->section($graph, 'baselines') as $branch => $baseline) {
|
||||
if (is_string($branch)) {
|
||||
$baselines[$branch] = $baseline;
|
||||
}
|
||||
}
|
||||
|
||||
return $baselines;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, array<string, mixed>>
|
||||
*/
|
||||
private function results(?array $graph, string $branch): array
|
||||
{
|
||||
$results = $this->baselines($graph)[$branch]['results'] ?? null;
|
||||
|
||||
if (! is_array($results)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$entries = [];
|
||||
|
||||
foreach ($results as $testId => $entry) {
|
||||
if (is_string($testId) && is_array($entry)) {
|
||||
$entries[$testId] = $entry;
|
||||
}
|
||||
}
|
||||
|
||||
return $entries;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
*/
|
||||
private function baselineField(string $branch, string $field, ?array $graph): mixed
|
||||
{
|
||||
return $this->baselines($graph)[$branch][$field] ?? null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* The outcome of one `pest` invocation against a fixture project.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class PestResult
|
||||
{
|
||||
/**
|
||||
* The run's output, with the terminal's escape sequences taken back out.
|
||||
*/
|
||||
public string $output;
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function __construct(
|
||||
public array $arguments,
|
||||
string $output,
|
||||
public int $exitCode,
|
||||
) {
|
||||
$this->output = (string) preg_replace([
|
||||
'#\x1b[[][^A-Za-z]*[A-Za-z]#', // colours, cursor moves
|
||||
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#', // hyperlinks
|
||||
], '', $output);
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests whose cached result was replayed instead of executed.
|
||||
*/
|
||||
public function replayed(): int
|
||||
{
|
||||
return $this->recapFragment('replayed');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because the graph held nothing for them — the count that
|
||||
* betrays a fallback which never resolved.
|
||||
*/
|
||||
public function uncached(): int
|
||||
{
|
||||
return $this->recapFragment('uncached');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because a file they depend on changed.
|
||||
*/
|
||||
public function affected(): int
|
||||
{
|
||||
return $this->recapFragment('affected');
|
||||
}
|
||||
|
||||
/**
|
||||
* The `Tests:` summary line, without its label or leading whitespace.
|
||||
*/
|
||||
public function tally(): string
|
||||
{
|
||||
if (preg_match('/^\s*Tests:\s+(.+)$/m', $this->output, $matches) !== 1) {
|
||||
return '';
|
||||
}
|
||||
|
||||
return trim($matches[1]);
|
||||
}
|
||||
|
||||
public function contains(string $needle): bool
|
||||
{
|
||||
return str_contains($this->output, $needle);
|
||||
}
|
||||
|
||||
/**
|
||||
* A description of the run, for failure messages that would otherwise say
|
||||
* only that 0 !== 6.
|
||||
*/
|
||||
public function describe(): string
|
||||
{
|
||||
return sprintf(
|
||||
"pest %s exited %d:\n%s",
|
||||
implode(' ', $this->arguments),
|
||||
$this->exitCode,
|
||||
$this->output,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read off the `Tests:` line rather than the whole output: the TIA headline
|
||||
* counts affected *files*, and matching that instead would be a quietly
|
||||
* wrong number.
|
||||
*/
|
||||
private function recapFragment(string $label): int
|
||||
{
|
||||
if (preg_match('/(\d+) '.preg_quote($label, '/').'/', $this->tally(), $matches) !== 1) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
return (int) $matches[1];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,638 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use FilesystemIterator;
|
||||
use Pest\Factories\TestCaseFactory;
|
||||
use Pest\Plugins\Tia;
|
||||
use Pest\Plugins\Tia\ChangedFiles;
|
||||
use Pest\Plugins\Tia\FileState;
|
||||
use Pest\Plugins\Tia\Fingerprint;
|
||||
use Pest\Plugins\Tia\Graph;
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use Pest\Support\Str;
|
||||
use RecursiveDirectoryIterator;
|
||||
use RecursiveIteratorIterator;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A throwaway Pest project the TIA scenario tests drive.
|
||||
*
|
||||
* Two facts about Pest shape everything here:
|
||||
*
|
||||
* 1. `bin/pest` derives the project root from **the autoloader it finds**, not
|
||||
* from the working directory — `dirname($autoloadPath, 2)`. So the project
|
||||
* owns a real `vendor/autoload.php` and a real copy of `bin/pest` at the path
|
||||
* a composer install would have put them. A symlinked `vendor` would resolve
|
||||
* `__DIR__` straight back to the Pest repository, and every scenario would
|
||||
* silently measure the wrong project.
|
||||
* 2. TIA cannot *record* without pcov or Xdebug, and CI has neither. So a
|
||||
* scenario never records: {@see self::seed()} writes the graph a recording
|
||||
* run would have written, and the run under test exercises the read path.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class Project
|
||||
{
|
||||
/**
|
||||
* Test file → the source files a recording run would have linked it to. The
|
||||
* self-edge every test file gets is added on top of these.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array EDGES = [
|
||||
'tests/Unit/CalculatorTest.php' => ['app/Calculator.php'],
|
||||
'tests/Unit/GreeterTest.php' => ['app/Greeter.php'],
|
||||
'tests/Feature/CoversCalculatorTest.php' => ['app/Calculator.php'],
|
||||
];
|
||||
|
||||
/**
|
||||
* Test file → the descriptions it declares, in declaration order.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array TESTS = [
|
||||
'tests/Unit/CalculatorTest.php' => ['adds two numbers', 'subtracts two numbers'],
|
||||
'tests/Unit/GreeterTest.php' => ['greets a person', 'greets the world'],
|
||||
'tests/Feature/CoversCalculatorTest.php' => ['adds within a feature test', 'subtracts within a feature test'],
|
||||
];
|
||||
|
||||
/**
|
||||
* Every test in the fixture suite.
|
||||
*/
|
||||
public const int TOTAL_TESTS = 6;
|
||||
|
||||
/**
|
||||
* Every project scaffolded so far, so a row cannot leak one by failing
|
||||
* before its own cleanup.
|
||||
*
|
||||
* @var array<int, self>
|
||||
*/
|
||||
private static array $created = [];
|
||||
|
||||
private ?GitRepo $repo = null;
|
||||
|
||||
/**
|
||||
* @var array<string, mixed>|null
|
||||
*/
|
||||
private ?array $snapshot = null;
|
||||
|
||||
/**
|
||||
* @var array<int, string>
|
||||
*/
|
||||
private array $extraPaths = [];
|
||||
|
||||
/**
|
||||
* The root whose graph this project reads and writes.
|
||||
*
|
||||
* Its own, except where a row seeds a worktree: a worktree's `.git` is a
|
||||
* file rather than a directory, so {@see Storage} cannot read the remote
|
||||
* from it and resolves a storage key of its own.
|
||||
*/
|
||||
private string $graphRoot;
|
||||
|
||||
private function __construct(public readonly string $path)
|
||||
{
|
||||
$this->graphRoot = $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scaffolds a project whose default branch is `$branch`, and hands it back
|
||||
* checked out there.
|
||||
*
|
||||
* The repository is realistic on purpose: it has an `origin`, and an
|
||||
* `origin/HEAD` naming `$branch`, which is what a checkout of a real project
|
||||
* looks like and what the default branch is autodetected from.
|
||||
*/
|
||||
public static function make(string $branch = 'master', ?string $overlay = null): self
|
||||
{
|
||||
$project = self::scaffold($overlay);
|
||||
|
||||
$project->git()->init($branch);
|
||||
$project->git()->addOrigin();
|
||||
$project->git()->setOriginHead($branch);
|
||||
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Destroys every project scaffolded so far. Belongs in an `afterEach`.
|
||||
*/
|
||||
public static function destroyAll(): void
|
||||
{
|
||||
while (self::$created !== []) {
|
||||
array_pop(self::$created)->destroy();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A project that is not a git repository at all — for the rows that assert
|
||||
* TIA still demands git, and that a plain run does not care.
|
||||
*
|
||||
* Lives in the system temp directory, so it is outside any enclosing
|
||||
* repository, and owns a real `vendor` rather than a symlinked one, so its
|
||||
* baseline key cannot collide with another fixture's.
|
||||
*/
|
||||
public static function withoutGit(?string $overlay = null): self
|
||||
{
|
||||
return self::scaffold($overlay);
|
||||
}
|
||||
|
||||
public function git(): GitRepo
|
||||
{
|
||||
return $this->repo ??= new GitRepo($this->path);
|
||||
}
|
||||
|
||||
public function path(string $relative = ''): string
|
||||
{
|
||||
return $relative === '' ? $this->path : $this->path.DIRECTORY_SEPARATOR.$relative;
|
||||
}
|
||||
|
||||
/**
|
||||
* Overwrites a file in the project.
|
||||
*
|
||||
* Edits must be semantic: TIA hashes PHP at the AST level, so a
|
||||
* comment-only change is not a change at all.
|
||||
*/
|
||||
public function write(string $relative, string $contents): void
|
||||
{
|
||||
$path = $this->path($relative);
|
||||
$directory = dirname($path);
|
||||
|
||||
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
|
||||
}
|
||||
|
||||
if (file_put_contents($path, $contents) === false) {
|
||||
throw new RuntimeException(sprintf('Unable to write [%s].', $path));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch, scaffolded so `pest` can run in it, and
|
||||
* returns its path.
|
||||
*
|
||||
* The graph is shared with the main checkout, which is the whole point: both
|
||||
* resolve the same storage key, because {@see Storage} prefers the `origin`
|
||||
* identity over the path.
|
||||
*/
|
||||
public function worktree(string $branch): string
|
||||
{
|
||||
$path = $this->path.'-worktree-'.preg_replace('/[^a-z0-9]+/i', '-', $branch);
|
||||
|
||||
$this->git()->worktree($path, $branch);
|
||||
$this->scaffoldVendor($path);
|
||||
$this->extraPaths[] = $path;
|
||||
|
||||
return $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in the project and returns what happened.
|
||||
*/
|
||||
public function pest(string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestIn($this->path, ...$arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in `$directory` — a worktree, say — against this project's
|
||||
* graph.
|
||||
*/
|
||||
public function pestIn(string $directory, string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestWithEnvironment($directory, [], ...$arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, string> $environment
|
||||
*/
|
||||
public function pestWithEnvironment(string $directory, array $environment, string ...$arguments): PestResult
|
||||
{
|
||||
$process = new Process(
|
||||
[PHP_BINARY, $directory.'/vendor/pestphp/pest/bin/pest', ...$arguments],
|
||||
$directory,
|
||||
[
|
||||
...GitRepo::ENV,
|
||||
'COLLISION_PRINTER' => 'DefaultPrinter',
|
||||
'COLLISION_IGNORE_DURATION' => 'true',
|
||||
'PARATEST' => '0',
|
||||
'PAO_DISABLE' => '1',
|
||||
'HOME' => $this->home(),
|
||||
...$environment,
|
||||
],
|
||||
);
|
||||
|
||||
$process->setTimeout(180.0);
|
||||
$process->run();
|
||||
|
||||
return new PestResult(
|
||||
array_values($arguments),
|
||||
$process->getOutput().$process->getErrorOutput(),
|
||||
(int) $process->getExitCode(),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the graph a clean, green recording run on `$branch` would have
|
||||
* written, and remembers it as the snapshot the next {@see self::delta()}
|
||||
* compares against.
|
||||
*
|
||||
* Sentinelled by default: a row almost always wants to know which entries
|
||||
* were written, and only a real recording run's values can answer that.
|
||||
*
|
||||
* `$failing` names descriptions to record as failures — a clean green run
|
||||
* can never cache one, so a row that needs a cached failure to re-run has to
|
||||
* be handed it.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seed(string $branch, bool $sentinel = true, array $failing = []): void
|
||||
{
|
||||
$this->seedFor($this->path, $branch, $sentinel, $failing);
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeds the graph belonging to `$root` — a worktree, say, which resolves a
|
||||
* storage key of its own.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seedFor(string $root, string $branch, bool $sentinel = true, array $failing = []): void
|
||||
{
|
||||
$this->graphRoot = $root;
|
||||
|
||||
$changedFiles = new ChangedFiles($root);
|
||||
$sha = new GitRepo($root)->sha();
|
||||
|
||||
$graph = new Graph($root);
|
||||
$graph->setFingerprint(Fingerprint::compute($root));
|
||||
$graph->setRecordedAtSha($branch, $sha);
|
||||
|
||||
// Hashes the tree as it stands, so the run under test sees nothing as
|
||||
// changed — the same call the recording path makes.
|
||||
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
|
||||
|
||||
$graph->markKnownTestFiles(array_keys(self::EDGES));
|
||||
|
||||
foreach (self::EDGES as $testFile => $sourceFiles) {
|
||||
$graph->link($testFile, $testFile);
|
||||
|
||||
foreach ($sourceFiles as $sourceFile) {
|
||||
$graph->link($testFile, $sourceFile);
|
||||
}
|
||||
}
|
||||
|
||||
foreach (self::TESTS as $testFile => $descriptions) {
|
||||
foreach ($descriptions as $description) {
|
||||
$failed = in_array($description, $failing, true);
|
||||
|
||||
$graph->setResult(
|
||||
$branch,
|
||||
self::testId($testFile, $description),
|
||||
$failed ? 7 : 0,
|
||||
$failed ? 'cached failure' : '',
|
||||
0.05,
|
||||
1,
|
||||
$testFile,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
$json = $graph->encode();
|
||||
|
||||
if ($json === null) {
|
||||
throw new RuntimeException('Unable to encode the seeded graph.');
|
||||
}
|
||||
|
||||
if (! $this->state()->write(Tia::KEY_GRAPH, $json)) {
|
||||
throw new RuntimeException('Unable to persist the seeded graph.');
|
||||
}
|
||||
|
||||
$sentinel ? $this->sentinel() : $this->snapshot();
|
||||
}
|
||||
|
||||
/**
|
||||
* The id PHPUnit reports for a test in the fixture suite.
|
||||
*
|
||||
* Mirrors how Pest names generated test classes
|
||||
* ({@see TestCaseFactory}): a wrong id here shows up as
|
||||
* `0 replayed`, which every scenario asserts against.
|
||||
*/
|
||||
public static function testId(string $testFile, string $description): string
|
||||
{
|
||||
$basename = basename($testFile, '.php');
|
||||
$dotPosition = strpos($basename, '.');
|
||||
|
||||
if ($dotPosition !== false) {
|
||||
$basename = substr($basename, 0, $dotPosition);
|
||||
}
|
||||
|
||||
$relative = dirname(ucfirst($testFile)).DIRECTORY_SEPARATOR.$basename;
|
||||
|
||||
return 'P\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relative).'::'.Str::evaluable($description);
|
||||
}
|
||||
|
||||
/**
|
||||
* Falsifies every cached value, so the next {@see self::delta()} can tell
|
||||
* "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* `assertions` is only ever falsified where it is already non-zero: risky,
|
||||
* skipped and incomplete statuses are *derived* from "performed no
|
||||
* assertions", so patching those would rewrite the status on replay and
|
||||
* destroy the very discriminator this exists to provide.
|
||||
*/
|
||||
public function sentinel(): void
|
||||
{
|
||||
$graph = $this->graph();
|
||||
|
||||
if ($graph === null) {
|
||||
throw new RuntimeException('There is no graph to sentinel.');
|
||||
}
|
||||
|
||||
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
|
||||
foreach (array_keys($baseline['results'] ?? []) as $testId) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
|
||||
|
||||
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($graph, JSON_UNESCAPED_SLASHES));
|
||||
|
||||
$this->snapshot();
|
||||
}
|
||||
|
||||
/**
|
||||
* The decoded graph, or `null` when there is none.
|
||||
*
|
||||
* @return array<string, mixed>|null
|
||||
*/
|
||||
public function graph(): ?array
|
||||
{
|
||||
$json = $this->state()->read(Tia::KEY_GRAPH);
|
||||
|
||||
if ($json === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$graph = json_decode($json, true);
|
||||
|
||||
return is_array($graph) ? $graph : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeys(): array
|
||||
{
|
||||
$baselines = $this->graph()['baselines'] ?? [];
|
||||
|
||||
return is_array($baselines) ? array_keys($baselines) : [];
|
||||
}
|
||||
|
||||
public function graphDir(): string
|
||||
{
|
||||
return $this->withHome(fn (): string => Storage::tempDir($this->graphRoot));
|
||||
}
|
||||
|
||||
public function graphExists(): bool
|
||||
{
|
||||
return is_file($this->graphDir().DIRECTORY_SEPARATOR.Tia::KEY_GRAPH);
|
||||
}
|
||||
|
||||
/**
|
||||
* Remembers the graph as it stands now.
|
||||
*/
|
||||
public function snapshot(): void
|
||||
{
|
||||
$this->snapshot = $this->graph();
|
||||
}
|
||||
|
||||
/**
|
||||
* What has happened to the graph since the last snapshot.
|
||||
*/
|
||||
public function delta(): GraphDelta
|
||||
{
|
||||
return new GraphDelta($this->snapshot, $this->graph());
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the `vendor` a composer install would have produced.
|
||||
*
|
||||
* Pest is mirrored in at `vendor/pestphp/pest` rather than pointed at,
|
||||
* because Pest locates things from where its own files sit:
|
||||
* `bin/pest` finds the project root by walking up from the autoloader it
|
||||
* loads, and a parallel run picks its worker binary — and with it the
|
||||
* worker's project root — from the directory the runner class was loaded
|
||||
* from. Deferring to the repository's copy would resolve both back to the
|
||||
* Pest repository, and every scenario would quietly measure that instead.
|
||||
*
|
||||
* Hardlinked where the filesystem allows it, so the mirror costs almost
|
||||
* nothing and can never drift from the working tree.
|
||||
*/
|
||||
public function scaffoldVendor(string $directory): void
|
||||
{
|
||||
$pestRoot = dirname(__DIR__, 3);
|
||||
$pest = $directory.'/vendor/pestphp/pest';
|
||||
|
||||
// `overrides`, `resources` and `stubs` come along because Pest loads
|
||||
// them relative to `src` — the same list `BootExcludeList` walks.
|
||||
foreach (['src', 'overrides', 'resources', 'stubs'] as $tree) {
|
||||
self::mirror($pestRoot.'/'.$tree, $pest.'/'.$tree);
|
||||
}
|
||||
|
||||
foreach (['pest', 'worker.php'] as $binary) {
|
||||
self::mirror($pestRoot.'/bin/'.$binary, $pest.'/bin/'.$binary);
|
||||
}
|
||||
|
||||
self::mirror($pestRoot.'/composer.json', $pest.'/composer.json');
|
||||
|
||||
// Pest's own autoloader, with the mirrored copy taking precedence: the
|
||||
// repository's `vendor` supplies PHPUnit, Symfony and the plugin
|
||||
// packages, none of which care where they are loaded from.
|
||||
file_put_contents($directory.'/vendor/autoload.php', sprintf(
|
||||
"<?php\n\n\$loader = require %s;\n\$loader->addPsr4('Pest\\\\', __DIR__.'/pestphp/pest/src', true);\n\nreturn \$loader;\n",
|
||||
var_export($pestRoot.'/vendor/autoload.php', true),
|
||||
));
|
||||
|
||||
// Invoking the binary directly skips composer's bin proxy, which is
|
||||
// what would otherwise define `$GLOBALS['_composer_bin_dir']`. Without
|
||||
// it `Pest\Plugin\Loader` looks for `vendor/bin/../pest-plugins.json`
|
||||
// relative to the working directory — so that is where the plugin list
|
||||
// goes, and mirroring the repository's keeps it in step with
|
||||
// composer.json. `vendor/bin` has to exist for the `..` in that path to
|
||||
// resolve, empty though it is.
|
||||
if (! is_dir($directory.'/vendor/bin') && ! @mkdir($directory.'/vendor/bin', 0755, true)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory.'/vendor/bin'));
|
||||
}
|
||||
|
||||
self::mirror($pestRoot.'/vendor/pest-plugins.json', $directory.'/vendor/pest-plugins.json');
|
||||
}
|
||||
|
||||
public function destroy(): void
|
||||
{
|
||||
foreach ([...$this->extraPaths, $this->path] as $path) {
|
||||
$this->remove($path);
|
||||
}
|
||||
}
|
||||
|
||||
private function home(): string
|
||||
{
|
||||
return $this->path.DIRECTORY_SEPARATOR.'.home';
|
||||
}
|
||||
|
||||
private function state(): FileState
|
||||
{
|
||||
return new FileState($this->graphDir());
|
||||
}
|
||||
|
||||
/**
|
||||
* `Storage` reads `HOME` from the environment, so the graph lands inside the
|
||||
* throwaway project instead of the developer's real `~/.pest`.
|
||||
*
|
||||
* @template TReturn
|
||||
*
|
||||
* @param callable(): TReturn $callback
|
||||
* @return TReturn
|
||||
*/
|
||||
private function withHome(callable $callback): mixed
|
||||
{
|
||||
$original = getenv('HOME');
|
||||
|
||||
putenv('HOME='.$this->home());
|
||||
|
||||
try {
|
||||
return $callback();
|
||||
} finally {
|
||||
putenv($original === false ? 'HOME' : 'HOME='.$original);
|
||||
}
|
||||
}
|
||||
|
||||
private static function scaffold(?string $overlay): self
|
||||
{
|
||||
$path = sys_get_temp_dir().DIRECTORY_SEPARATOR.'pest-tia-'.bin2hex(random_bytes(8));
|
||||
|
||||
if (! @mkdir($path, 0755, true) && ! is_dir($path)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
|
||||
}
|
||||
|
||||
// Realpathed because Pest realpaths its own project root, and macOS
|
||||
// hands out a symlinked temp directory.
|
||||
$real = realpath($path);
|
||||
|
||||
$project = new self($real === false ? $path : $real);
|
||||
|
||||
self::$created[] = $project;
|
||||
|
||||
self::copy(__DIR__.'/app', $project->path);
|
||||
|
||||
if ($overlay !== null) {
|
||||
self::copy(__DIR__.'/overlays/'.$overlay, $project->path);
|
||||
}
|
||||
|
||||
// `ChangedFiles` asks git what changed, so anything the scaffold writes
|
||||
// but the project does not own has to be invisible to it.
|
||||
$project->write('.gitignore', implode("\n", ['/vendor/', '/.home/', '/.phpunit.cache/', '']));
|
||||
|
||||
$project->scaffoldVendor($project->path);
|
||||
@mkdir($project->home(), 0755, true);
|
||||
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirrors a file or directory, hardlinking where the filesystem allows it
|
||||
* and copying where it does not.
|
||||
*/
|
||||
private static function mirror(string $from, string $to): void
|
||||
{
|
||||
if (is_dir($from)) {
|
||||
foreach (self::contentsOf($from) as $path) {
|
||||
self::mirror($path->getPathname(), $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1));
|
||||
}
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
$directory = dirname($to);
|
||||
|
||||
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
|
||||
}
|
||||
|
||||
if (@link($from, $to) || @copy($from, $to)) {
|
||||
return;
|
||||
}
|
||||
|
||||
throw new RuntimeException(sprintf('Unable to mirror [%s] into [%s].', $from, $to));
|
||||
}
|
||||
|
||||
/**
|
||||
* @return iterable<\SplFileInfo>
|
||||
*/
|
||||
private static function contentsOf(string $directory): iterable
|
||||
{
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($directory, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::SELF_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $path) {
|
||||
if (! $path->isDir()) {
|
||||
yield $path;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static function copy(string $from, string $to): void
|
||||
{
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($from, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::SELF_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $path) {
|
||||
$target = $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1);
|
||||
|
||||
if ($path->isDir()) {
|
||||
if (! is_dir($target) && ! @mkdir($target, 0755, true) && ! is_dir($target)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $target));
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (! is_dir(dirname($target)) && ! @mkdir(dirname($target), 0755, true) && ! is_dir(dirname($target))) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', dirname($target)));
|
||||
}
|
||||
|
||||
if (! @copy($path->getPathname(), $target)) {
|
||||
throw new RuntimeException(sprintf('Unable to copy [%s].', $path->getPathname()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private function remove(string $path): void
|
||||
{
|
||||
if (! is_dir($path)) {
|
||||
return;
|
||||
}
|
||||
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($path, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::CHILD_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $entry) {
|
||||
$entry->isDir() && ! $entry->isLink() ? @rmdir($entry->getPathname()) : @unlink($entry->getPathname());
|
||||
}
|
||||
|
||||
@rmdir($path);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Calculator
|
||||
{
|
||||
public function add(int $a, int $b): int
|
||||
{
|
||||
return $a + $b;
|
||||
}
|
||||
|
||||
public function subtract(int $a, int $b): int
|
||||
{
|
||||
return $a - $b;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Greeter
|
||||
{
|
||||
public function greet(string $name): string
|
||||
{
|
||||
return sprintf('Hello, %s!', $name);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"name": "pest/tia-fixture",
|
||||
"description": "Throwaway project the TIA scenario tests scaffold into a temp directory.",
|
||||
"license": "MIT",
|
||||
"require": {},
|
||||
"config": {
|
||||
"lock": true
|
||||
}
|
||||
}
|
||||
Generated
+17
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"_readme": [
|
||||
"Hand-written: the fixture project installs nothing. It exists so the TIA",
|
||||
"fingerprint has a composer.lock to hash, the way a real project does."
|
||||
],
|
||||
"content-hash": "0000000000000000000000000000000000",
|
||||
"packages": [],
|
||||
"packages-dev": [],
|
||||
"aliases": [],
|
||||
"minimum-stability": "stable",
|
||||
"stability-flags": {},
|
||||
"prefer-stable": true,
|
||||
"prefer-lowest": false,
|
||||
"platform": {},
|
||||
"platform-dev": {},
|
||||
"plugin-api-version": "2.6.0"
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
|
||||
bootstrap="vendor/autoload.php"
|
||||
cacheDirectory=".phpunit.cache"
|
||||
colors="true"
|
||||
failOnRisky="true"
|
||||
failOnWarning="false"
|
||||
>
|
||||
<testsuites>
|
||||
<testsuite name="default">
|
||||
<directory suffix="Test.php">./tests</directory>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
<source>
|
||||
<include>
|
||||
<directory suffix=".php">./app</directory>
|
||||
</include>
|
||||
</source>
|
||||
</phpunit>
|
||||
@@ -0,0 +1,15 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// A second test file over the same source file, so a `Calculator` edit narrows
|
||||
// to two of the three test files rather than to one.
|
||||
test('adds within a feature test', function (): void {
|
||||
expect((new Calculator)->add(10, 5))->toBe(15);
|
||||
});
|
||||
|
||||
test('subtracts within a feature test', function (): void {
|
||||
expect((new Calculator)->subtract(10, 5))->toBe(5);
|
||||
});
|
||||
@@ -0,0 +1,9 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// The fixture project has no composer autoloader of its own — its `vendor/`
|
||||
// holds nothing but a shim pointing back at Pest's. Requiring the two classes
|
||||
// here is enough for every test file in the suite.
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
@@ -0,0 +1,16 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// `test()` rather than `it()`: the harness seeds results by test id, and `it()`
|
||||
// would prefix every description with `it `, leaving Project::TESTS a step away
|
||||
// from what is written here.
|
||||
test('adds two numbers', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
expect((new Calculator)->subtract(3, 1))->toBe(2);
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
@@ -0,0 +1,11 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// Declared default branch. Wins over whatever the repository autodetects — the
|
||||
// escape hatch for a checkout with no `origin/HEAD` and a misleading
|
||||
// `init.defaultBranch`.
|
||||
pest()->tia()->defaultBranch('master');
|
||||
@@ -0,0 +1,10 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// A branch the repository does not have. Accepted as configured — a name that
|
||||
// resolves to no baseline degrades to a full run, which is safe.
|
||||
pest()->tia()->defaultBranch('nope');
|
||||
@@ -19,6 +19,10 @@ pest()->in('PHPUnit/GlobPatternTests/SubFolder2/*AsPattern.php')->use(CustomTest
|
||||
|
||||
pest()->in('Visual')->group('integration');
|
||||
|
||||
// Every row scaffolds a throwaway project and runs `pest` in it as a
|
||||
// subprocess, which is far too slow for the unit suite.
|
||||
pest()->in('Features/Tia')->group('integration');
|
||||
|
||||
// NOTE: global test value container to be mutated and checked across files, as needed
|
||||
$_SERVER['globalHook'] = (object) ['calls' => (object) ['beforeAll' => 0, 'afterAll' => 0]];
|
||||
|
||||
|
||||
@@ -66,8 +66,26 @@ describe('applyMigrationChanges()', function (): void {
|
||||
});
|
||||
|
||||
describe('rerun tracking', function (): void {
|
||||
beforeEach(function (): void {
|
||||
// `hasUnlocatedTestsToRerun()` stats each recorded file to tell a
|
||||
// deleted test apart from a live one, so the files have to exist.
|
||||
$this->projectRoot = sys_get_temp_dir().'/pest-tia-rerun-'.bin2hex(random_bytes(4));
|
||||
mkdir($this->projectRoot.'/tests/Feature', 0755, true);
|
||||
|
||||
touch($this->projectRoot.'/tests/Feature/FooTest.php');
|
||||
touch($this->projectRoot.'/tests/Feature/BarTest.php');
|
||||
});
|
||||
|
||||
afterEach(function (): void {
|
||||
@unlink($this->projectRoot.'/tests/Feature/FooTest.php');
|
||||
@unlink($this->projectRoot.'/tests/Feature/BarTest.php');
|
||||
@rmdir($this->projectRoot.'/tests/Feature');
|
||||
@rmdir($this->projectRoot.'/tests');
|
||||
@rmdir($this->projectRoot);
|
||||
});
|
||||
|
||||
it('reruns cached failures via their file', function (): void {
|
||||
$graph = new Graph(sys_get_temp_dir());
|
||||
$graph = new Graph($this->projectRoot);
|
||||
$graph->setResult('main', 'Tests\FooTest::it fails', 7, 'boom', 0.1, 1, 'tests/Feature/FooTest.php');
|
||||
$graph->setResult('main', 'Tests\BarTest::it passes', 0, '', 0.1, 1, 'tests/Feature/BarTest.php');
|
||||
|
||||
@@ -76,7 +94,7 @@ describe('rerun tracking', function (): void {
|
||||
});
|
||||
|
||||
it('flags cached failures whose file is unknown', function (): void {
|
||||
$graph = new Graph(sys_get_temp_dir());
|
||||
$graph = new Graph($this->projectRoot);
|
||||
$graph->setResult('main', 'Tests\EvalTest::it fails', 7, 'boom', 0.1, 1);
|
||||
|
||||
expect($graph->testFilesToRerun('main'))->toBeEmpty()
|
||||
|
||||
Reference in New Issue
Block a user