Compare commits

...

27 Commits

Author SHA1 Message Date
nuno maduro e90e4f70fc wip 2026-08-05 20:20:35 +01:00
nuno maduro d112582857 wip 2026-08-05 19:58:57 +01:00
nuno maduro 9f3c4e1e82 fix 2026-08-05 18:51:49 +01:00
nuno maduro 71f39366c2 wip 2026-08-05 18:21:34 +01:00
nuno maduro bfd5b75677 fix 2026-08-05 16:49:35 +01:00
nuno maduro 411b9954b5 wip 2026-08-05 16:18:03 +01:00
nuno maduro db70017cb2 wip 2026-08-05 03:30:50 +01:00
nuno maduro 86adaedbbd fix: tia filtered 2026-08-04 23:30:03 +01:00
nuno maduro 92c7677c6e fix: don't allow invalid class names 2026-08-04 20:46:21 +01:00
nuno maduro 872f0a50c2 fix(tia): replayable scenarios running afterEach 2026-08-04 20:19:12 +01:00
nuno maduro 668809bc75 chore: adds missing test 2026-08-04 19:28:58 +01:00
nuno maduro 8d8f45c843 chore: bump deps 2026-08-04 19:28:52 +01:00
nuno maduro 086b3e9107 chore: ai instructions 2026-08-04 19:01:16 +01:00
nuno maduro 43fe26f324 chore: bumps deps 2026-08-04 19:01:08 +01:00
nuno maduro 865c1e5113 ci: snapshots 2026-08-04 18:51:04 +01:00
stephanbouman 19eed8d581 Fixed typos (#1819)
* Fixed typo in message

* Fixed typo in message
2026-08-03 11:29:03 -04:00
Danny de Wit ad1850b110 fix: keep TableExtractor table names as strings (#1794)
PHP coerces numeric-string array keys to ints, so collecting table names in
$tables[$name] and returning array_keys() can yield ints. That breaks the
declared list<string> and throws a TypeError in Recorder::linkTable(string).

Standard SQL like substring(x FROM 1 FOR 10) is enough to trigger it, since
the numeric operand matches the FROM pattern.

Closes #1793

Co-authored-by: Danny de Wit <d.dewit@fenceweb.com>
2026-08-02 22:50:47 -04:00
nuno maduro 585de259a2 chore: style 2026-08-02 22:26:34 -04:00
nuno maduro 953664dce0 release: v5.0.3 2026-08-02 22:24:32 -04:00
nuno maduro 1dd959848c chore: updates snapshots 2026-08-02 22:24:24 -04:00
Jeffrey van Hees 047753c836 fix: key Tia results per dataset row (#1799)
The result cache was keyed `Class::method` on both sides, so every row of a
`->with()` test shared one entry and the last writer won. On replay that single
status was handed to every row.

`TestMethod::id()` already appends `#dataSetName`, and `TestCase` reaches the
same value object through `valueObjectForEvents()`, so both sides can use it.
2026-08-02 21:43:15 -04:00
nuno maduro b63626a94d docs: adjusts readme
Signed-off-by: nuno maduro <enunomaduro@gmail.com>
2026-07-30 11:53:41 -04:00
nuno maduro ae080becd6 chore: style 2026-07-29 15:27:11 -04:00
nuno maduro aa86304794 release: v5.0.2 2026-07-29 14:59:13 -04:00
nuno maduro 75963ea973 chore: bumps deps 2026-07-29 14:59:03 -04:00
ohnotnow 5d0ceb1a5d Check for pcov being enabled for tia (#1779)
* Check for pcov being enabled for tia

* Delete tests/Features/TiaDriverGate.php

Signed-off-by: nuno maduro <enunomaduro@gmail.com>

---------

Signed-off-by: nuno maduro <enunomaduro@gmail.com>
Co-authored-by: Ohffs <ohffsnotnow@gmail.com>
Co-authored-by: nuno maduro <enunomaduro@gmail.com>
2026-07-29 14:28:38 -04:00
talaridisTh 6f0c0536a4 feat: document the tia options in pest --help (#1780) 2026-07-29 14:11:17 -04:00
97 changed files with 4742 additions and 131 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
.idea/*
.idea/codeStyleSettings.xml
.temp/*
composer.lock
/composer.lock
/vendor/
coverage.xml
.phpunit.result.cache
+18
View File
@@ -0,0 +1,18 @@
# CLAUDE.md
**Do not edit this file.** Agents must never add, remove, or reword anything here. If a change seems needed, say so and let a human do it.
## Ask before testing
When asked to review code or build a feature, do not run the test suite and do not write new tests. Make the change, report it, then ask the user whether tests should be added — describing the tests you have in mind — and wait for the user to confirm.
Two reasons this matters here: the suite takes minutes, and `tests/.snapshots/success.txt` plus the tally in `tests/Visual/Parallel.php` encode the whole suite's result, so a single added test breaks both.
Once the user confirms:
```bash
composer test:unit # fast, excludes the visual group
composer test:integration # visual and snapshot tests
composer test # everything CI runs, in CI's order
composer update:snapshots # only when a test was added or removed
```
+342
View File
@@ -0,0 +1,342 @@
# TIA write-tier conformance — plan
Baseline sweep: **147 cases, 144 PASS / 2 FAIL / 1 SKIP** against `dev-fix/tia-filtered` (`db70017`)
on `playground/laravel` with pcov enabled.
This file records (1) what is left to fix, and (2) the full case matrix with the target
outcome for every row, so the sweep can be re-run as a conformance check after the fixes land.
---
## Part 1 — What's left
### 1. `--parallel` silently under-records `edges` (G12, I5) — **high**
Any `--tia --parallel` complete run writes `files=4` where sequential writes `files=25`. Only
`app/**` and blade views survive; every `config/`, `routes/`, `bootstrap/` edge and every test
self-edge is lost. A `routes/web.php` edit that sequential TIA catches, parallel TIA reports as
`No affected tests found`.
Root cause — `src/Restarters/PcovRestarter.php:33`:
```php
if (! Tia::isEnabledForRun($arguments)) {
return;
}
```
`Tia::isEnabledForRun()` (`src/Plugins/Tia.php:275-288`) returns true only for `--tia` in argv or
the `PEST_TIA` env flag. A paratest worker's argv carries neither, so workers never re-exec with
`-d pcov.directory=<projectRoot>` and record under pcov's empty default scope.
Proven both directions:
- `PEST_PCOV_RESTARTER_RESTARTED=1 pest --tia` (sequential, restart suppressed) reproduces
`files=4` with edge sets identical to the parallel graph.
- `PEST_TIA=1 pest --parallel` restores the full `files=25` graph. **Working workaround today.**
Coverage config is irrelevant: widening `phpunit.xml` `<include>` to `app|config|routes|bootstrap|tests`
still yields `files=4`.
Sticky, not just per-run: `pest --tia --filtered --parallel` re-records only the affected test
through a worker and strips that entry's edges from an otherwise-healthy sequential graph, so one
parallel run degrades a good graph incrementally.
**Fix direction:** propagate the TIA-enabled decision into workers (e.g. set `PEST_TIA` in the
worker env, or have the parent stamp a recording global the restarter also consults) so
`PcovRestarter` restarts them. Then assert `edges` equivalence between sequential and parallel.
### 2. Deleted test file with a cached failure wedges `--filtered` forever (I9) — **high**
Expected a WARN and a full-suite fallback. Actual: `--filtered` selects the phantom file, runs
nothing, exits **0 green**, on every subsequent invocation.
Root cause — `src/Plugins/Tia/Graph.php:1523-1525`:
```php
if ($real === false) {
$real = $path;
}
```
`relative()` falls back to the raw path when `realpath()` fails, so a deleted-but-in-project test
file is never "unlocated", `Graph::hasUnlocatedTestsToRerun()` never fires, and the WARN branch at
`src/Plugins/Tia.php:982-986` is unreachable for the deletion case. `pruneMissingTests()` only runs
under `--fresh`, so nothing clears it.
**Fix direction:** distinguish "path could not be resolved" from "path resolved outside the root" in
`relative()`, or have `hasUnlocatedTestsToRerun()` stat the file directly.
### 3. `--coverage-text` does not disable filtered mode (I8) — **low**
`coverageReportActive()` (`src/Plugins/Tia.php:1705-1710`) reads only Pest's own `--coverage`, so
raw PHPUnit coverage flags (`--coverage-text`, `--coverage-html`, `--coverage-clover`, …) leave
filtered mode on. `Tia::COVERAGE_FLAGS` at `:113-115` already enumerates them — that list is not
consulted here.
### 4. `--coverage` narrows edges like parallel does — **medium, needs triage**
`pest --tia --coverage` prints `fresh graph (recording coverage baseline)` even when a graph
exists, and writes `Feature/ExampleTest` with 2 files instead of 16, dropping self-edges. Same
observable shape as G12, likely a different mechanism (piggyback collector scoped by
`phpunit.xml <include>` rather than the pcov-restarted recorder). Theme: **edges silently narrow to
`app/**` whenever recording does not go through the pcov-restarted TIA recorder.** Confirm whether
these are one fix or two.
### 5. Smaller items
| Item | Where | Note |
|---|---|---|
| Warning/deprecation recorded as `status=0` | write path | Codes `6` and `4` appear unreachable; risky/skipped/incomplete record faithfully |
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
| Replay clobbers cached `time` | write path | `0.058``0.001`; timing data degrades toward zero across runs |
| `pest --parallel` without `--tia` writes nothing | G3, G6 | Sequential `pest --filter=…` does record. Parallel CI contributes nothing to the cached-failure replay path |
| `--tia --parallel --filter`/`--shard` record nothing | G2, G8 | More conservative than the results-only contract requires |
| `--min=50` without `--coverage` is a silent no-op | — | — |
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
### 6. Test-harness gaps to close before re-running
- The playground has no `UsesClass`, `->note()`, `->flaky()`, `->issue()`, `->pr()`, `->ticket()` or
`->assignee()` annotations, so **C19, C23C28, C37C39 matched zero tests** and their
graph-invariant assertions proved nothing. Add annotated fixtures to make those rows load-bearing.
- Use **sentinel patching** (rewrite every cached entry to `time=9.999 assertions=42`, then see which
entries get overwritten) as the discriminator. It is the only way to tell "wrote identical values"
from "wrote nothing". A canary test absent from `edges` is unreliable under `--filter` because it
never matches the filter and so never runs.
- `--tia` on a clean green tree can never cache a failure (unchanged tests replay rather than
execute); seeding one requires `--fresh` or an env-driven flaky fixture.
- Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
---
## Part 2 — Full case matrix (target outcomes)
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Status column: `PASS` = conforming today · `FIX` = must pass once the item above lands ·
`SKIP` = case not runnable as written.
### A — Setup & sanity
| # | Case | Target outcome | Status |
|---|---|---|---|
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS |
| A2 | `pest --baseline` | prints an existing dir; exit 0 | PASS |
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | PASS |
| A4 | `git status` after reset ritual | clean | PASS |
| A5 | `extension_loaded("pcov")` | `true` | PASS |
| A6 | delete graph, plain `pest` | no graph created | PASS |
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS |
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | PASS |
### B — COMPLETE runs still write
| # | Case | Target outcome | Status |
|---|---|---|---|
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | PASS |
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS |
| B3 | `pest` (plain) | same as B2 | PASS |
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | PASS |
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS |
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS |
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS |
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS |
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS |
| B10 | `pest --tia` twice | `time` differs; statuses stable | PASS |
| B11 | add a test file, `pest --tia` | new `edges` key + new result appear | PASS |
| B12 | `pest --tia --coverage` | completes; graph written | PASS |
### C — Selection narrowing → RESULTS-ONLY
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
| # | Case | Target outcome | Status |
|---|---|---|---|
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS |
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS |
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS |
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS |
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS |
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS |
| C7 | `pest --group=smoke` | RO; no notice | PASS |
| C8 | `pest --tia --group=smoke` | RO; notice | PASS |
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS |
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS |
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS |
| C12 | `pest --tia tests/Unit` | RO; notice | PASS |
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS |
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS |
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS |
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS |
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS |
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS |
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS |
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS |
| C22 | `pest --tia --dirty` | RO; notice | PASS |
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS |
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS |
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS |
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS |
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS |
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS |
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS |
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS |
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
### D — Truncation → RESULTS-ONLY
D1D5, D11D13, D16 precondition: `trio one` broken. D6D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
| # | Case | Target outcome | Status |
|---|---|---|---|
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS |
| D2 | `pest --retry` | RO; siblings survive | PASS |
| D3 | `pest --stop-on-failure` | RO | PASS |
| D4 | `pest --stop-on-defect` | RO | PASS |
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS |
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS |
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS |
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS |
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS |
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS |
| D11 | `pest --tia --bail` | RO | PASS |
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS |
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS |
| D14 | D13 config, green suite | COMPLETE | PASS |
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS |
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS |
### E — Result merge semantics
| # | Case | Target outcome | Status |
|---|---|---|---|
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS |
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS |
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS |
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS |
| E5 | any partial run | `assertions` and `time` update for the test that ran | PASS |
| E6 | any partial run | `message` of untouched tests unchanged | PASS |
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS |
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS |
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS |
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS |
| E11 | partial run of a risky test | `status=5` | PASS |
| E12 | any partial run | `fingerprint` byte-identical | PASS |
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS |
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS |
### F — Guard rails
| # | Case | Target outcome | Status |
|---|---|---|---|
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | PASS |
| F2 | complete `pest --tia` first, then the same filter | result is recorded | PASS |
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS |
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS |
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS |
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS |
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS |
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS |
### G — Parallel
| # | Case | Target outcome | Status |
|---|---|---|---|
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written | PASS |
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS |
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS |
| G4 | `pest --tia --parallel --bail` | RO | PASS |
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | PASS (but see G12 — it strips edges) |
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | PASS |
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS |
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS |
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS |
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS |
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | PASS |
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** (`files=25` both) | **FIX (item 1)** |
### H — Baseline key / branch resolution
| # | Case | Target outcome | Status |
|---|---|---|---|
| H1 | `pest --tia` on `master` | only a `master` key | PASS |
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS |
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS |
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS |
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS |
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS |
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS |
| H8 | after every C and D case | no baseline key other than the real branch | PASS |
| H9 | non-git dir, `pest --tia` | `MissingDependency``The feature "Tia mode" requires "git".` | PASS |
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS |
### I — Filtered mode
| # | Case | Target outcome | Status |
|---|---|---|---|
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS |
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | PASS |
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS |
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS |
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **FIX (item 1)** |
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS |
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS |
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report | **FIX (item 3)** |
| I9 | cached failure whose test file was deleted | WARN `could not be located on disk`; falls back to the full suite with replay | **FIX (item 2)** |
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS |
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS |
| I12 | edit `composer.lock` | fingerprint drift → full rebuild (and the drift reason should be printed — item 5) | PASS |
### J — Interactions & regressions
| # | Case | Target outcome | Status |
|---|---|---|---|
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS |
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS |
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS |
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS |
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS |
| J6 | `pest --tia --compact` | COMPLETE | PASS |
| J7 | `pest --tia -v` | COMPLETE | PASS |
| J8 | `pest --tia --profile` | COMPLETE | PASS |
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS |
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS |
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP**`--repeat` is not a Pest option |
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS |
| J13 | `pest --tia --min=50` | COMPLETE | PASS |
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (signal must go to the re-exec'd child — item 5) |
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS |
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS |
---
## Re-run procedure
```bash
cd /Users/nunomaduro/Work/projects/playground/laravel
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json"
# per case
git checkout -q . && git clean -qfd tests app
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
```
Every pest invocation must be prefixed with `PAO_DISABLE=1` (the app has `laravel/pao`, which emits
JSON when it detects an agent). The shell is zsh — build commands with arrays or `eval`; unquoted
`$args` does not word-split.
+455
View File
@@ -0,0 +1,455 @@
# TIA default-branch fallback — phase three
## Your task
Fix [pestphp/pest#1823](https://github.com/pestphp/pest/issues/1823) — TIA's cached results never
hit for repos whose default branch is not literally `main` — then re-run the **whole** conformance
matrix against the playground app, plus the new section **L** that covers the fix.
Work in this order, and **stop where the plan says stop**:
1. **Part 1** — read the diagnosis. It is already measured; do not re-derive it, but do re-confirm
the two reproductions in Part 1.3 take ~2 minutes and prove your environment is sane.
2. **Part 2** — implement the fix in the `pestphp/pest` repo. **Do not commit. Do not touch the
playground's `vendor/`.**
3. **HARD STOP → Part 3.** Report the diff to Nuno and wait. He validates, commits, and syncs it
into the playground himself. You must not proceed until he confirms.
4. **Part 4** — verify the sync landed, then run section **L** (new) and the full phase-two matrix
(**AK**, all 156 rows) against the playground.
5. **Part 5** — report in the given format.
Per `CLAUDE.md`: **do not write new `pestphp/pest` unit tests and do not run `composer test`.** Make
the change, report it, and ask whether repo tests should be added. The section-L rows are *playground
invocations*, not repo tests — those are the deliverable and are always in scope.
---
## Part 1 — The diagnosis (already measured; commit `bfd5b756`)
### 1.1 Root cause — two independent hardcoded `'main'` literals
**(a) The read fallback.** `src/Plugins/Tia/Graph.php` — seven methods default the fallback to the
literal `'main'`:
| line | method |
|---|---|
| 579 | `recordedAtSha(string $branch, string $fallbackBranch = 'main')` |
| 614 | `getAssertions(…, string $fallbackBranch = 'main')` |
| 625 | `getTime(…, string $fallbackBranch = 'main')` |
| 636 | `getResult(…, string $fallbackBranch = 'main')` |
| 663 | `testFilesToRerun(string $branch, string $fallbackBranch = 'main')` |
| 700 | `hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main')` |
| 811 | `lastRunTree(string $branch, string $fallbackBranch = 'main')` |
They all funnel into `Graph::baselineFor()` (line 819), which *does* implement a real cross-branch
fallback:
```php
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
return ['sha' => null, 'tree' => [], 'results' => []];
```
The mechanism is deliberate. The problem is that **no caller ever passes `$fallbackBranch`** — all
nine call sites in `src/Plugins/Tia.php` (lines 419, 429, 432, 785, 857, 1026, 1031, 1048, 1056) pass
only `$this->branch`. So on a `master`-named repo the second branch of `baselineFor()` can never
fire, and the whole mechanism is dead code.
`grep -rniE 'defaultBranch|symbolic-ref|init\.defaultBranch|origin/HEAD' src/` returns **nothing**
no default-branch resolution exists anywhere.
**(b) The detached-HEAD default.** `src/Plugins/Tia.php:206` declares `private string $branch = 'main';`
and `ChangedFiles::currentBranch()` (`ChangedFiles.php:208`) returns `null` for detached HEAD. So on
detached HEAD `$this->branch` stays the literal `'main'` and is used for **both reads and writes**
minting a baseline key for a branch that does not exist. This is a *write*-side bug and a separate fix
from (a).
### 1.2 Not a regression
`git log -S"fallbackBranch = 'main'"` bottoms out at `c7e32f5d feat(tia): continues to work on poc`.
This is original PoC code, untouched by phase one. Phase one's change 1 modified
`hasUnlocatedTestsToRerun()`'s file-existence check — one of the seven methods — without going near
the fallback. Do not report it as a phase-one regression.
No test anywhere exercises the mechanism: `tests/Unit/Plugins/Tia/Graph.php` uses `'main'` as the
*actual* branch name, so those assertions pass whether or not the fallback exists. A branch-name
mismatch is never tested.
### 1.3 The two reproductions (re-confirm these before you start)
Both on the playground, `master`-named default branch, zero local changes:
```
default = master default = main
1. record on default → full run (cold) → full run (cold)
2. 1st run on feature-x → 25 UNCACHED → 25 replayed ← (a)
3. 2nd run on feature-x → 25 replayed → 25 replayed
4. back on default → 25 replayed → 25 replayed
5. 1st run on feature-y → 25 UNCACHED → 25 replayed
```
```
master-only graph, then `git checkout --detach`, then `pest --tia`:
→ 25 uncached, and keys become [master,main] ← (b) spurious key
```
The cost is **one full run per new branch, forever**, with no output explaining why — the only clue
is the `N uncached` count; the headline is just `─ Experimental TIA mode enabled.`
### 1.4 Correction to phase two
Phase two reported **H6 and H7 as passing. Both were false passes.** They ran after H5, which had
renamed `master``main` and left a `main` key in the graph, so the hardcoded fallback resolved by
accident. Re-measured against a clean `master`-only graph, both fail. Section L replaces them as the
load-bearing rows; H6/H7 must be re-run **from a cold graph** this time (see Part 4.2).
---
## Part 2 — The fix to implement
### 2.1 Recommended design
**Step 1 — add a non-throwing resolver** to `src/Plugins/Tia/ChangedFiles.php`, next to
`currentBranch()`:
```php
public function defaultBranch(): ?string
```
Resolution order, each step failing soft to the next:
1. `git symbolic-ref --short refs/remotes/origin/HEAD` → strip a leading `origin/`
2. `git config --get init.defaultBranch`
3. `null`
Unlike `currentBranch()`, this must **never throw** `MissingDependency` — it is advisory. Return
`null` on any non-zero exit or empty output.
**Step 2 — add a config surface.** `src/Plugins/Tia/Configuration.php` already exposes `always()`,
`locally()`, `filtered()`, `baselined()`, `watch()`. Add:
```php
public function defaultBranch(string $branch): self
```
so `pest()->tia()->defaultBranch('master')` works in `tests/Pest.php`. Explicit config **always
wins** over autodetection — that is the escape hatch when `origin/HEAD` is unset.
**Step 3 — resolve once, in `Tia.php`.** The read path is hot (`getResult()` is called per test at
line 419), so resolution must not shell out per call. Resolve alongside `$this->branch` at
`Tia.php:1910`, under the existing `$branchResolved` guard:
```php
$this->fallbackBranch = $configuredDefaultBranch
?? $changedFiles->defaultBranch()
?? 'main';
```
**Step 4 — thread it into `Graph`.** Prefer a `Graph`-level property over editing nine call sites:
add `Graph::setFallbackBranch(string $branch)`, change the seven signatures to
`?string $fallbackBranch = null`, and resolve inside each with
`$fallbackBranch ??= $this->fallbackBranch;`. `baselineFor()` itself needs no change. This keeps the
public signatures backward-compatible and minimises blast radius.
**Step 5 — fix the detached-HEAD write.** `Tia.php:206`'s `= 'main'` default must become the
resolved default branch, so detached HEAD stops minting a phantom key.
### 2.2 Invariants the fix must not break
These are all covered by existing matrix rows — the fix is wrong if any of them moves:
- **Read-only.** The fallback must affect *reads* only. Writes go through `ensureBaseline($branch)`
and must keep using the real current branch. Otherwise H1H4/H8 ("no baseline key other than the
real branch") break.
- **H9** — a non-git dir with `--tia` must still raise
`MissingDependency: The feature "Tia mode" requires "git".` Adding a soft resolver must not
swallow that.
- **H10** — plain `pest` in a non-git dir must still run and create no baseline dir.
- **A2/A3** — cold-graph recording unchanged.
- **I1/E3** — clean+green `--tia --filtered` must still be a true zero-delta run.
- Filtered mode reads `testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so the fallback must
reach those two as well, not just `getResult()`.
### 2.3 Decisions for Nuno (raise these at the Part 3 stop)
- **D1** — Consult `origin/HEAD` at all? It requires `git remote set-head` and is absent in many CI
checkouts and all remote-less repos. Config + `init.defaultBranch` only is simpler but helps fewer
people out of the box. *Recommendation: keep it, first in the chain, since it fails soft.*
- **D2** — Single-key heuristic: if the graph holds exactly one baseline key, use it as the fallback?
Fixes the issue with zero git calls, but is implicit and surprising when several keys exist.
*Recommendation: no.*
- **D3** — Should detached HEAD write a baseline at all, or be read-only? Current behaviour mints a
key. *Recommendation: read-only.*
- **D4** — Should `pest()->tia()->defaultBranch()` validate that the branch exists, or accept any
string? *Recommendation: accept any string; a nonexistent name degrades to a full run, which is
safe.*
---
## Part 3 — HARD STOP
When the code is written:
1. Show Nuno the diff (`git -C /Users/nunomaduro/Work/projects/pestphp/pest diff`) and a one-paragraph
summary of each file's change.
2. Answer/raise the D1D4 decisions.
3. State explicitly that you have **not** committed and have **not** synced `vendor/`.
4. Ask whether repo unit tests should be added (per `CLAUDE.md`), describing the tests you have in
mind — do not write them yet.
5. **Wait.** Nuno commits and applies the change to the playground.
**Never sync the playground's `vendor/` yourself.** `vendor/pestphp/pest` there is a dist copy, not a
symlink (composer installed `dev-fix/tia-filtered as 5.2.0`), so pest-repo edits do not reach it. Say
what is stale and wait. This applies to before/after contrasts too.
Once he confirms, verify the sync actually landed before measuring anything:
```bash
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
grep -c 'defaultBranch' "$V/src/Plugins/Tia/ChangedFiles.php" # must be ≥ 1
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php src/Plugins/Tia/ChangedFiles.php \
src/Plugins/Tia/Configuration.php; do
diff -q "/Users/nunomaduro/Work/projects/pestphp/pest/$f" "$V/$f" >/dev/null \
&& echo "SAME $f" || echo "STALE $f"
done
```
If anything reports `STALE`, stop and tell him. State in your final report which pest commit produced
the playground numbers.
---
## Part 4 — Measurement
### 4.1 Environment and traps
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel` (branch `master`).
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON when
it detects an agent.
2. **Pin the interpreter to `php85`.** The playground requires PHP `>= 8.4.1` *and* pcov. Locally only
`php85` (8.5.8) has both — `php84` (8.4.23) has no pcov, and the bare Herd `php` shim has been
observed drifting to 8.3.32 mid-session, which kills every run in
`vendor/composer/platform_check.php`. Also put a `php``php85` symlink first on `PATH`: the
`--shard` list-tests probe spawns a subprocess via bare `php`, not `PHP_BINARY`.
3. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`. Scope
resets to `git checkout -- tests app` / `git clean -fd tests app`. For `phpunit.xml` and
`composer.lock`, copy aside and copy back, verifying with `shasum`.
4. **zsh does not word-split unquoted parameters.** A `$PEST` string containing a space becomes one
command name. Route every invocation through `eval` (the `pest()` helper below does this).
5. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
6. **The sentinel technique must not falsify `assertions` on zero-assertion tests.** A
risky/skipped/incomplete status is *derived* from "performed no assertions", so patching those to
`42` rewrites the status on replay and destroys the discriminator — it shows up as a phantom
`status 5→0` defect. The `sentinel.php` below only patches `assertions` where it is already
non-zero. With that, a full replay gives a clean `rewritten=0`.
7. Restore branch state after every L row. Several rows rename or detach; leaving a stray `main` key
in the graph is exactly what produced phase two's false H6/H7 passes.
### 4.2 Reference numbers
The playground already carries the phase-two fixtures at commit `fb2e77e`**do not rebuild them.**
A healthy sequential `--tia` graph is:
> **`files=27`, 10 edge keys (one per test file), self-edge on all 10, `n=25` results.**
`sha` will differ once Nuno commits the vendor sync — re-derive it once and use it throughout. The
`files`/`edges`/`n` numbers hold as long as no test fixture changes. Suite shape: 10 test files, 25
tests, including six deliberate status fixtures (skipped, todo, incomplete, risky, warning,
deprecation), a 3-row dataset, a `smoke` group, an env-driven flaky test (green unless
`FLAKY_FAIL=1`), and the annotation set (`covers`/`note`/`flaky`/`issue`/`pr`/`ticket`/`assignee`).
**Re-run H6 and H7 from a cold graph** (`rm -rf` the graph dir, record on `master` only, *then*
branch/detach). Their phase-two results are void.
### 4.3 Harness
Write these to your scratchpad. `$SP` is your own scratchpad dir.
<details>
<summary><code>lib.sh</code></summary>
```bash
#!/bin/zsh
export PAO_DISABLE=1
PG=/Users/nunomaduro/Work/projects/playground/laravel
SP="<your scratchpad>"
PHPBIN="php85"
PEST="$PHPBIN $PG/vendor/bin/pest"
cd "$PG" || exit 1
pest() { eval "$PEST $*"; } # zsh: no word-splitting, must eval
GRAPHDIR="$(pest --baseline)"
GRAPH="$GRAPHDIR/graph.json"
mkdir -p "$SP/bin" && ln -sf "$(command -v php85)" "$SP/bin/php"
export PATH="$SP/bin:$PATH" # --shard spawns bare `php`
reset_tree() { git checkout -- tests app 2>/dev/null; git clean -qfd tests app 2>/dev/null; }
seed() { rm -rf "$GRAPHDIR"; pest --tia >/dev/null 2>&1; $PHPBIN "$SP/sentinel.php" "$GRAPH" >/dev/null; cp "$GRAPH" "$SP/before.json"; }
snap() { cp "$GRAPH" "$SP/before.json"; }
delta() { $PHPBIN "$SP/cmp.php" "$SP/before.json" "$GRAPH"; }
keys() { $PHPBIN -r '$g=json_decode(file_get_contents($argv[1]),true);echo "[".implode(",",array_keys($g["baselines"]??[]))."]";' "$GRAPH"; }
tally() { sed -E $'s/\x1b\\[[0-9;]*[a-zA-Z]//g' <"$SP/out.txt" | grep -E 'Tests:' | sed -E 's/^ +//;s/Tests: +//'; }
```
</details>
<details>
<summary><code>sentinel.php</code> — the write discriminator</summary>
```php
<?php // sentinel.php <graph.json>
$p = $argv[1];
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR);
$n = 0;
foreach ($g['baselines'] ?? [] as $br => $b) {
foreach (array_keys($b['results'] ?? []) as $id) {
$g['baselines'][$br]['results'][$id]['time'] = 9.999;
// Only falsify a non-zero assertion count: risky/skipped/incomplete are
// DERIVED from "performed no assertions", so patching those to 42 would
// rewrite the status on replay and destroy the discriminator.
if ((int) ($b['results'][$id]['assertions'] ?? 0) > 0) {
$g['baselines'][$br]['results'][$id]['assertions'] = 42;
}
$n++;
}
}
file_put_contents($p, json_encode($g, JSON_THROW_ON_ERROR));
echo "sentinelled $n results\n";
```
</details>
<details>
<summary><code>oneline.php</code> — one compact tier verdict per row</summary>
```php
<?php // oneline.php <before.json> <after.json>
function load(string $p): ?array {
return is_file($p) ? json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR) : null;
}
$a = load($argv[1]); $b = load($argv[2]);
if ($a === null || $b === null) { echo 'GRAPH '.($b === null ? 'DELETED' : 'CREATED'); exit; }
function edgeSets(array $g): array {
$out = [];
foreach ($g['edges'] ?? [] as $t => $ids) { $s = array_map(fn($i) => $g['files'][$i] ?? "?$i", (array) $ids); sort($s); $out[$t] = $s; }
ksort($out); return $out;
}
$moved = [];
if (edgeSets($a) !== edgeSets($b)) $moved[] = 'edges';
if (($a['files'] ?? []) !== ($b['files'] ?? [])) $moved[] = 'files';
if (($a['fingerprint'] ?? null) !== ($b['fingerprint'] ?? null)) $moved[] = 'fingerprint';
$brA = array_keys($a['baselines'] ?? []); $brB = array_keys($b['baselines'] ?? []);
if ($brA !== $brB) $moved[] = 'branchkeys('.implode('|', $brA).'->'.implode('|', $brB).')';
$add = $rem = $wr = 0; $shaMoved = $treeMoved = false;
foreach ($brB as $br) {
$ra = $a['baselines'][$br]['results'] ?? []; $rb = $b['baselines'][$br]['results'] ?? [];
if (($a['baselines'][$br]['sha'] ?? null) !== ($b['baselines'][$br]['sha'] ?? null)) $shaMoved = true;
if (($a['baselines'][$br]['tree'] ?? null) !== ($b['baselines'][$br]['tree'] ?? null)) $treeMoved = true;
$add += count(array_diff(array_keys($rb), array_keys($ra)));
$rem += count(array_diff(array_keys($ra), array_keys($rb)));
foreach ($ra as $id => $x) {
if (! isset($rb[$id])) continue;
foreach (['status','time','assertions','message'] as $f) {
if (($x[$f] ?? null) !== ($rb[$id][$f] ?? null)) { $wr++; break; }
}
}
}
$n = 0; foreach ($brB as $br) $n = max($n, count($b['baselines'][$br]['results'] ?? []));
printf('n=%d w=%-2d +%d -%d %s%s%s', $n, $wr, $add, $rem,
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
$shaMoved ? ' sha:CHANGED' : '', $treeMoved ? ' tree:chg' : '');
```
</details>
`cmp.php` (verbose per-entry version of the same) and `summarise.php` / `edgediff.php` are in
`PLAN_PHASE_TWO.md` §"Graph summariser" — reuse them for drill-downs. Reading the verdict:
- `w=` — entries actually **written**. Under sentinel patching this is the only reliable way to tell
"wrote identical values" from "wrote nothing". A full replay must give `w=0`.
- `struct:ok` + `+0 -0` — no prune, no edges/files/fingerprint movement. Required by RESULTS-ONLY.
- `STRUCT:branchkeys(...)` — a new baseline key appeared. For section L this is the headline signal.
### 4.4 Section L — new rows for this fix
Tiers, unchanged from phase two: **COMPLETE** may change everything · **RESULTS-ONLY (RO)** may
change only `baselines[<branch>].results` for tests that ran, and must never remove an entry, add a
result for a test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Every L row starts from a **cold graph recorded on the named default branch only** — verify
`keys=[<default>]` before branching. Restore branch state afterwards.
| # | Case | Target outcome |
|---|---|---|
| L1 | default `master`, record, `git switch -c feature-x`, `pest --tia`, zero changes | **all 25 replayed** (`w=0`), not `25 uncached`. The headline fix. |
| L2 | as L1 but default `main` | still all replayed — regression guard, this already worked |
| L3 | default `trunk`, then `develop` | replayed for both; the fix must not special-case two names |
| L4 | L1, then a *second* new branch `feature-y` | replayed too — the toll must not return per branch |
| L5 | L1 then edit `app/Services/Calculator.php` on `feature-x` | narrows to the 2 affected files (`CalculatorTest` + `AnnotationsTest`, which `covers` it); the other 17 replay |
| L6 | L1 with `--tia --filtered` | filtered mode reads the fallback too (`testFilesToRerun`, `hasUnlocatedTestsToRerun`) → `No affected tests found`, zero delta |
| L7 | L1 with `--tia --parallel` | fallback works in workers as well as the parent |
| L8 | L1, then `pest --tia` twice on `feature-x` | idempotent; second run also `w=0` |
| L9 | detached HEAD on a `master`-only graph | replays, and **no `main` key minted**`keys` stays `[master]`. Bug (b). |
| L10 | `pest()->tia()->defaultBranch('master')` in `tests/Pest.php`, repo default renamed away | config wins over autodetect |
| L11 | config set to a nonexistent branch (`defaultBranch('nope')`) | degrades to a full run; no crash, no phantom key |
| L12 | no remote at all (`git remote remove origin` if present) | still resolves (via `init.defaultBranch`) or degrades safely — must not throw |
| L13 | branch name with a slash (`feature/x/y`) | replayed; no key-splitting bugs |
| L14 | L1, then confirm writes | `feature-x` gets its **own** key; the `master` key is **not** written to (fallback is read-only) |
| L15 | git worktree on a new branch (the issue's scenario) | replays from the default-branch baseline |
| L16 | non-git dir, `pest --tia` | still `MissingDependency: The feature "Tia mode" requires "git".` — the soft resolver must not swallow it |
| L17 | non-git dir, plain `pest` | runs normally; no baseline dir created |
| L18 | count `git` subprocesses during one `--tia` run | default-branch resolution is cached, not one call per test. Probe by shimming `git` on `PATH` to a logging wrapper. |
L10L11 need a `tests/Pest.php` edit — that file is tracked and **outside** the `tests app` reset
scope in practice (it lives in `tests/`, so `git checkout -- tests` does restore it; verify with
`git status` after).
For L16/L17, build a throwaway non-git project — and note the trap that burned phase two: a
**symlinked** `vendor` makes Pest resolve the project root back to the playground (identical baseline
hash), silently invalidating the test. Use a hardlinked copy:
```bash
NG="$SP/nogit"; rm -rf "$NG"; mkdir -p "$NG"
cp -R composer.json composer.lock phpunit.xml artisan tests app bootstrap config routes resources storage "$NG/"
[ -f .env ] && cp .env "$NG/"
cp -Rl vendor "$NG/vendor" || cp -R vendor "$NG/vendor"
```
Confirm the baseline path differs (`nogit-<hash>`, not `laravel-4a455a95622ac0ec`), and delete both
the temp project and its `~/.pest/tia/nogit-*` dir afterwards.
### 4.5 Re-run the phase-two matrix (AK, 156 rows)
Re-run every row of `PLAN_PHASE_TWO.md` Part 2 against the fixed build. No row's status is trusted
until re-measured — the fix touches `Graph`'s read path, which nearly every row exercises. Sections
**A, B, H, I** are the load-bearing ones here (H is branch-key resolution; I is filtered mode; both
consume the changed methods directly). **H6 and H7 must be re-derived from a cold graph** (Part 1.4).
Most rows batch cheaply — phase two ran C1C20 in one call at roughly one line of output each. Use
`oneline.php` for the sweep and `cmp.php` only to drill into anomalies.
### 4.6 Known pre-existing failures — do not report as regressions
| item | status |
|---|---|
| **G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests. `mergeWorkerReplayPartials()` takes `$result['time']` verbatim at `Tia.php:1451`, never routing through `resultTime()` as the sequential sites (1681, 1762) do. Assertions survive because workers replay those themselves. | Pre-existing (pre-fix had no preservation at all), **out of scope for phase three**. Report it as still-present; do not fix it unless Nuno asks. |
| **C19**`--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
| **J11**`--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
| **J10**`--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
---
## Part 5 — Reporting
Per row: **tier respected (yes/no)**, the **graph delta** under sentinel patching, and for any failure
the **pre-fix contrast** so a regression is told apart from a pre-existing defect. For a pre-fix
contrast you need `db70017c` (or `bfd5b756` for "before phase three") files swapped into the
playground's `vendor/` — that is a sync, so **ask Nuno first** and always restore afterwards.
State which pest commit produced the numbers. Close with:
1. Whether L1L18 all pass (the fix works).
2. Whether AK regressed anywhere relative to phase two's 154/156.
3. The D1D4 decisions as implemented.
4. Anything still open — including G4, which will still be failing.
Leave the playground on `master` with only the four user-modified files dirty, no stray branches, and
no leftover `~/.pest/tia/*` dirs beyond `laravel-4a455a95622ac0ec`.
+466
View File
@@ -0,0 +1,466 @@
# TIA write-tier conformance — phase two
## Your task
**Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app** at
`/Users/nunomaduro/Work/projects/playground/laravel`. Every row, including the ones already marked
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
re-measured against the current code. The `Phase 1` column is prior evidence and a hint at what to
watch, never a reason to skip a row.
These are **not** the `pestphp/pest` repo's own tests (`composer test`) — do not run those. Each row is
a `pest` invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any
failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do **not**
stop at reading this file or summarising it; the deliverable is executed results.
Work in this order: **Part 1 (environment traps) → Part 1b (build the fixtures — the playground has
none of them) → Part 2 (the matrix) → Part 3 (priorities)**. The matrix is too large for one context;
take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are
the load-bearing ones — do those first if you run short.
Phase one implemented `PLAN.md` Part 1 items 14 plus three §5 items. This file turns the matrix into
a conformance check rather than a bug list.
Code under test: `pestphp/pest` at `/Users/nunomaduro/Work/projects/pestphp/pest`, branch
`fix/tia-filtered`, commit **`bfd5b756`** or later. Verify with
`grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php` → at least 3 hits. Pre-fix
baseline for every contrast is **`db70017c`**.
---
## Part 0 — What phase one changed
| # | Change | Files |
|---|---|---|
| 1 | `hasUnlocatedTestsToRerun()` stats the file, so a deleted test file is "unlocated" | `src/Plugins/Tia/Graph.php` |
| 2 | `enterReplayMode()` uses `activateLinkTracking()` under piggyback coverage | `src/Plugins/Tia.php` |
| 3 | `enterReplayMode()` stamps `TIA_PIGGYBACK_COVERAGE` for workers | `src/Plugins/Tia.php` |
| 4 | `replaceEdges(…, keepExisting:)` — piggyback edges seed empty sets, never overwrite populated ones | `Graph.php`, `Tia.php` |
| 5 | `renderFreshGraph()` stops claiming "fresh graph" when the graph is kept; reason reworded to `recording a coverage baseline` | `Tia.php` |
| 6 | `COVERAGE_REPORT_FLAGS` + `coverageReportActive()` union over `originalArguments`; new `pestCoverageActive()` keeps the coverage-cache marker/hijack on Pest's own `--coverage` | `Tia.php` |
| 7 | `Tia::recordsEdgesInWorkers()` + `WrapperRunner::handleTia()` inject `-d pcov.directory=<root>` into worker argv | `Tia.php`, `src/Plugins/Parallel/Paratest/WrapperRunner.php` |
| 8 | Sequential record runs announce structural drift via `renderFreshGraph()` | `Tia.php` |
| 9 | `Graph::getTime()` + `cachedTimeByTestId` + `resultTime()` preserve replayed durations; edge-less write guard is now `$recordsEdges = $complete && ($markKnownTestFiles \|\| $this->recordingActive)` | `Graph.php`, `Tia.php` |
Deliberately **not** done: `PLAN.md` §5 SIGINT propagation, §5 warning/deprecation `status=0`
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
### Target-outcome changes this forces
Two rows in the original matrix asserted the **old**, buggy behaviour. Their targets are updated
below — do not report them as regressions:
- **B10** was "`time` differs; statuses stable". Change 9 means replayed entries now **keep** their
recorded `time`. New target: `time` differs only for tests that actually executed.
- **B12** gains an edge-preservation assertion it never had (see K1).
### One known-failing repo test
`tests/Unit/Plugins/Tia/Graph.php:69-76` asserts `hasUnlocatedTestsToRerun('main')` is `false` for
`tests/Feature/FooTest.php` under `new Graph(sys_get_temp_dir())` — a path that does not exist. That
assertion encodes the I9 bug and **will fail** under change 1. It needs re-pointing at a root/file
that exists (e.g. `dirname(__DIR__, 4)` + `'tests/Unit/Plugins/Tia/Graph.php'`). Left untouched by
request; it is a repo-test matter, not a playground one.
---
## Part 1 — Environment, and the traps in it
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel`
```bash
cd /Users/nunomaduro/Work/projects/playground/laravel
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
```
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON
when it detects an agent.
2. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`
that a blanket reset would destroy. `PLAN.md`'s "reset ritual" is unsafe as written. Scope resets
to what you touched: `git checkout -- tests app` / `git clean -fd tests app`, and for
`composer.lock` (needed for the drift rows) **copy it aside and copy it back**, verifying with
`shasum`.
3. **`vendor/pestphp/pest` is a dist copy, not a symlink.** Composer installed
`dev-fix/tia-filtered as 5.2.0`, so edits in the pest repo do **not** reach the playground.
**Tell Nuno whenever a sync is needed to move forward — do not sync silently.** Say what is stale
and what the sync would be, then wait. This includes temporarily swapping in `db70017c` files for a
before/after contrast. When you report any playground result, state which commit produced it.
The sync itself, once he agrees:
```bash
PEST=/Users/nunomaduro/Work/projects/pestphp/pest
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \
src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; done
```
Verify with `grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"` → `1`. **Always restore the
current version before continuing** after a pre-fix contrast.
4. **Coverage driver:** pcov only, no xdebug. `ini_get('pcov.directory')` is `''` by default — that
emptiness is the entire mechanism behind G12.
5. **Suite shape as found:** 7 tests in 5 files — `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,
TrioTest}.php`, `tests/Feature/ExampleTest.php`. A healthy sequential graph is **`files=22`,
5 edge keys, self-edge on every test** (`PLAN.md`'s `files=25` is stale). Pre-fix parallel gives
`files=4` with zero self-edges. **These numbers shift the moment you add the Part 1b fixtures** —
re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants
that do *not* shift: sequential and parallel must agree, and every test must have a self-edge.
6. **Sentinel patching is the only reliable discriminator** between "wrote identical values" and
"wrote nothing": rewrite every cached result to `time=9.999 assertions=42`, snapshot, run the
case, diff. A canary test absent from `edges` is unreliable under `--filter` because it never
matches the filter and so never runs.
7. Seeding a cached failure needs `--fresh` (or an env-driven flaky fixture): `--tia` on a clean
green tree replays rather than executes, so it can never cache a failure. Working recipe — break
an assertion, `pest --tia --fresh`, then restore the source.
8. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
9. The shell is zsh — build commands with arrays or `eval`; unquoted `$args` does not word-split.
### Graph summariser
Write this to a scratch path and use it for every diff.
```php
<?php // summarise.php <graph.json> [label]
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
ksort($edges);
foreach ($edges as $test => $ids) {
$self = 'no';
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
}
foreach ($g['baselines'] ?? [] as $branch => $b) {
$r = $b['results'] ?? [];
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
foreach ($r as $id => $x) {
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
}
}
```
Normalised edge-set equality (for G12 / I5 / K1):
```php
<?php // edgediff.php <a.json> <b.json>
function edges(string $p): array {
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
ksort($out); return $out;
}
$a = edges($argv[1]); $b = edges($argv[2]);
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
foreach ($a as $t => $s) {
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
}
```
### Per-case loop
```bash
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
```
---
## Part 1b — Fixtures you must build first
**The playground has none of the fixtures the matrix depends on.** Verified inventory: the only test
files are `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php` and
`tests/Feature/ExampleTest.php` — 7 plain tests, zero occurrences of `->group()`, `->only()`,
`->skip()`, `->todo()`, `->note()`, `->flaky()`, `->covers()`, `->uses()`, `->issue()`, `->pr()`,
`->ticket()`, `->assignee()`, datasets, or any `FLAKY_OK`-style env hook. `phpunit.xml` defines only
the `Unit` and `Feature` testsuites.
So roughly 30 rows below cannot run as written until you create the fixtures. Build them **all up
front, in one commit**, then re-derive the baseline graph shape once and use those numbers for the
whole sweep — every fixture you add changes `files`, the `edges` key count and `n`, so adding them
piecemeal invalidates earlier rows.
| Fixture to create | Rows that need it |
|---|---|
| a test with `->group('smoke')` | C7, C8, C9, C10 |
| an env-driven flaky test (passes iff `FLAKY_OK=1`) | E1, E2, E3, E4 |
| a `->skip()`ed test | E9, D8 |
| a `->todo()` test | E10, C23, C39 |
| a risky test (no assertions; pair with `--disallow-test-output`) | E11, D7 |
| a test that triggers a PHPUnit warning | D6 |
| a test calling `markTestIncomplete()` | D9 |
| a test that triggers a deprecation | D10 |
| a dataset test with ≥3 rows | E13, E14 |
| `->only()` — added and removed per case, not left in | C31, C32, G7 |
| `->covers(App\Services\Calculator::class)` | C18 |
| `->uses(...)` / `UsesClass` annotation | C19 |
| `->note(...)` | C24 |
| `->flaky()` **annotation** (distinct from the env-driven flaky test above) | C25 |
| `->issue(123)`, `->pr(1)`, `->ticket('X')`, `->assignee('X')` | C26, C27, C28, C37, C38 |
This is `PLAN.md` §6's "test-harness gaps to close before re-running", now itemised: without these,
the listed rows' graph-invariant assertions match zero tests and **prove nothing** — they pass
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous *only because* its fixture is
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
Rows needing an *action* rather than a fixture — breaking `trio one` for the D rows, an uncommitted
test edit for `--dirty`, branch renames and a non-git dir for H5H10, `--mutate` against
`app/Services` for the F rows — are fine as written; `pestphp/pest-plugin-mutate` is installed.
---
## Part 2 — Full case matrix
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Phase-1 column: **VERIFIED** = re-run against the fixed code in the phase-one session, pre-fix
contrast captured · **PASS (sweep)** = passed in the original `db70017` sweep and *not* re-checked
since the changes — these are the bulk of phase two's work · **SKIP** = not runnable as written.
### A — Setup & sanity
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS (sweep) |
| A2 | `pest --baseline` | prints an existing dir; exit 0 | VERIFIED |
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | VERIFIED |
| A4 | `git status` after reset ritual | clean *except the four user-modified files* (see trap 2) | VERIFIED |
| A5 | `extension_loaded("pcov")` | `true` | VERIFIED |
| A6 | delete graph, plain `pest` | no graph created | PASS (sweep) |
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS (sweep) |
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | VERIFIED |
### B — COMPLETE runs still write
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | VERIFIED |
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS (sweep) — recheck under change 9 |
| B3 | `pest` (plain) | same as B2 | PASS (sweep) — recheck under change 9 |
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | VERIFIED |
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS (sweep) |
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS (sweep) |
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS (sweep) |
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS (sweep) |
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS (sweep) |
| B10 | `pest --tia` twice | **UPDATED TARGET:** `time` differs only for tests that executed; replayed entries keep their recorded `time`; statuses stable | VERIFIED |
| B11 | add a test file, `pest --tia` | new `edges` key **and** new result appear in the **same** run | VERIFIED (regression guard for change 9) |
| B12 | `pest --tia --coverage` | completes; graph written; coverage report prints | VERIFIED |
### C — Selection narrowing → RESULTS-ONLY
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS (sweep) |
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS (sweep) |
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS (sweep) |
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS (sweep) |
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS (sweep) |
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS (sweep) |
| C7 | `pest --group=smoke` | RO; no notice | PASS (sweep) |
| C8 | `pest --tia --group=smoke` | RO; notice | PASS (sweep) |
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS (sweep) |
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS (sweep) |
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS (sweep) |
| C12 | `pest --tia tests/Unit` | RO; notice | PASS (sweep) |
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS (sweep) |
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS (sweep) |
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS (sweep) |
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS (sweep) |
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS (sweep) |
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS (sweep) |
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS (sweep) |
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS (sweep) |
| C22 | `pest --tia --dirty` | RO; notice | PASS (sweep) |
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS (sweep) |
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS (sweep) |
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS (sweep) |
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS (sweep) |
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS (sweep) |
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS (sweep) |
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS (sweep) |
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
### D — Truncation → RESULTS-ONLY
D1D5, D11D13, D16 precondition: `trio one` broken. D6D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS (sweep) |
| D2 | `pest --retry` | RO; siblings survive | PASS (sweep) |
| D3 | `pest --stop-on-failure` | RO | PASS (sweep) |
| D4 | `pest --stop-on-defect` | RO | PASS (sweep) |
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS (sweep) |
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS (sweep) |
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS (sweep) |
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS (sweep) |
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS (sweep) |
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS (sweep) |
| D11 | `pest --tia --bail` | RO | PASS (sweep) |
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS (sweep) |
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS (sweep) |
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS (sweep) |
### E — Result merge semantics
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS (sweep) |
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS (sweep) |
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS (sweep) |
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS (sweep) |
| E5 | any partial run | `assertions` and `time` update for the test that ran (it executed, so change 9 does not apply) | PASS (sweep) |
| E6 | any partial run | `message` of untouched tests unchanged | PASS (sweep) |
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS (sweep) |
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS (sweep) |
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS (sweep) |
| E11 | partial run of a risky test | `status=5` | PASS (sweep) |
| E12 | any partial run | `fingerprint` byte-identical | PASS (sweep) |
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS (sweep) |
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS (sweep) |
### F — Guard rails
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | VERIFIED |
| F2 | complete `pest --tia` first, then the same filter | result is recorded | VERIFIED |
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS (sweep) |
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS (sweep) |
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS (sweep) |
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS (sweep) |
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS (sweep) |
### G — Parallel
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written; all 7 results present | VERIFIED |
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS (sweep) |
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS (sweep) |
| G4 | `pest --tia --parallel --bail` | RO | PASS (sweep) |
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | VERIFIED (see I5) |
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | VERIFIED |
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS (sweep) |
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS (sweep) |
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS (sweep) |
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS (sweep) |
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | VERIFIED |
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** — `files=22`, self-edge on all 5 tests, both | **VERIFIED (fixed)** — pre-fix: `files=4`, 0 self-edges |
### H — Baseline key / branch resolution
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| H1 | `pest --tia` on `master` | only a `master` key | VERIFIED |
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS (sweep) |
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS (sweep) |
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS (sweep) |
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS (sweep) |
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS (sweep) |
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS (sweep) |
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS (sweep) |
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS (sweep) |
### I — Filtered mode
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS (sweep) |
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | VERIFIED |
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS (sweep) |
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS (sweep) |
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **VERIFIED (fixed)** — 1 affected test, edges identical |
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS (sweep) |
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS (sweep) |
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report → full suite runs | **VERIFIED (fixed)** — pre-fix: `No affected tests found` |
| I9 | cached failure whose test file was deleted | WARN `Some cached tests due a re-run could not be located on disk` + `Running the full suite with replay instead of a filtered run` | **VERIFIED (fixed)** — pre-fix: `No tests found`, exit 0 green |
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS (sweep) |
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
| I12 | edit `composer.lock` | fingerprint drift → full rebuild, **and the reason is printed sequentially**: `fresh graph (composer.lock changed)` | **VERIFIED (fixed)** — pre-fix: bare `Running in TIA mode.` |
### J — Interactions & regressions
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS (sweep) |
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS (sweep) |
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS (sweep) |
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS (sweep) |
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
| J6 | `pest --tia --compact` | COMPLETE | PASS (sweep) |
| J7 | `pest --tia -v` | COMPLETE | PASS (sweep) |
| J8 | `pest --tia --profile` | COMPLETE | PASS (sweep) |
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS (sweep) |
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS (sweep) |
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS (sweep) |
| J13 | `pest --tia --min=50` | COMPLETE (silent no-op without `--coverage`) | PASS (sweep) |
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS (sweep) |
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS (sweep) |
### K — New rows for the phase-one fixes
These assert behaviour no original row covered. All were verified in phase one; re-run them as
regression guards.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| K1 | healthy `pest --tia` graph, then `pest --tia --coverage` | `edges` **byte-identical** — piggyback data may seed empty sets, never narrow populated ones; `files=22` stays `22` | **VERIFIED (fixed)** — pre-fix: `ExampleTest` 16→2 edges, self-edges lost |
| K2 | same run's headline | `Experimental TIA mode enabled / recording a coverage baseline.` — no false `fresh graph` | **VERIFIED (fixed)** |
| K3 | `pest --tia` ×3 on a clean tree | replayed entries keep their recorded `time` across all three | **VERIFIED (fixed)** — pre-fix: `0.053 → 0.001 → 0.001 → 0.001` |
| K4 | add a test file, one `pest --tia` (graph exists, fingerprint matches → replay+refresh) | result **and** edges appear in that same run; `n` grows by 1 | **VERIFIED** — regression guard for change 9 |
| K5 | add a test file, plain `pest --no-tia` | no result written for it; `n` unchanged; no edges | **VERIFIED (fixed)** — pre-fix: edge-less result inflated `n` |
| K6 | `pest --tia --parallel` worker argv | carries `-d pcov.directory=<projectRoot>`; `PEST_TIA` unset; not re-exec'd | **VERIFIED** — probe `bin/worker.php` |
| K7 | second `pest --tia --coverage` (cache primed → replay) | recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — **new work** |
| K8 | `pest --tia --parallel --coverage` | workers read `TIA_PIGGYBACK_COVERAGE`; no widened pcov scope; report intact | Not yet measured — **new work** |
| K9 | `pest --tia --filtered --coverage-html=<dir>` / `--coverage-clover=<file>` | filtered mode off, same as I8, for every flag in `COVERAGE_REPORT_FLAGS` | Not yet measured — **new work** |
---
## Part 3 — Priorities for phase two
0. **Part 1b fixtures** — nothing in C, D6D10, or E9E14 means anything until they exist.
1. **K7, K8, K9** — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
reasoned about but not observed; K9 covers the seven `COVERAGE_REPORT_FLAGS` beyond
`--coverage-text`.
2. **B2, B3, F4** and all of **D** and **E** — change 9 touched the shared result-write path, and
these are the rows that exercise it hardest. `$recordsEdges` is the thing to falsify: it must be
false for every partial and every non-recording run.
3. **F3F8** — change 7 injects a `-d` into worker argv; `--mutate --parallel` is the one place that
both spawns workers and must write nothing.
4. **H1H10** — untouched by these changes; cheapest bulk confirmation.
5. Raw PHPUnit coverage flags print **no report at all** in Pest, with or without TIA
(`--no-tia --coverage-text` is equally silent). Pre-existing, unrelated to change 6 — do not
chase it as a regression, but it means I8/K9 can only assert the selection half.
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
failure the pre-fix contrast (write `git show db70017c:<file>` into vendor, re-run, restore) so a
regression is told apart from a pre-existing defect.
+2 -1
View File
@@ -35,6 +35,7 @@ We cannot thank our sponsors enough for their incredible support in funding Pest
- **[SerpApi](https://serpapi.com/?ref=nunomaduro)**
- **[Typesense](https://typesense.org/?ref=nunomaduro)**
- **[Bento](https://bentonow.com/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/)**
- **[Pixel](https://wearepixel.com.au/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/?ref=nunomaduro)**
Pest is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.
+7 -7
View File
@@ -18,19 +18,19 @@
],
"require": {
"php": "^8.4",
"brianium/paratest": "^7.23.0",
"brianium/paratest": "^7.23.1",
"nunomaduro/collision": "^8.9.5",
"nunomaduro/termwind": "^2.4.0",
"pestphp/pest-plugin": "^5.0.0",
"pestphp/pest-plugin-arch": "^5.0.0",
"pestphp/pest-plugin-mutate": "^5.0.0",
"pestphp/pest-plugin-mutate": "^5.0.1",
"pestphp/pest-plugin-profanity": "^5.0.0",
"phpunit/phpunit": "^13.2.4",
"phpunit/phpunit": "^13.2.6",
"symfony/process": "^8.1.0"
},
"conflict": {
"filp/whoops": "<2.18.3",
"phpunit/phpunit": ">13.2.4",
"phpunit/phpunit": ">13.2.6",
"sebastian/exporter": "<7.0.0",
"webmozart/assert": "<1.11.0"
},
@@ -48,6 +48,7 @@
"Tests\\Fixtures\\Covers\\": "tests/Fixtures/Covers",
"Tests\\Fixtures\\Inheritance\\": "tests/Fixtures/Inheritance",
"Tests\\Fixtures\\Arch\\": "tests/Fixtures/Arch",
"Tests\\Fixtures\\Tia\\": "tests/Fixtures/Tia",
"Tests\\": "tests/PHPUnit/"
},
"classmap": [
@@ -58,12 +59,11 @@
]
},
"require-dev": {
"laravel/pao": "^1.1.2",
"pestphp/pest-dev-tools": "^5.0.0",
"pestphp/pest-plugin-browser": "^5.0.0",
"pestphp/pest-plugin-phpstan": "^5.0.0",
"pestphp/pest-plugin-rector": "^5.0.0",
"pestphp/pest-plugin-type-coverage": "^5.0.0",
"pestphp/pest-plugin-rector": "^5.0.2",
"pestphp/pest-plugin-type-coverage": "^5.0.2",
"psy/psysh": "^0.12.24"
},
"minimum-stability": "dev",
+2 -2
View File
@@ -90,11 +90,11 @@ final class TestSuiteLoader
(static function () use ($suiteClassFile) {
try {
include_once $suiteClassFile;
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
} catch (Throwable $e) {
Panic::with($e);
}
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
})();
$loadedClasses = array_values(
+2 -1
View File
@@ -18,8 +18,9 @@
<directory suffix=".php">./tests</directory>
<directory suffix=".php">./tests-external</directory>
<exclude>./tests/.snapshots</exclude>
<exclude>./tests/.tests</exclude>
<exclude>./tests/Fixtures/Inheritance</exclude>
<exclude>./tests/Fixtures/Suites</exclude>
<exclude>./tests/Fixtures/Tia</exclude>
</testsuite>
</testsuites>
<source>
+5
View File
@@ -0,0 +1,5 @@
{
"exclude": [
"tests/Fixtures/Suites"
]
}
+3
View File
@@ -28,6 +28,9 @@ return RectorConfig::configure()
->withSkip([
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
__DIR__.'/tests/Fixtures/Arch',
// Fixture suites are pinned by the TeamCity / JUnit snapshots, down to
// the line numbers — rewriting their source would break them.
__DIR__.'/tests/Fixtures/Suites',
ReturnNeverTypeRector::class,
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
NarrowObjectReturnTypeRector::class,
+7 -2
View File
@@ -280,12 +280,17 @@ trait Testable
/** @var Tia $tia */
$tia = Container::getInstance()->get(Tia::class);
$status = $tia->getStatus(self::$__filename, $this::class.'::'.$this->name());
$status = $tia->getStatus(self::$__filename, $this->valueObjectForEvents()->id());
$replay = ReplayType::fromStatus($status);
if ($replay !== ReplayType::None) {
assert($status !== null);
// Marks the replay before the branches below throw, so `tearDown`
// short-circuits for every replayed result — the throwing branches
// never reach `parent::setUp`, so no user hook may run after them.
$this->__replay = $replay;
match ($replay) {
ReplayType::Pass, ReplayType::Risky => $this->__beginReplay($replay, $tia),
ReplayType::Skipped => $this->markTestSkipped($status->message()),
@@ -319,7 +324,7 @@ trait Testable
private function __beginReplay(ReplayType $replay, Tia $tia): void
{
$this->__replay = $replay;
$this->__replayAssertions = $tia->getAssertionCount($this::class.'::'.$this->name());
$this->__replayAssertions = $tia->getAssertionCount($this->valueObjectForEvents()->id());
$this->__ran = true;
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllAlreadyExist extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllWithinDescribe extends InvalidArgumentException implements E
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterBeforeTestFunction extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct('After method cannot be used with before the [test|it] functions in the filename `['.$filename.']`.');
parent::__construct('After method cannot be used with before the [test|it] functions in the filename ['.$filename.'].');
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllAlreadyExist extends InvalidArgumentException implements Ex
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllWithinDescribe extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetAlreadyExists extends InvalidArgumentException implements Exc
*/
public function __construct(string $name, string $scope)
{
parent::__construct(sprintf('A dataset with the name `%s` already exists in scope [%s].', $name, $scope));
parent::__construct(sprintf('A dataset with the name [%s] already exists in scope [%s].', $name, $scope));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetDoesNotExist extends InvalidArgumentException implements Exce
*/
public function __construct(string $name)
{
parent::__construct(sprintf("A dataset with the name `%s` does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
parent::__construct(sprintf("A dataset with the name [%s] does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class FileOrFolderNotFound extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The file or folder with the name `%s` could not be found.', $filename));
parent::__construct(sprintf('The file or folder with the name [%s] could not be found.', $filename));
}
}
+41
View File
@@ -0,0 +1,41 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use InvalidArgumentException;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Symfony\Component\Console\Exception\ExceptionInterface;
/**
* @internal
*/
final class InvalidTestClassName extends InvalidArgumentException implements ExceptionInterface, RenderlessEditor, RenderlessTrace
{
/**
* Creates a new Exception instance for the given class name.
*/
public static function fromClassName(string $filename, string $className): self
{
return new self(sprintf(
'The test file [%s] would create the class [%s], which is not a valid PHP class name. Please rename the test file.',
$filename,
$className,
));
}
/**
* Creates a new Exception instance for the given namespace.
*/
public static function fromNamespace(string $filename, string $namespace, string $part): self
{
return new self(sprintf(
'The test file [%s] would create the namespace [%s], which is not a valid PHP namespace, as [%s] may not be used as a namespace name. Please rename the folder in question.',
$filename,
$namespace,
$part,
));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestAlreadyExist extends InvalidArgumentException implements Excepti
*/
public function __construct(string $fileName, string $description)
{
parent::__construct(sprintf('A test with the description `%s` already exists in the filename `%s`.', $description, $fileName));
parent::__construct(sprintf('A test with the description [%s] already exists in the filename [%s].', $description, $fileName));
}
}
@@ -19,6 +19,6 @@ final class TestCaseClassOrTraitNotFound extends InvalidArgumentException implem
*/
public function __construct(string $testCaseClass)
{
parent::__construct(sprintf('The class `%s` was not found.', $testCaseClass));
parent::__construct(sprintf('The class [%s] was not found.', $testCaseClass));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestDescriptionMissing extends InvalidArgumentException implements E
*/
public function __construct(string $fileName)
{
parent::__construct(sprintf('Test description is missing in the filename `%s`.', $fileName));
parent::__construct(sprintf('Test description is missing in the filename [%s].', $fileName));
}
}
@@ -0,0 +1,52 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresDefaultBranch extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct()
{
parent::__construct(
'Tia mode could not determine the default branch every other branch falls back to reading.',
);
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode could not determine the default branch.',
'',
' It is the branch whose baseline every other branch falls back to reading, and',
' nothing in this checkout names it: the repository has a remote, but no',
' <fg=yellow>origin/HEAD</>, and no CI provider stated it either. Guessing would re-run the',
' whole suite on every new branch while reporting it as a cache hit.',
'',
' Name the branch in <fg=yellow>tests/Pest.php</>:',
'',
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
'',
' Or let git answer, once per clone:',
'',
' <fg=yellow>git remote set-head origin --auto</>',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
+47
View File
@@ -0,0 +1,47 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresRemote extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct()
{
parent::__construct(
'Tia mode requires a repository with a remote, so the default branch every other branch falls back to reading can be resolved.',
);
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires a repository with a remote.',
'',
' Without one there is no way to tell which branch is the default — the branch',
' whose baseline every other branch falls back to reading — so the first run on',
' each new branch would silently re-run the whole suite.',
'',
' Add a remote, or name the branch yourself in <fg=yellow>tests/Pest.php</>:',
'',
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
+11
View File
@@ -9,6 +9,7 @@ use Pest\Concerns;
use Pest\Contracts\HasPrintableTestCaseName;
use Pest\Evaluators\Attributes;
use Pest\Exceptions\DatasetMissing;
use Pest\Exceptions\InvalidTestClassName;
use Pest\Exceptions\ShouldNotHappen;
use Pest\Exceptions\TestAlreadyExist;
use Pest\Exceptions\TestClosureMustNotBeStatic;
@@ -138,6 +139,16 @@ final class TestCaseFactory
if (trim($className) === '') {
$className = 'InvalidTestName'.Str::random();
} elseif (! Str::isValidClassName($className)) {
throw InvalidTestClassName::fromClassName($this->filename, $className);
}
if ($this->namespace === null) {
foreach ($partsFQN as $partFQN) {
if (! Str::isValidIdentifier($partFQN)) {
throw InvalidTestClassName::fromNamespace($this->filename, $namespace, $partFQN);
}
}
}
$this->attributes = [
+1 -1
View File
@@ -6,7 +6,7 @@ namespace Pest;
function version(): string
{
return '5.0.0-beta.1';
return '5.0.3';
}
function testDirectory(string $file = ''): string
+35
View File
@@ -114,6 +114,41 @@ final readonly class Help implements HandlesArguments
],
];
$content['Tia'] = [
[
'arg' => '--tia',
'desc' => 'Re-run only the tests affected by your changes, replaying the rest from cache',
],
[
'arg' => '--no-tia',
'desc' => 'Disable test impact analysis for this run',
],
[
'arg' => '--tia --fresh',
'desc' => 'Discard the recorded dependency graph and record it again',
],
[
'arg' => '--tia --filtered',
'desc' => 'Narrow the run to the affected test files only',
],
[
'arg' => '--tia --locally',
'desc' => 'Enable test impact analysis on local machines only',
],
[
'arg' => '--tia --baselined',
'desc' => 'Fetch the shared dependency graph recorded by the CI baseline',
],
[
'arg' => '--tia --refetch',
'desc' => 'Force a fresh fetch of the shared dependency graph',
],
[
'arg' => '--baseline',
'desc' => 'Output to standard output the test impact analysis storage directory',
],
];
$content['Execution'] = [...[
[
'arg' => '--parallel',
@@ -14,6 +14,7 @@ use ParaTest\RunnerInterface;
use ParaTest\WrapperRunner\MissingResultsException;
use ParaTest\WrapperRunner\SuiteLoader;
use ParaTest\WrapperRunner\WrapperWorker;
use Pest\Plugins\Tia;
use Pest\Result;
use Pest\TestSuite;
use PHPUnit\Event\Facade as EventFacade;
@@ -155,6 +156,7 @@ final class WrapperRunner implements RunnerInterface
/** @var array<int, non-empty-string> $parameters */
$parameters = $this->handleLaravelHerd($parameters);
$parameters = $this->handleTia($parameters);
$parameters[] = $wrapper;
$parameters[] = '--test-directory='.TestSuite::getInstance()->testPath;
@@ -202,6 +204,28 @@ final class WrapperRunner implements RunnerInterface
return $parameters;
}
/**
* Widens pcov's instrumentation scope to the whole project for workers that
* record TIA edges.
*
* pcov's default scope is a single source directory it auto-detects, so
* `config/`, `routes/`, `bootstrap/` and every test's own file never reach
* the recorder — a worker-recorded graph selects a fraction of what a
* sequential one does. `pcov.directory` is only settable at startup, hence
* the command line rather than an `ini_set()` inside the worker.
*
* @param array<int, non-empty-string> $parameters
* @return array<int, non-empty-string>
*/
private function handleTia(array $parameters): array
{
if (! Tia::recordsEdgesInWorkers()) {
return $parameters;
}
return array_merge($parameters, ['-d', 'pcov.directory='.TestSuite::getInstance()->rootPath]);
}
private function startWorkers(): void
{
for ($token = 1; $token <= $this->options->processes; $token++) {
+574 -25
View File
@@ -7,13 +7,19 @@ namespace Pest\Plugins;
use NunoMaduro\Collision\Adapters\Phpunit\Printers\DefaultPrinter;
use Pest\Contracts\Plugins\AddsOutput;
use Pest\Contracts\Plugins\HandlesArguments;
use Pest\Contracts\Plugins\HandlesOriginalArguments;
use Pest\Contracts\Plugins\Terminable;
use Pest\Exceptions\InvalidOption;
use Pest\Exceptions\MissingDependency;
use Pest\Exceptions\NoAffectedTestsFound;
use Pest\Exceptions\TiaRequiresDefaultBranch;
use Pest\Exceptions\TiaRequiresRemote;
use Pest\Exceptions\TiaRequiresRepositoryRoot;
use Pest\Panic;
use Pest\Plugins\Concerns\HandleArguments;
use Pest\Plugins\Tia\BaselineSync;
use Pest\Plugins\Tia\ChangedFiles;
use Pest\Plugins\Tia\CiDefaultBranch;
use Pest\Plugins\Tia\Contracts\State;
use Pest\Plugins\Tia\CoverageCollector;
use Pest\Plugins\Tia\Fingerprint;
@@ -30,13 +36,14 @@ use Pest\Support\View;
use Pest\TestCaseFilters\TiaTestCaseFilter;
use Pest\TestSuite;
use PHPUnit\Framework\TestStatus\TestStatus;
use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Process\Process;
/**
* @internal
*/
final class Tia implements AddsOutput, HandlesArguments, Terminable
final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArguments, Terminable
{
use HandleArguments;
@@ -56,6 +63,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private const string BASELINE_PATH_OPTION = '--baseline';
/**
* Set by the mutation plugin on the subprocess running a single mutant,
* and nowhere else. Its own `--mutate` flag is popped before the argv is
* handed to that subprocess, so the flag cannot be matched instead.
*/
private const string ENV_MUTATION_TESTING = 'PEST_MUTATION_TESTING';
private const string ENV_TIA = 'PEST_TIA';
private const string ENV_FILTERED = 'PEST_TIA_FILTERED';
@@ -88,6 +102,21 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private const string PIGGYBACK_COVERAGE_GLOBAL = 'TIA_PIGGYBACK_COVERAGE';
/**
* The parent's resolved fallback branch, handed to the workers.
*
* A worker cannot resolve it for itself: the restarters run before
* `tests/Pest.php` is loaded, so a `defaultBranch()` declared there is
* invisible to it — and autodetecting again would spend a git call per
* worker to reach the answer the parent already has.
*/
private const string FALLBACK_BRANCH_GLOBAL = 'TIA_FALLBACK_BRANCH';
/**
* The branch assumed when a repository cannot name its own default.
*/
private const string DEFAULT_BRANCH = 'main';
/**
* PHPUnit/Pest CLI flags whose subsequent argument is a value, not a path.
*
@@ -101,12 +130,76 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
'--include-path', '--whitelist',
'--log-junit', '--log-teamcity', '--testdox-html', '--testdox-text',
'--coverage-clover', '--coverage-cobertura', '--coverage-crap4j',
'--coverage-html', '--coverage-php', '--coverage-text', '--coverage-xml',
'--coverage-html', '--coverage-openclover', '--coverage-php',
'--coverage-text', '--coverage-xml',
'--coverage-filter', '--path-coverage',
'--repeat', '--retry-times', '--memory-limit', '--seed',
'--compact', '--ci-build-id', '--min',
];
/**
* PHPUnit flags that make this run produce a coverage report.
*
* Pest's own `--coverage` is tracked by the Coverage plugin, but a raw
* PHPUnit report flag never reaches it. A run that reports coverage must
* not be narrowed to the affected tests — the report would then describe a
* subset of the suite — and must let PHPUnit own the coverage driver rather
* than have the TIA recorder clear it mid-collection.
*
* Flags that only shape collection or an existing report — `--coverage-filter`,
* `--path-coverage`, `--warm-coverage-cache`, `--only-summary-for-coverage-text`,
* `--show-uncovered-for-coverage-text`, `--disable-coverage-ignore` — produce no
* report on their own, so they are deliberately absent.
*
* @var list<string>
*/
private const array COVERAGE_REPORT_FLAGS = [
'--coverage-clover', '--coverage-cobertura', '--coverage-crap4j',
'--coverage-html', '--coverage-openclover', '--coverage-php',
'--coverage-text', '--coverage-xml',
];
/**
* Flags that narrow this run to a subset of the suite.
*
* Only user-supplied, per-run narrowing belongs here. A filter that is
* always in force — `<groups>` in phpunit.xml, or a plugin registering a
* test case filter from `boot()` — applies equally to the runs that build
* the baseline, so it does not make this run narrower than the baseline
* and must not disable baseline writes.
*
* `--shard` is rewritten to `--filter` before this plugin sees the
* arguments, so it is covered here too. The `bin/pest`-only flags are
* stripped from the handled arguments, so they are matched against the
* original argv instead.
*
* Flags that cut a run short instead of narrowing it — `--bail`, `--retry`,
* `--stop-on-*` — do not belong here either. They only narrow the run when
* something actually fails, and that is not known until it is over, so they
* are handled by stoppedEarly() from addOutput().
*
* @var list<string>
*/
private const array PARTIAL_SELECTION_FLAGS = [
'--filter', '--exclude-filter', '--group', '--exclude-group',
'--covers', '--uses', '--testsuite', '--exclude-testsuite', '--test-suffix',
'--dirty', '--todo', '--todos', '--flaky', '--notes',
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
];
/**
* Options that cannot be combined with Tia mode.
*
* `--covers` and `--uses` select on coverage metadata Tia does not model,
* so they resolve to no tests at all rather than to the ones the user meant.
* `--random-order-seed` exits non-zero on its own, with or without Tia.
* Either way the run cannot honour both things it was asked for, so it says
* so instead of silently dropping Tia and running something else.
*/
private const array UNSUPPORTED_OPTIONS = [
'--covers', '--uses', '--random-order-seed',
];
private bool $graphWritten = false;
private bool $replayRan = false;
@@ -120,9 +213,50 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
/** @var array<string, int> */
private array $cachedAssertionsByTestId = [];
/**
* Recorded durations of the tests this run replayed rather than executed.
*
* A replayed test never runs, so the duration PHPUnit reports for it is the
* cost of replaying it — near zero. Writing that back would decay every
* cached timing toward zero one run at a time.
*
* @var array<string, float>
*/
private array $cachedTimeByTestId = [];
private ?Graph $replayGraph = null;
private string $branch = 'main';
/**
* The baseline this run reads from and writes to.
*
* The repository's default branch is only the fallback for a checkout whose
* branch cannot be read — a detached HEAD. It is also the branch every
* other baseline falls back to reading, so writing there by accident
* corrupts the shared baseline. Resolved through resolveBranch() rather
* than at every use site, because the git call it needs is not free.
*/
private string $branch = self::DEFAULT_BRANCH;
/**
* The baseline branches with none of their own read from.
*
* Read-only, and the whole point of the exercise: without it the first run
* on every new branch re-runs a suite whose results the default branch
* already holds.
*/
private string $fallbackBranch = self::DEFAULT_BRANCH;
/**
* Whether anything actually named the branch above.
*
* When nothing did, the value is a guess, and a guess is what the TIA path
* refuses to run on: an unresolved fallback reads no baseline at all, which
* looks exactly like a hit in the output. Runs that never asked for TIA
* still have to write somewhere, so the guess stands for them.
*/
private bool $fallbackBranchResolved = false;
private bool $branchResolved = false;
/** @var array<string, true> */
private array $affectedFiles = [];
@@ -142,6 +276,27 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private bool $filteredMode = false;
/**
* Bars this run from touching the graph at all, results included.
*
* Reserved for runs whose results describe something other than the code
* in the working tree, which is nothing the baseline can ever use.
*/
private bool $writesSuppressed = false;
/**
* Narrows this run's writes to the results of the tests it actually ran.
*
* A run that covered only part of the suite still learns something true
* about the tests it did reach. What it cannot do is speak for the rest:
* pruning results, advancing the recorded sha and replacing the edge map
* all claim the whole suite reported, so they stay behind a complete run.
*/
private bool $resultsOnlyWrites = false;
/** @var array<int, string> */
private array $originalArguments = [];
private ?string $driftLabel = null;
private ?string $driftDetails = null;
@@ -185,7 +340,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return null;
}
return Graph::decode($json, $projectRoot);
$graph = Graph::decode($json, $projectRoot);
// Every read of a baseline goes through a graph loaded here, so this is
// the one place the resolved fallback has to reach.
$graph?->setFallbackBranch($this->fallbackBranch);
return $graph;
}
private function saveGraph(Graph $graph): bool
@@ -227,6 +388,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return ! self::argumentPresent('--ci', $arguments);
}
/**
* Whether the workers of this run record their own coverage edges.
*
* Stamped by the parent before paratest spawns anything, because a worker
* cannot tell on its own: its argv carries no `--tia`, and the restarters
* run before `tests/Pest.php` is loaded, so {@see self::isEnabledForRun()}
* sees an empty {@see WatchPatterns} too. Left unanswered, pcov keeps its
* default scope — a single auto-detected source directory — and every edge
* outside it, test self-edges included, is silently dropped.
*
* Piggyback runs are excluded: their edges come from PHPUnit's own coverage
* session, so widening pcov there costs time and buys nothing.
*/
public static function recordsEdgesInWorkers(): bool
{
return (string) Parallel::getGlobal(self::RECORDING_GLOBAL) === '1'
&& (string) Parallel::getGlobal(self::PIGGYBACK_COVERAGE_GLOBAL) !== '1';
}
/**
* @param array<int, string> $arguments
*/
@@ -305,6 +485,12 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$this->replayedCount++;
$assertions = $this->replayGraph->getAssertions($this->branch, $testId);
$this->cachedAssertionsByTestId[$testId] = $assertions ?? 0;
$time = $this->replayGraph->getTime($this->branch, $testId);
if ($time !== null) {
$this->cachedTimeByTestId[$testId] = $time;
}
} else {
$this->executedCount++;
}
@@ -317,6 +503,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $this->cachedAssertionsByTestId[$testId] ?? 0;
}
/**
* {@inheritDoc}
*/
public function handleOriginalArguments(array $arguments): void
{
$this->originalArguments = $arguments;
}
/**
* {@inheritDoc}
*/
@@ -339,9 +533,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$cliEnabled = $this->hasArgument(self::OPTION, $arguments) || self::envFlagEnabled(self::ENV_TIA);
$alwaysEnabled = $watchPatterns->isEnabled()
&& (! $watchPatterns->isLocally() || Environment::name() === Environment::LOCAL);
if (! $isWorker && ! $disabled && ($cliEnabled || $alwaysEnabled)) {
$this->guardUnsupportedOptions($arguments);
}
$hasExplicitPath = $this->hasExplicitPathArgument($arguments);
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
$disabled = $disabled || $partial;
// A mutation subprocess runs the suite against source the mutation
// plugin has deliberately broken. Its failures describe the mutant, not
// the working tree, so unlike every other narrowed run there is nothing
// in its results worth keeping. The parent `--mutate` run is untouched
// by this: it runs the whole suite against real source.
if (getenv(self::ENV_MUTATION_TESTING) !== false) {
$this->writesSuppressed = true;
}
$enabled = ! $disabled && ($cliEnabled || $alwaysEnabled);
$this->filteredMode = ($this->hasArgument(self::FILTERED_OPTION, $arguments) || self::envFlagEnabled(self::ENV_FILTERED) || $watchPatterns->isFiltered())
&& ! $this->hasExplicitPathArgument($arguments)
&& ! $hasExplicitPath
&& ! $this->coverageReportActive();
$freshRequested = $this->hasArgument(self::FRESH_OPTION, $arguments);
$this->forceRefetch = $this->hasArgument(self::REFETCH_OPTION, $arguments);
@@ -355,6 +565,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$arguments = $this->popArgument(self::BASELINED_OPTION, $arguments);
if ($disabled) {
if ($partial) {
// TIA cannot choose what runs here — the user already did — but
// the tests they picked still report honestly, so their results
// are kept and everything that would speak for the excluded ones
// is not. `--no-tia` needs none of this: it still runs the whole
// suite, so it remains a complete run.
$this->resultsOnlyWrites = true;
// `$this->filteredMode` counts as asking for it: reaching here
// means the narrowing came from the command line while filtered
// mode came from the environment or the config, and a run that
// silently declines what the config asked for is the one most
// in need of the explanation.
if ($cliEnabled || $freshRequested || $this->forceRefetch || $this->filteredMode) {
$this->output->writeln('');
$this->renderChild('TIA does not apply to partial runs — running the selected tests directly.');
}
}
$this->forceRefetch = false;
$this->filteredMode = false;
$this->freshRebuild = false;
@@ -392,6 +621,18 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$this->flushWorkerReplay();
}
// Both only ever set for the parent — addOutput() returns early in
// workers, whose partials are ephemeral and only reach the baseline if
// the parent consumes them. Everything this method goes on to write is
// whole-suite by nature — the edge map above all — so a narrowed run
// stops here too, its results already persisted by addOutput().
if ($this->writesSuppressed || $this->resultsOnlyWrites) {
$this->recorder->reset();
$this->coverageCollector->reset();
return;
}
$recorder = $this->recorder;
if (! $this->recordingActive && ! $recorder->isActive()) {
@@ -453,7 +694,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$this->branch,
$changedFiles->snapshotTree($changedFiles->since($currentSha) ?? []),
);
$graph->replaceEdges($perTest);
$graph->replaceEdges($perTest, keepExisting: $this->piggybackCoverage);
$graph->replaceTestTables($perTestTables);
$graph->replaceTestInertiaComponents($perTestInertia);
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
@@ -481,12 +722,34 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $exitCode;
}
// `->only()` narrows the executed set exactly like `--filter` does, but
// is only knowable once the suite has been collected — too late to turn
// TIA off up front. Sampled in addOutput() because Only's lock file is
// already gone by the time terminate() runs (its plugin terminates
// first). Whether the run was cut short is likewise only knowable now.
if (Only::isEnabled() || $this->stoppedEarly()) {
$this->resultsOnlyWrites = true;
}
$this->reportMissingWorkerDrivers();
// Runs before the checks below: it is what fills the parent's result
// collector in parallel, and a worker that stopped early narrows the
// whole run.
if (Parallel::isEnabled()) {
$this->mergeWorkerReplayPartials();
}
if ($this->writesSuppressed) {
return $exitCode;
}
if ($this->resultsOnlyWrites) {
$this->snapshotTestResults(complete: false);
return $exitCode;
}
if ($this->replayRan) {
$this->bumpRecordedSha();
}
@@ -547,7 +810,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $exitCode;
}
$graph->replaceEdges($finalised);
$graph->replaceEdges($finalised, keepExisting: $this->piggybackCoverage);
$graph->replaceTestTables($finalisedTables);
$graph->replaceTestInertiaComponents($finalisedInertia);
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
@@ -635,7 +898,19 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
Panic::with(new TiaRequiresRepositoryRoot($subdirectoryPrefix));
}
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
$this->resolveBranch($projectRoot);
// After resolveBranch(), so a directory that is no repository at all
// still reports the missing git dependency rather than an unresolved
// default branch. Nothing named the branch every other baseline reads
// through, so every new branch would re-run the whole suite while the
// output called it a hit. A repository with no remote is the likeliest
// reason and gets said out loud.
if (! $this->fallbackBranchResolved) {
Panic::with(new ChangedFiles($projectRoot)->hasRemote()
? new TiaRequiresDefaultBranch
: new TiaRequiresRemote);
}
$fingerprint = Fingerprint::compute($projectRoot);
$this->startFingerprint = $fingerprint;
@@ -673,13 +948,20 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
}
}
if ($this->piggybackCoverage) {
// Both of these belong to the coverage cache, which only Pest's own
// `--coverage` ever writes or merges. A raw PHPUnit report flag takes
// the piggyback path — it must not drive the driver itself — but must
// not leave a marker behind, nor force a recording run to prime a cache
// that nothing on its path will fill.
$coverageCacheOwned = $this->piggybackCoverage && $this->pestCoverageActive();
if ($coverageCacheOwned) {
$this->state->write(self::KEY_COVERAGE_MARKER, '');
}
if ($this->piggybackCoverage && ! $this->state->exists(self::KEY_COVERAGE_CACHE)) {
if ($coverageCacheOwned && ! $this->state->exists(self::KEY_COVERAGE_CACHE)) {
if ($graph instanceof Graph && $this->driftLabel === null) {
$this->freshGraphReason = 'recording coverage baseline';
$this->freshGraphReason = 'recording a coverage baseline';
}
return $this->enterRecordMode($arguments);
@@ -698,7 +980,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
*/
private function handleWorker(array $arguments, string $projectRoot, bool $recordingGlobal, bool $replayingGlobal): array
{
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
$this->resolveBranch($projectRoot);
if ($replayingGlobal) {
$this->installWorkerReplay($projectRoot);
@@ -875,7 +1157,15 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
if (! Parallel::isEnabled()) {
if ($canRefreshReplayEdges) {
$this->recorder->activate();
// Piggyback runs read PHPUnit's own coverage session. Driving
// the driver alongside it would clear the data PHPUnit is about
// to read, so only link tracking may run here.
if ($this->piggybackCoverage) {
$this->recorder->activateLinkTracking();
} else {
$this->recorder->activate();
}
$this->recordingActive = true;
}
@@ -894,6 +1184,10 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
if ($canRefreshReplayEdges) {
Parallel::setGlobal(self::RECORDING_GLOBAL, '1');
if ($this->piggybackCoverage) {
Parallel::setGlobal(self::PIGGYBACK_COVERAGE_GLOBAL, '1');
}
}
if ($this->filteredMode) {
@@ -1035,6 +1329,16 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$recorder->activate();
$this->recordingActive = true;
// Why this run is rebuilding is worth saying whenever there is a reason
// for it — the parallel and piggyback branches above already do. Runs
// that are simply recording for the first time have nothing to explain.
if ($this->driftLabel !== null || $this->freshGraphReason !== null) {
$this->output->writeln('');
$this->renderFreshGraph();
return $arguments;
}
$this->renderChild('Running in TIA mode.');
return $arguments;
@@ -1042,14 +1346,18 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private function renderFreshGraph(): void
{
$headline = 'Experimental TIA mode enabled / fresh graph';
if ($this->driftLabel !== null) {
$headline .= sprintf(' (%s changed)', $this->driftLabel);
} elseif ($this->freshGraphReason !== null) {
$headline .= sprintf(' (%s)', $this->freshGraphReason);
if ($this->driftLabel === null && $this->freshGraphReason !== null) {
// The reason is only ever set for a run that keeps its graph and
// records alongside it, so "fresh graph" would be a lie here.
$headline = sprintf('Experimental TIA mode enabled / %s.', $this->freshGraphReason);
} else {
$headline .= '.';
$headline = 'Experimental TIA mode enabled / fresh graph';
if ($this->driftLabel !== null) {
$headline .= sprintf(' (%s changed)', $this->driftLabel);
} else {
$headline .= '.';
}
}
$this->renderChild($headline);
@@ -1065,7 +1373,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
{
$this->output->writeln('');
$this->renderChild('Running in TIA mode, however TIA as skipped as it needs Needs ext-pcov or Xdebug.');
$this->renderChild('Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug.');
}
/**
@@ -1136,11 +1444,24 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return;
}
// A replayed result carries the duration PHPUnit measured for a test
// that never ran — near zero. Unlike the assertion count, which the
// replay injects into the result itself, the cached duration lives only
// in this process: the parent replayed nothing of its own, so once the
// partial is written the real value is unrecoverable. Launder it here
// and the parent's verbatim read is correct by construction.
foreach ($results as $testId => $result) {
$results[$testId]['time'] = $this->resultTime($testId, $result['time']);
}
$json = json_encode([
'results' => $results,
'replayed' => $this->replayedCount,
'affected' => $this->affectedCount,
'executed' => $this->executedCount,
// Only the worker knows it stopped early — the parent runs no tests
// of its own, so its own check would always come back clean.
'truncated' => $this->stoppedEarly(),
], JSON_UNESCAPED_SLASHES);
if ($json === false) {
@@ -1177,6 +1498,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
continue;
}
// One worker stopping early leaves the whole suite incomplete: the
// tests it never reached are missing from the merged result set just
// as if they had been filtered out.
if (($decoded['truncated'] ?? false) === true) {
$this->resultsOnlyWrites = true;
}
if (isset($decoded['replayed']) && is_int($decoded['replayed'])) {
$this->replayedCount += $decoded['replayed'];
}
@@ -1400,6 +1728,15 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $coverage;
}
/**
* The duration to record for a test: its own, unless it was replayed rather
* than executed, in which case the duration it was recorded with stands.
*/
private function resultTime(string $testId, float $time): float
{
return $this->cachedTimeByTestId[$testId] ?? $time;
}
private function seedResultsInto(Graph $graph): void
{
/** @var ResultCollector $collector */
@@ -1424,7 +1761,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$testId,
$result['status'],
$result['message'],
$result['time'],
$this->resultTime($testId, $result['time']),
$result['assertions'],
$file,
);
@@ -1436,7 +1773,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$collector->reset();
}
private function snapshotTestResults(bool $markKnownTestFiles = false): void
/**
* Folds the run's results into the existing graph.
*
* An incomplete run passes `$complete: false`, which keeps the additive
* half — the results of the tests it did run — and drops the half that
* speaks for the suite as a whole.
*/
private function snapshotTestResults(bool $markKnownTestFiles = false, bool $complete = true): void
{
/** @var ResultCollector $collector */
$collector = Container::getInstance()->get(ResultCollector::class);
@@ -1455,8 +1799,28 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return;
}
try {
$this->resolveBranch($projectRoot);
} catch (MissingDependency) {
// This run never asked for TIA, so a missing git must not turn it
// into a failure the way it does on the TIA path. Writing to the
// fallback baseline is the lesser of the two evils.
}
// The graph above was loaded before the branch was known — this path
// only writes, but a graph carrying an unresolved fallback is the exact
// bug this whole change is about.
$graph->setFallbackBranch($this->fallbackBranch);
$touchedFiles = [];
// Whether this run is the one that records the edges its results will be
// invalidated through. A recording run's edges are written after this
// (terminate() runs last), and a parallel one's arrive with the worker
// partials that ask for $markKnownTestFiles — either way the graph on
// disk cannot be asked yet, so the run is taken at its word.
$recordsEdges = $complete && ($markKnownTestFiles || $this->recordingActive);
foreach ($results as $testId => $result) {
$file = $result['file'] ?? null;
@@ -1468,12 +1832,22 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$touchedFiles[$file] = true;
}
// A result is only ever invalidated through the edges of the test
// that produced it, so one recorded for a test the graph has no
// edges for could never be invalidated again — it would be replayed
// as settled however far the code around it moved. A run that
// records no edges leaves such a test exactly as unknown as it
// found it, whether or not it ran the whole suite.
if (! $recordsEdges && (! is_string($file) || ! $graph->knowsTest($file))) {
continue;
}
$graph->setResult(
$this->branch,
$testId,
$result['status'],
$result['message'],
$result['time'],
$this->resultTime($testId, $result['time']),
$result['assertions'],
$file,
);
@@ -1483,7 +1857,11 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$graph->markKnownTestFiles(array_keys($touchedFiles));
}
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
// Pruning reads the absence of a test from this run as the test being
// gone. That only holds if every test was invited to report.
if ($complete) {
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
}
$this->saveGraph($graph);
$collector->reset();
@@ -1520,7 +1898,29 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return null;
}
/**
* Whether this run produces a coverage report, however it was asked for.
*
* The original argv, not the handled arguments: Pest's own Coverage plugin
* appends `--coverage-php <path>` to those and runs before this one, and a
* paratest worker's arguments always carry it too. `bin/worker.php` never
* hands over the original argv, so a worker sees `[]` here and keeps taking
* this from {@see self::PIGGYBACK_COVERAGE_GLOBAL} instead.
*/
private function coverageReportActive(): bool
{
if ($this->pestCoverageActive()) {
return true;
}
return array_any(self::COVERAGE_REPORT_FLAGS, fn (string $flag): bool => $this->hasArgument($flag, $this->originalArguments));
}
/**
* Whether Pest's own `--coverage` was given — the only entry point that
* writes the coverage cache these two flags read and clean up.
*/
private function pestCoverageActive(): bool
{
$coverage = Container::getInstance()->get(Coverage::class);
assert($coverage instanceof Coverage);
@@ -1528,6 +1928,155 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $coverage->coverage;
}
/**
* Panics when the run asks for Tia alongside an option Tia cannot honour.
*
* Checked against the original argv as well, because `bin/pest` consumes
* some of these itself before PHPUnit ever sees them.
*
* @param array<int, string> $arguments
*/
private function guardUnsupportedOptions(array $arguments): void
{
foreach (self::UNSUPPORTED_OPTIONS as $option) {
if (! $this->hasArgument($option, $arguments) && ! $this->hasArgument($option, $this->originalArguments)) {
continue;
}
Panic::with(new InvalidOption(sprintf(
'The [%s] option cannot be combined with [%s].',
$option,
self::OPTION,
)));
}
}
/**
* Whether a selection-narrowing flag was given, either among the arguments
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
* the original argv. Explicit path arguments and `->only()` are detected
* separately.
*
* @param array<int, string> $arguments
*/
private function hasPartialSelection(array $arguments): bool
{
foreach (self::PARTIAL_SELECTION_FLAGS as $flag) {
if ($this->hasArgument($flag, $arguments)) {
return true;
}
if ($this->hasArgument($flag, $this->originalArguments)) {
return true;
}
}
return false;
}
/**
* Whether the run stopped before reaching every test it had queued.
*
* Covers `--bail`, `--retry` and every `--stop-on-*` flag, the equivalent
* `phpunit.xml` attributes, and an interrupted run — none of which narrow
* the selection up front, so hasPartialSelection() cannot see them. The
* tests queued behind the defect that halted the run never reported, and
* folding what did report into the baseline prunes the cached results of
* their siblings in every file the run had already entered.
*
* Deliberately unguarded. Both callers run only once PHPUnit's
* configuration is registered — the kernel reads it unguarded itself just
* before dispatching addOutput(), and flushWorkerReplay() bails out unless
* the worker actually executed something. Swallowing a failure here would
* report every truncated run as complete, which is the corruption this
* guards against in the first place.
*/
private function stoppedEarly(): bool
{
return TestResultFacade::shouldStop();
}
/**
* Resolves the baselines this run reads from and writes to, once.
*
* Results are written on runs where TIA itself took no part, and those
* never reach handleParent(). Without this the default would stand and
* every such run would write its results to `main`, whatever branch it
* actually ran on.
*/
private function resolveBranch(string $projectRoot): void
{
if ($this->branchResolved) {
return;
}
$this->branchResolved = true;
$changedFiles = new ChangedFiles($projectRoot);
// Resolved before the current branch, which throws where git is
// missing: the fallback is advisory, so a run that cannot name its
// branch at all should still carry the best answer available.
$resolved = $this->resolveFallbackBranch($changedFiles);
$this->fallbackBranchResolved = $resolved !== null;
$this->fallbackBranch = $resolved ?? self::DEFAULT_BRANCH;
Parallel::setGlobal(self::FALLBACK_BRANCH_GLOBAL, $this->fallbackBranch);
// A detached HEAD has no branch of its own to write to. The default
// branch is the honest key there — it is the commit the checkout most
// likely sits on, and it keeps a phantom baseline from being minted
// under a branch name the repository never had.
$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch;
}
/**
* The branch every other baseline falls back to reading, or null when
* nothing in the checkout can name it.
*
* Ordered by how much the source actually knows. Configuration first: it is
* the escape hatch for a repository whose git-side answers disagree with its
* branches. Then the CI provider, which states the answer outright where git
* is at its least informed. Then git itself. Then the recorded graph, whose
* single baseline can only have come from the branch this repository
* integrates on.
*/
private function resolveFallbackBranch(ChangedFiles $changedFiles): ?string
{
$inherited = Parallel::getGlobal(self::FALLBACK_BRANCH_GLOBAL);
if (is_string($inherited) && $inherited !== '') {
return $inherited;
}
return $this->watchPatterns->defaultBranch()
?? CiDefaultBranch::detect()
?? $changedFiles->defaultBranch()
?? $this->soleRecordedBranch();
}
/**
* The one branch a recorded graph holds a baseline for.
*
* Last in the chain and deliberately narrow: with a single baseline on disk
* there is only one branch whose results can be read at all, so naming it is
* strictly better than resolving to a branch that holds nothing. Two or more
* baselines carry no such implication and are left alone.
*/
private function soleRecordedBranch(): ?string
{
$json = $this->state->read(self::KEY_GRAPH);
if ($json === null) {
return null;
}
$branches = Graph::branchesIn($json);
return count($branches) === 1 ? $branches[0] : null;
}
/**
* @param array<int, string> $arguments
*/
+67
View File
@@ -219,6 +219,73 @@ final readonly class ChangedFiles
return $branch === '' || $branch === 'HEAD' ? null : $branch;
}
/**
* The repository's default branch — the one every other branch's baseline
* falls back to reading.
*
* Advisory, unlike {@see self::currentBranch()}: a repository that cannot
* answer the question is not a broken repository. A remote-less checkout
* has no `origin/HEAD`, and plenty of CI checkouts never run
* `git remote set-head`, so every step here fails soft and the caller is
* left to pick its own default.
*/
public function defaultBranch(): ?string
{
$head = $this->gitOutput(['git', 'symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
if ($head !== null) {
$branch = preg_replace('#^origin/#', '', $head);
if (is_string($branch) && $branch !== '') {
return $branch;
}
}
// `init.defaultBranch` is a setting of the machine, not of the
// repository — it names what `git init` would have called the first
// branch here, which is worth nothing once the repository disagrees.
// Taken only when a branch by that name actually exists.
$configured = $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
if ($configured === null) {
return null;
}
$exists = $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/heads/'.$configured]) !== null
|| $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/remotes/origin/'.$configured]) !== null;
return $exists ? $configured : null;
}
/**
* Whether the repository has any remote configured.
*
* Advisory like {@see self::defaultBranch()} — a `git` that cannot answer
* is reported as "no remote", and the caller decides what that means.
*/
public function hasRemote(): bool
{
return $this->gitOutput(['git', 'remote']) !== null;
}
/**
* @param array<int, string> $command
*/
private function gitOutput(array $command): ?string
{
$process = new Process($command, $this->projectRoot);
$process->setTimeout(5.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
$output = trim($process->getOutput());
return $output === '' ? null : $output;
}
private function shaIsReachable(string $sha): bool
{
$process = new Process(
+77
View File
@@ -0,0 +1,77 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
/**
* The default branch as the CI provider itself reports it.
*
* Worth asking before git: a CI checkout is the one place where git knows the
* least. `actions/checkout` builds the working copy with `git init` plus a
* single-ref `fetch` rather than a `clone`, so `origin/HEAD` is never set and
* `init.defaultBranch` — a setting of the runner image, not of the repository —
* is all git has left to offer. The provider, meanwhile, states the answer
* outright in the environment it handed us.
*
* @internal
*/
final class CiDefaultBranch
{
/**
* Advisory, like every other source in the chain: anything unreadable,
* unparsable, or simply absent means "no answer", never a failure.
*/
public static function detect(): ?string
{
return self::fromGitLab() ?? self::fromGitHubEvent();
}
private static function fromGitLab(): ?string
{
return self::environment('CI_DEFAULT_BRANCH');
}
/**
* GitHub publishes no default-branch variable, but every repository-scoped
* event payload carries `repository.default_branch`, and the path to that
* payload is in the environment.
*/
private static function fromGitHubEvent(): ?string
{
$path = self::environment('GITHUB_EVENT_PATH');
if ($path === null || ! is_file($path) || ! is_readable($path)) {
return null;
}
$contents = @file_get_contents($path);
if ($contents === false) {
return null;
}
$payload = json_decode($contents, true);
if (! is_array($payload) || ! is_array($payload['repository'] ?? null)) {
return null;
}
$branch = $payload['repository']['default_branch'] ?? null;
return is_string($branch) && $branch !== '' ? $branch : null;
}
private static function environment(string $name): ?string
{
$value = getenv($name);
if (! is_string($value)) {
return null;
}
$value = trim($value);
return $value === '' ? null : $value;
}
}
+18
View File
@@ -60,6 +60,24 @@ final class Configuration
return $this;
}
/**
* The branch whose baseline every other branch falls back to reading.
*
* Autodetected from the repository when left unset; declare it here when
* the repository cannot answer for itself — no `origin/HEAD`, or an
* `init.defaultBranch` that disagrees with reality.
*
* @return $this
*/
public function defaultBranch(string $branch): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->setDefaultBranch($branch);
return $this;
}
/**
* @param array<string, string> $patterns glob → project-relative test dir
* @return $this
+98 -9
View File
@@ -48,6 +48,17 @@ final class Graph
*/
private array $baselines = [];
/**
* The baseline a branch with none of its own reads from.
*
* Only ever read from: a branch writes to its own key, so a fallback that
* leaked into the write path would corrupt the baseline every other branch
* depends on. Resolved once per run by the plugin — see
* {@see self::setFallbackBranch()} — because the git calls it takes are not
* free and the read path runs per test.
*/
private string $fallbackBranch = 'main';
private readonly string $projectRoot;
/** @var array<string, true>|null */
@@ -576,7 +587,12 @@ final class Graph
return $this->fingerprint;
}
public function recordedAtSha(string $branch, string $fallbackBranch = 'main'): ?string
public function setFallbackBranch(string $branch): void
{
$this->fallbackBranch = $branch;
}
public function recordedAtSha(string $branch, ?string $fallbackBranch = null): ?string
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
@@ -611,7 +627,7 @@ final class Graph
$this->baselines[$branch]['results'][$testId] = $entry;
}
public function getAssertions(string $branch, string $testId, string $fallbackBranch = 'main'): ?int
public function getAssertions(string $branch, string $testId, ?string $fallbackBranch = null): ?int
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
@@ -622,7 +638,18 @@ final class Graph
return $baseline['results'][$testId]['assertions'];
}
public function getResult(string $branch, string $testId, string $fallbackBranch = 'main'): ?TestStatus
public function getTime(string $branch, string $testId, ?string $fallbackBranch = null): ?float
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
if (! isset($baseline['results'][$testId]['time'])) {
return null;
}
return $baseline['results'][$testId]['time'];
}
public function getResult(string $branch, string $testId, ?string $fallbackBranch = null): ?TestStatus
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
@@ -649,7 +676,7 @@ final class Graph
/**
* @return array<int, string>
*/
public function testFilesToRerun(string $branch, string $fallbackBranch = 'main'): array
public function testFilesToRerun(string $branch, ?string $fallbackBranch = null): array
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
$files = [];
@@ -677,7 +704,16 @@ final class Graph
return array_keys($files);
}
public function hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main'): bool
/**
* Whether any cached result due a re-run points at a test file that is not
* on disk — deleted, or never locatable in the first place (`eval()`'d code,
* a path outside the project).
*
* A filtered run cannot honour such an entry: it would select a file that
* collects no tests, so the run reports green without ever re-running the
* failure — and does so again on every subsequent invocation.
*/
public function hasUnlocatedTestsToRerun(string $branch, ?string $fallbackBranch = null): bool
{
$baseline = $this->baselineFor($branch, $fallbackBranch);
@@ -688,7 +724,16 @@ final class Graph
$file = $result['file'] ?? null;
if ($file === null || $file === '' || $this->relative($file) === null) {
if ($file === null || $file === '') {
return true;
}
$rel = $this->relative($file);
// Results are stored relative, so `relative()` answers "is this
// inside the project" without ever touching the filesystem. The
// stat is what tells a deleted test file apart from a live one.
if ($rel === null || ! is_file($this->projectRoot.'/'.$rel)) {
return true;
}
}
@@ -779,7 +824,7 @@ final class Graph
/**
* @return array<string, string>
*/
public function lastRunTree(string $branch, string $fallbackBranch = 'main'): array
public function lastRunTree(string $branch, ?string $fallbackBranch = null): array
{
return $this->baselineFor($branch, $fallbackBranch)['tree'];
}
@@ -787,8 +832,10 @@ final class Graph
/**
* @return array{sha: ?string, tree: array<string, string>, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
*/
private function baselineFor(string $branch, string $fallbackBranch): array
private function baselineFor(string $branch, ?string $fallbackBranch): array
{
$fallbackBranch ??= $this->fallbackBranch;
if (isset($this->baselines[$branch])) {
return $this->baselines[$branch];
}
@@ -809,8 +856,15 @@ final class Graph
/**
* @param array<string, array<int, string>> $testToFiles
* @param bool $keepExisting Leave already-recorded edge sets alone. For runs
* whose edges are piggybacked off a PHPUnit coverage
* session: that data is scoped by `<source>`, so it
* can only ever be narrower than what the TIA
* recorder sees — it never contains the test's own
* file, for one — and a narrower edge set silently
* stops selecting the tests it used to select.
*/
public function replaceEdges(array $testToFiles): void
public function replaceEdges(array $testToFiles, bool $keepExisting = false): void
{
foreach ($testToFiles as $testFile => $sources) {
$testRel = $this->relative($testFile);
@@ -819,6 +873,12 @@ final class Graph
continue;
}
// An empty set means "known, covers nothing", so piggyback data is
// still an improvement there — only a populated set is protected.
if ($keepExisting && ($this->edges[$testRel] ?? []) !== []) {
continue;
}
$this->edges[$testRel] = [];
foreach ($sources as $source) {
@@ -1422,6 +1482,35 @@ final class Graph
}
}
/**
* The branches a recorded graph holds baselines for, read straight from the
* encoded form.
*
* Answerable before the graph is hydrated because the default branch has to
* be resolved first: the fallback is what every hydrated graph reads its
* baselines through.
*
* @return list<string>
*/
public static function branchesIn(string $json): array
{
$data = json_decode($json, true);
if (! is_array($data) || ! is_array($data['baselines'] ?? null)) {
return [];
}
$branches = [];
foreach (array_keys($data['baselines']) as $branch) {
if (is_string($branch) && $branch !== '') {
$branches[] = $branch;
}
}
return $branches;
}
public static function decode(string $json, string $projectRoot): ?self
{
$data = json_decode($json, true);
+1 -1
View File
@@ -68,7 +68,7 @@ final class Recorder
public function driverAvailable(): bool
{
if (! $this->driverChecked) {
if (function_exists('pcov\\start')) {
if (function_exists('pcov\\start') && filter_var((string) ini_get('pcov.enabled'), FILTER_VALIDATE_BOOL)) {
$this->driver = 'pcov';
$this->driverAvailable = true;
} elseif (function_exists('xdebug_start_code_coverage') && function_exists('xdebug_info')) {
+2 -2
View File
@@ -56,7 +56,7 @@ final class TableExtractor
$tables[strtolower($name)] = true;
}
$out = array_keys($tables);
$out = array_map(strval(...), array_keys($tables));
sort($out);
return $out;
@@ -112,7 +112,7 @@ final class TableExtractor
}
}
$out = array_keys($tables);
$out = array_map(strval(...), array_keys($tables));
sort($out);
return $out;
+13
View File
@@ -44,6 +44,8 @@ final class WatchPatterns
private bool $baselined = false;
private ?string $defaultBranch = null;
public function useDefaults(string $projectRoot): void
{
$testPath = TestSuite::getInstance()->testPath;
@@ -177,6 +179,16 @@ final class WatchPatterns
return $this->baselined;
}
public function setDefaultBranch(string $branch): void
{
$this->defaultBranch = $branch;
}
public function defaultBranch(): ?string
{
return $this->defaultBranch;
}
public function reset(): void
{
$this->patterns = [];
@@ -185,6 +197,7 @@ final class WatchPatterns
$this->locally = false;
$this->filtered = false;
$this->baselined = false;
$this->defaultBranch = null;
}
private function keyMatches(string $key, string $file): bool
@@ -22,7 +22,7 @@ final readonly class EnsureTiaAssertionsAreRecordedOnFinished implements Finishe
if ($test instanceof TestMethod) {
$this->collector->recordAssertions(
$test->className().'::'.$test->methodName(),
$test->id(),
$event->numberOfAssertionsPerformed(),
);
}
@@ -21,7 +21,7 @@ final readonly class EnsureTiaResultsAreCollected implements PreparationStartedS
$test = $event->test();
if ($test instanceof TestMethod) {
$this->collector->testPrepared($test->className().'::'.$test->methodName(), $test->file());
$this->collector->testPrepared($test->id(), $test->file());
}
}
}
+56
View File
@@ -17,6 +17,33 @@ final class Str
private const string PREFIX = '__pest_evaluable_';
/**
* The list of names PHP reserves, and therefore refuses, as class names.
*
* @see https://github.com/php/php-src/blob/master/Zend/zend_compile.c
*
* @var array<int, string>
*/
private const array RESERVED_CLASS_NAMES = [
'array',
'bool',
'callable',
'false',
'float',
'int',
'iterable',
'mixed',
'never',
'null',
'object',
'parent',
'self',
'static',
'string',
'true',
'void',
];
/**
* Create a (unsecure & non-cryptographically safe) random alpha-numeric
* string value.
@@ -64,6 +91,35 @@ final class Str
return (string) preg_replace('/[^a-zA-Z0-9_\x80-\xff]/', '_', $code);
}
/**
* Determine if the given name is a valid PHP identifier, and therefore may
* be used as a single namespace name.
*/
public static function isValidIdentifier(string $name): bool
{
return preg_match('/^[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*$/', $name) === 1;
}
/**
* Determine if the given name may be declared as a class name by an `eval`.
*/
public static function isValidClassName(string $name): bool
{
if (! self::isValidIdentifier($name)) {
return false;
}
if (in_array(strtolower($name), self::RESERVED_CLASS_NAMES, true)) {
return false;
}
$tokens = token_get_all(sprintf('<?php %s;', $name));
// Anything the lexer sees as a keyword, like `list` or `match`, may not
// be used as a class name.
return is_array($tokens[1] ?? null) && $tokens[1][0] === T_STRING;
}
/**
* Get the portion of a string before the last occurrence of a given value.
*/
@@ -1,5 +1,5 @@
Pest Testing Framework 5.0.0-beta.1.
Pest Testing Framework 5.0.3.
USAGE: pest <file> [options]
@@ -168,6 +168,16 @@
AI OPTIONS:
--ai ..... Run a code snippet as a fully scaffolded test for AI verification
TIA OPTIONS:
--tia Re-run only the tests affected by your changes, replaying the rest from cache
--no-tia ......................... Disable test impact analysis for this run
--tia --fresh .... Discard the recorded dependency graph and record it again
--tia --filtered ............ Narrow the run to the affected test files only
--tia --locally ......... Enable test impact analysis on local machines only
--tia --baselined Fetch the shared dependency graph recorded by the CI baseline
--tia --refetch ......... Force a fresh fetch of the shared dependency graph
--baseline Output to standard output the test impact analysis storage directory
MUTATION TESTING OPTIONS:
--mutate .... Runs mutation testing, to understand the quality of your tests
--mutate --parallel ...................... Runs mutation testing in parallel
@@ -1,3 +1,3 @@
Pest Testing Framework 5.0.0-beta.1.
Pest Testing Framework 5.0.3.
+13 -13
View File
@@ -1,27 +1,27 @@
##teamcity[testSuiteStarted name='Tests/tests/Failure' locationHint='pest_qn://tests/.tests/Failure.php' flowId='1234']
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/Failure' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php' flowId='1234']
##teamcity[testCount count='8' flowId='1234']
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/.tests/Failure.php::it can fail with comparison' flowId='1234']
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/.tests/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail with comparison' flowId='1234']
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/Fixtures/Suites/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
##teamcity[testFinished name='it can fail with comparison' duration='100000' flowId='1234']
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because of no assertions' flowId='1234']
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because of no assertions' flowId='1234']
##teamcity[testIgnored name='it can be ignored because of no assertions' message='This test did not perform any assertions' details='' flowId='1234']
##teamcity[testFinished name='it can be ignored because of no assertions' duration='100000' flowId='1234']
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because it is skipped' flowId='1234']
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because it is skipped' flowId='1234']
##teamcity[testIgnored name='it can be ignored because it is skipped' message='This test was ignored.' details='' flowId='1234']
##teamcity[testFinished name='it can be ignored because it is skipped' duration='100000' flowId='1234']
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/.tests/Failure.php::it can fail' flowId='1234']
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/.tests/Failure.php:18' flowId='1234']
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail' flowId='1234']
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/Fixtures/Suites/Failure.php:18' flowId='1234']
##teamcity[testFinished name='it can fail' duration='100000' flowId='1234']
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/.tests/Failure.php::it throws exception' flowId='1234']
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/.tests/Failure.php:22' flowId='1234']
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it throws exception' flowId='1234']
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/Fixtures/Suites/Failure.php:22' flowId='1234']
##teamcity[testFinished name='it throws exception' duration='100000' flowId='1234']
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/.tests/Failure.php::it is not done yet' flowId='1234']
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is not done yet' flowId='1234']
##teamcity[testFinished name='it is not done yet' duration='100000' flowId='1234']
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/.tests/Failure.php::build this one.' flowId='1234']
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::build this one.' flowId='1234']
##teamcity[testFinished name='build this one.' duration='100000' flowId='1234']
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/.tests/Failure.php::it is passing' flowId='1234']
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is passing' flowId='1234']
##teamcity[testFinished name='it is passing' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='Tests/tests/Failure' flowId='1234']
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/Failure' flowId='1234']
Tests: 3 failed, 1 risky, 2 todos, 1 skipped, 1 passed (3 assertions)
Duration: 1.00s
+8 -8
View File
@@ -1,18 +1,18 @@
##teamcity[testSuiteStarted name='Tests/tests/SuccessOnly' locationHint='pest_qn://tests/.tests/SuccessOnly.php' flowId='1234']
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/SuccessOnly' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php' flowId='1234']
##teamcity[testCount count='4' flowId='1234']
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/.tests/SuccessOnly.php::it can pass with comparison' flowId='1234']
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison' flowId='1234']
##teamcity[testFinished name='it can pass with comparison' duration='100000' flowId='1234']
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can also pass' flowId='1234']
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can also pass' flowId='1234']
##teamcity[testFinished name='can also pass' duration='100000' flowId='1234']
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset' flowId='1234']
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset' flowId='1234']
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
##teamcity[testFinished name='can pass with dataset with data set "(true)"' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='can pass with dataset' flowId='1234']
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
##teamcity[testFinished name='`block` → can pass with dataset in describe block with data set "(1)"' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testSuiteFinished name='Tests/tests/SuccessOnly' flowId='1234']
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/SuccessOnly' flowId='1234']
Tests: 4 passed (4 assertions)
Duration: 1.00s
+1 -14
View File
@@ -1,17 +1,4 @@
WARN Tests\Fixtures\CollisionTest
- error
- success
PASS Tests\Fixtures\DirectoryWithTests\ExampleTest
✓ it example 1
PASS Tests\Fixtures\ExampleTest
✓ it example 2
WARN Tests\Fixtures\Inheritance\Base\ExampleTest
- example
PASS Tests\Fixtures\Inheritance\ExampleTest
✓ example
Tests: 3 skipped, 3 passed (3 assertions)
Tests: 1 passed (1 assertions)
+40 -1
View File
@@ -1565,6 +1565,37 @@
✓ it allows performing no expectations without being risky
✓ a "describe" group of tests → it allows performing no expectations without being risky
PASS Tests\Features\Tia
✓ it does not run user hooks when replaying cached skipped and incomplete results
PASS Tests\Features\Tia\DefaultBranchReplay
✓ replays the default branch baseline on a new branch
✓ replays whatever the default branch is called with ('main')
✓ replays whatever the default branch is called with ('master')
✓ replays whatever the default branch is called with ('trunk')
✓ replays whatever the default branch is called with ('develop')
✓ replays on a second new branch too
✓ writes nothing on a second run on the same branch
✓ replays on a branch whose name contains slashes
✓ replays inside a worktree on a new branch
PASS Tests\Features\Tia\DefaultBranchResolution
✓ a declared default branch beats autodetection
✓ a declared default branch that does not exist degrades to a full run
✓ a repository with no remote is refused rather than silently re-run
✓ a declared default branch stands in for a missing remote
✓ tia still requires git
✓ a plain run outside a repository creates no baseline
✓ the default branch is resolved once per run, not once per test
PASS Tests\Features\Tia\DefaultBranchWriteTier
✓ narrows to the affected tests on a new branch
✓ filtered mode reads the fallback too
✓ filtered mode finds nothing to do on a clean green feature branch
✓ a detached HEAD replays without minting a branch key
✓ the branch that ran gets its own key and the default branch keeps its baseline
✓ the fallback reaches parallel workers
PASS Tests\Features\Ticket
✓ it may be associated with an ticket #1, #2
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
@@ -1908,6 +1939,11 @@
✓ activateLinkTracking() → it tracks linked sources across consecutive tests
✓ activateLinkTracking() → it records nothing while inactive
PASS Tests\Unit\Plugins\Tia\ResultKey
✓ it keys a result per dataset row rather than per method
✓ it records assertions against the same per-dataset key
✓ it leaves a test without a dataset keyed by class and method
PASS Tests\Unit\Plugins\Tia\TableExtractor
✓ fromSql() → it extracts tables from plain DML
✓ fromSql() → it extracts tables from joins
@@ -1916,10 +1952,12 @@
✓ fromSql() → it records the table, not the schema, for qualified identifiers
✓ fromSql() → it handles quoted identifiers
✓ fromSql() → it ignores schema metadata tables
✓ fromSql() → it does not leak int keys for numeric identifiers
✓ fromSql() → it returns nothing for non-DML statements
✓ fromMigrationSource() → it extracts tables from Schema builder calls
✓ fromMigrationSource() → it extracts tables from raw DDL statements
✓ fromMigrationSource() → it records the table, not the schema, in qualified DDL and DML
✓ fromMigrationSource() → it does not leak int keys for numeric table names
✓ fromMigrationSource() → it extracts tables from DB::table calls
PASS Tests\Unit\Plugins\Tia\TestPaths
@@ -2153,6 +2191,7 @@
✓ a parallel test can extend another test with same name
✓ parallel reports invalid datasets as failures
✓ parallel can have multiple exclude-groups
✓ parallel can have multiple groups
PASS Tests\Visual\ParallelNestedDatasets
✓ parallel loads nested datasets from nested directories
@@ -2186,4 +2225,4 @@
✓ pass with dataset with ('my-datas-set-value')
✓ within describe → pass with dataset with ('my-datas-set-value')
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1542 passed (3375 assertions)
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1571 passed (3446 assertions)
+5 -5
View File
@@ -18,27 +18,27 @@ $run = function (string $target): array {
};
test('reports missing datasets as errors for a single file run', function () use ($run): void {
$result = $run('tests/.tests/IssueOnly.php');
$result = $run('tests/Fixtures/Suites/IssueOnly.php');
expect($result['output'])
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain('FAILED')
->toContain('Tests: 1 failed')
->and($result['code'])->not->toBe(0);
})->skipOnWindows();
test('reports missing datasets as errors alongside passing tests', function () use ($run): void {
$result = $run('tests/.tests/IssueWithPassing.php');
$result = $run('tests/Fixtures/Suites/IssueWithPassing.php');
expect($result['output'])
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain('1 passed')
->toContain('1 failed')
->and($result['code'])->not->toBe(0);
})->skipOnWindows();
test('reports dataset closure exceptions as errors', function () use ($run): void {
$result = $run('tests/.tests/DatasetClosureThrows.php');
$result = $run('tests/Fixtures/Suites/DatasetClosureThrows.php');
expect($result['output'])
->toContain('boom from dataset')
+2 -2
View File
@@ -10,12 +10,12 @@ beforeEach(function (): void {
});
it('throws exception if dataset does not exist', function (): void {
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name `first` does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name [first] does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
});
it('throws exception if dataset already exist', function (): void {
DatasetsRepository::set('second', [[]], __DIR__);
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name `second` already exists in scope ['.__DIR__.'].');
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name [second] already exists in scope ['.__DIR__.'].');
});
it('sets closures', function (): void {
+1 -1
View File
@@ -38,7 +38,7 @@ class State
{
public array $runCount = [];
public array $appliedCount = [];
public array $appliedCount;
public function __construct()
{
+1 -1
View File
@@ -142,7 +142,7 @@ it('works as higher order test')
it('fails after exhausting all retries', function (): void {
$process = new Process(
['php', 'bin/pest', 'tests/.tests/FlakyFailure.php'],
['php', 'bin/pest', 'tests/Fixtures/Suites/FlakyFailure.php'],
dirname(__DIR__, 2),
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
);
+84
View File
@@ -0,0 +1,84 @@
<?php
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\ChangedFiles;
use Pest\Plugins\Tia\FileState;
use Pest\Plugins\Tia\Fingerprint;
use Pest\Plugins\Tia\Graph;
use Pest\Plugins\Tia\Storage;
use Pest\Support\Str;
use Symfony\Component\Process\Process;
it('does not run user hooks when replaying cached skipped and incomplete results', function (): void {
$projectRoot = dirname(__DIR__, 2);
$home = sys_get_temp_dir().'/pest-tia-'.bin2hex(random_bytes(8));
$fixture = 'tests/Fixtures/Suites/TiaReplayHooks.php';
mkdir($home, 0755, true);
try {
$changedFiles = new ChangedFiles($projectRoot);
$branch = $changedFiles->currentBranch() ?? 'main';
$sha = $changedFiles->currentSha();
$id = fn (string $description): string => 'P\Tests\Fixtures\Suites\TiaReplayHooks::'.Str::evaluable($description);
$graph = new Graph($projectRoot);
$graph->setFingerprint(Fingerprint::compute($projectRoot));
$graph->setRecordedAtSha($branch, $sha);
// Hashes the working tree as it stands, so the replay sees nothing as changed.
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
$graph->markKnownTestFiles([$fixture]);
$graph->setResult($branch, $id('replayed pass'), 0, '', 0.01, 1, $fixture);
$graph->setResult($branch, $id('replayed skip'), 1, 'cached skip', 0.01, 0, $fixture);
$graph->setResult($branch, $id('replayed incomplete'), 2, 'cached incomplete', 0.01, 0, $fixture);
$json = $graph->encode();
expect($json)->not->toBeNull();
$originalHome = getenv('HOME');
putenv('HOME='.$home);
try {
$storage = new FileState(Storage::tempDir($projectRoot));
} finally {
putenv($originalHome === false ? 'HOME' : 'HOME='.$originalHome);
}
expect($storage->write(Tia::KEY_GRAPH, (string) $json))->toBeTrue();
$process = new Process(
['php', 'bin/pest', $fixture, '--tia'],
$projectRoot,
[
'COLLISION_PRINTER' => 'DefaultPrinter',
'COLLISION_IGNORE_DURATION' => 'true',
'PARATEST' => 0,
'PAO_DISABLE' => '1',
'HOME' => $home,
],
);
$process->run();
$output = removeAnsiEscapeSequences($process->getOutput().$process->getErrorOutput());
// Both hooks throw, so the run stays green only if neither one ran.
expect($output)->toContain('3 replayed')
->and($output)->not->toContain('must not run for replayed tests')
->and($output)->toContain('1 incomplete, 1 skipped, 1 passed')
->and($process->getExitCode())->toBe(0);
} finally {
$paths = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($home, FilesystemIterator::SKIP_DOTS),
RecursiveIteratorIterator::CHILD_FIRST,
);
foreach ($paths as $path) {
$path->isDir() ? @rmdir($path->getPathname()) : @unlink($path->getPathname());
}
@rmdir($home);
}
})->skipOnWindows();
+108
View File
@@ -0,0 +1,108 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
/**
* Reading a baseline recorded on another branch.
*
* Without the default-branch fallback, the first `--tia` run on every new branch
* re-runs a whole suite whose results the default branch already holds — once
* per branch, forever, on any repository not named `main`.
*
* @see https://github.com/pestphp/pest/issues/1823
*/
afterEach(function (): void {
Project::destroyAll();
});
test('replays the default branch baseline on a new branch', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($result->exitCode)->toBe(0);
})->skipOnWindows();
test('replays whatever the default branch is called', function (string $defaultBranch): void {
$project = Project::make($defaultBranch);
$project->seed($defaultBranch);
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->with(['main', 'master', 'trunk', 'develop'])->skipOnWindows();
test('replays on a second new branch too', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->switchTo('feature-y', new: true);
$result = $project->pest('--tia');
// The toll is one full run per new branch. It must not come back for the
// second branch either.
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('writes nothing on a second run on the same branch', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->snapshot();
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('replays on a branch whose name contains slashes', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature/x/y', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toContain('feature/x/y');
})->skipOnWindows();
test('replays inside a worktree on a new branch', function (): void {
$project = Project::make('master');
$worktree = $project->worktree('feature-worktree');
// Seeded against the worktree rather than the main checkout: a worktree's
// `.git` is a file, so `Storage::originIdentity()` cannot read the remote
// from it and the worktree resolves a storage key of its own. That gap is
// separate from the branch fallback, and it is the fallback this row is
// about — the worktree is checked out on a branch the baseline does not
// name, which is the scenario from the issue.
$project->seedFor($worktree, 'master');
$result = $project->pestIn($worktree, '--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
@@ -0,0 +1,237 @@
<?php
declare(strict_types=1);
use Symfony\Component\Process\ExecutableFinder;
use Tests\Fixtures\Tia\Project;
/**
* How the default branch gets named: declared in `tests/Pest.php`, autodetected
* from the repository, or not answerable at all.
*/
afterEach(function (): void {
Project::destroyAll();
});
test('a declared default branch beats autodetection', function (): void {
// The repository autodetects `develop`, which holds no baseline. Only the
// declaration in `tests/Pest.php` can reach the `master` one.
$project = Project::make('develop', overlay: 'configured-default-branch');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('a declared default branch that does not exist degrades to a full run', function (): void {
$project = Project::make('master', overlay: 'unknown-default-branch');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
// Nothing to fall back to, so everything runs — and no baseline is minted
// under the name that resolved to nothing.
expect($result->uncached())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->exitCode)->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', 'feature-x']);
})->skipOnWindows();
test('the CI provider names the default branch where the checkout cannot', function (): void {
// A CI checkout: `actions/checkout` fetches a single ref instead of cloning,
// so there is no `origin/HEAD` for git to read the default branch from. The
// event payload GitHub hands the job says it outright.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->write('.home/event.json', (string) json_encode([
'repository' => ['default_branch' => 'master'],
]));
$result = $project->pestWithEnvironment($project->path(), [
'GITHUB_EVENT_PATH' => $project->path('.home/event.json'),
], '--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('GitLab names the default branch through its own variable', function (): void {
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pestWithEnvironment($project->path(), [
'CI_DEFAULT_BRANCH' => 'master',
], '--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('a lone recorded baseline names the default branch', function (): void {
// Nothing left to ask: no `origin/HEAD`, no CI provider, and an
// `init.defaultBranch` that names a branch this repository does not have.
// The graph holds exactly one baseline, and it is the only one any branch
// could read — so it is the answer.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'main');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('a default branch nothing can name is refused rather than guessed', function (): void {
// Same checkout as above, without the graph that answered it. Guessing here
// is what made this bug expensive: the guess reads no baseline at all, and
// the output calls that a hit.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'main');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->output)->toContain('Tia mode could not determine the default branch.')
->and($result->output)->toContain('git remote set-head origin --auto')
->and($result->exitCode)->toBe(1, $result->describe())
->and($project->graphExists())->toBeFalse();
})->skipOnWindows();
test('an init.defaultBranch naming a branch that exists is still trusted', function (): void {
// The setting is the machine's, not the repository's — worth taking only
// where the repository has a branch by that name. It does here, and with no
// graph on disk it is the only source left, so the run must not be refused.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->output)->not->toContain('could not determine the default branch')
->and($project->branchKeys())->toBe(['feature-x']);
})->skipOnWindows();
test('a repository with no remote is refused rather than silently re-run', function (): void {
$project = Project::make('master');
$project->git()->removeOrigin();
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
// Nothing can name the default branch, so every new branch would re-run the
// whole suite with no explanation. Saying so beats doing that quietly. The
// missing remote is the likeliest reason and gets named as such.
expect($result->output)->toContain('Tia mode requires a repository with a remote.')
->and($result->exitCode)->toBe(1, $result->describe());
})->skipOnWindows();
test('a remote-less repository holding one baseline is not refused', function (): void {
// The refusal above exists to stop a guess, not to demand a remote for its
// own sake. With a baseline on disk there is nothing left to guess at.
$project = Project::make('master');
$project->git()->removeOrigin();
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('a declared default branch stands in for a missing remote', function (): void {
// The escape hatch the refusal above points at: with the branch named by
// hand there is nothing left for a remote to answer.
$project = Project::make('master', overlay: 'configured-default-branch');
$project->git()->removeOrigin();
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
test('tia still requires git', function (): void {
$project = Project::withoutGit();
$result = $project->pest('--tia');
// The soft default-branch resolver runs before the branch is named, and it
// must not swallow this.
expect($result->output)->toContain('The feature "Tia mode" requires "git".')
->and($result->exitCode)->not->toBe(0);
})->skipOnWindows();
test('a plain run outside a repository creates no baseline', function (): void {
$project = Project::withoutGit();
$result = $project->pest();
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($project->graphExists())->toBeFalse()
->and($project->graphDir())->not->toBeDirectory();
})->skipOnWindows();
test('the default branch is resolved once per run, not once per test', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$git = (new ExecutableFinder)->find('git');
expect($git)->not->toBeNull();
// A `git` first on `PATH` that records what it was asked before handing over
// to the real one.
$log = $project->path('git-calls.log');
$project->write('shim/git', implode("\n", [
'#!/bin/sh',
'echo "$@" >> '.escapeshellarg($log),
'exec '.escapeshellarg((string) $git).' "$@"',
'',
]));
chmod($project->path('shim/git'), 0755);
$result = $project->pestWithEnvironment($project->path(), [
'PATH' => $project->path('shim').':'.getenv('PATH'),
], '--tia');
$calls = file_exists($log) ? explode("\n", trim((string) file_get_contents($log))) : [];
$resolutions = array_filter($calls, fn (string $call): bool => str_contains($call, 'symbolic-ref')
|| str_contains($call, 'init.defaultBranch'));
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($calls)->not->toBeEmpty('the git shim was never reached')
->and($resolutions)->toHaveCount(1, implode("\n", $calls));
})->skipOnWindows();
@@ -0,0 +1,133 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
/**
* What the fallback is allowed to touch.
*
* Reading another branch's baseline must stay a read: writes belong to the
* branch that ran, and a branch that has no name of its own must not mint one.
*/
afterEach(function (): void {
Project::destroyAll();
});
test('narrows to the affected tests on a new branch', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
// Semantic, not cosmetic: PHP is hashed at the AST level, so a comment
// would not register as a change at all.
$project->write('app/Calculator.php', <<<'PHP'
<?php
declare(strict_types=1);
namespace Fixture\App;
final class Calculator
{
public function add(int $a, int $b): int
{
return $a + $b;
}
public function subtract(int $a, int $b): int
{
return $a - $b;
}
public function multiply(int $a, int $b): int
{
return $a * $b;
}
}
PHP);
$result = $project->pest('--tia');
// Two of the three test files cover `Calculator`; the third replays from the
// default branch's baseline.
expect($result->affected())->toBe(4, $result->describe())
->and($result->replayed())->toBe(2, $result->describe())
->and($result->exitCode)->toBe(0, $result->describe());
})->skipOnWindows();
test('filtered mode reads the fallback too', function (): void {
$project = Project::make('master');
// A failure cached on the default branch. Filtered mode asks the graph which
// test files are due a re-run, and that read has to reach the fallback as
// well: without it a new branch sees nothing to do, reports green, and never
// re-runs the failure — on every subsequent invocation.
$project->seed('master', failing: ['adds two numbers']);
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia', '--filtered');
// Selection is by file, so the failure's two siblings in `CalculatorTest`
// come along; the other two test files stay out of the run entirely.
expect($result->output)->toContain('from 1 previously unsuccessful test')
->and($result->output)->not->toContain('No affected tests found')
->and($result->tally())->toContain('2 passed');
})->skipOnWindows();
test('filtered mode finds nothing to do on a clean green feature branch', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia', '--filtered');
$delta = $project->delta();
expect($result->output)->toContain('No affected tests found')
->and($result->exitCode)->toBe(0, $result->describe())
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a detached HEAD replays without minting a branch key', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$result = $project->pest('--tia');
// A detached HEAD has no branch of its own. The default branch is the
// honest key, and no phantom one appears beside it.
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master']);
})->skipOnWindows();
test('the branch that ran gets its own key and the default branch keeps its baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$delta = $project->delta();
expect($project->branchKeys())->toBe(['master', 'feature-x'])
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($delta->writtenCount())->toBe(0, $delta->summary());
})->skipOnWindows();
test('the fallback reaches parallel workers', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia', '--parallel', '--processes=2');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
@@ -0,0 +1,5 @@
<?php
// The folder name creates the namespace segment `2fa`, which starts with a number.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `list`, which is a PHP keyword.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `int`, which is a name PHP reserves.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `2fa`, which starts with a number.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
it('belongs to group one', function () {
expect(true)->toBeTrue();
})->group('one');
@@ -0,0 +1,5 @@
<?php
it('belongs to group three', function () {
expect(true)->toBeTrue();
})->group('three');
@@ -0,0 +1,5 @@
<?php
it('belongs to group two', function () {
expect(true)->toBeTrue();
})->group('two');
+24
View File
@@ -0,0 +1,24 @@
<?php
// Only ever run through `tests/Features/Tia.php`, against a seeded TIA graph.
// Both hooks throw, so a replayed test that wrongly runs one fails the run.
beforeEach(function (): void {
throw new RuntimeException('The beforeEach hook must not run for replayed tests.');
});
afterEach(function (): void {
throw new RuntimeException('The afterEach hook must not run for replayed tests.');
});
test('replayed pass', function (): void {
expect(true)->toBeTrue();
});
test('replayed skip', function (): void {
expect(true)->toBeTrue();
});
test('replayed incomplete', function (): void {
expect(true)->toBeTrue();
});
+181
View File
@@ -0,0 +1,181 @@
<?php
declare(strict_types=1);
namespace Tests\Fixtures\Tia;
use Pest\Plugins\Tia\Storage;
use RuntimeException;
use Symfony\Component\Process\Process;
/**
* A git repository the scenario tests drive.
*
* Hermetic by construction: every invocation neutralises the machine's global
* and system config and carries its own committer identity. Without that, an
* ambient `init.defaultBranch = main` would answer questions the scenario means
* to leave unanswered, and rows like "resolves to nothing, degrades safely"
* would pass by accident.
*
* @internal
*/
final readonly class GitRepo
{
/**
* @var array<string, string>
*/
public const array ENV = [
'GIT_CONFIG_GLOBAL' => '/dev/null',
'GIT_CONFIG_SYSTEM' => '/dev/null',
// `GIT_CONFIG_SYSTEM` does not cover every system-level file git reads:
// Apple's git also loads one from inside Xcode, and it sets
// `init.defaultBranch`. Only this suppresses all of them.
'GIT_CONFIG_NOSYSTEM' => '1',
'GIT_AUTHOR_NAME' => 'Pest Fixture',
'GIT_AUTHOR_EMAIL' => 'fixture@pestphp.io',
'GIT_COMMITTER_NAME' => 'Pest Fixture',
'GIT_COMMITTER_EMAIL' => 'fixture@pestphp.io',
];
public function __construct(public string $path) {}
/**
* Initialises the repository on `$branch` and commits everything in it.
*/
public function init(string $branch = 'master'): void
{
$this->run(['init', '--quiet']);
$this->run(['checkout', '--quiet', '-b', $branch]);
$this->commit('Initial commit');
}
public function commit(string $message): void
{
$this->run(['add', '-A']);
$this->run(['commit', '--quiet', '--allow-empty', '-m', $message]);
}
public function switchTo(string $branch, bool $new = false): void
{
$this->run($new ? ['checkout', '--quiet', '-b', $branch] : ['checkout', '--quiet', $branch]);
}
public function rename(string $from, string $to): void
{
$this->run(['branch', '-m', $from, $to]);
}
public function detach(): void
{
$this->run(['checkout', '--quiet', '--detach']);
}
/**
* Registers an `origin`, which also decides the graph's storage key:
* {@see Storage::projectKey()} prefers the remote's
* identity over the path, so two checkouts of one repository — a worktree,
* say — share a single graph.
*/
public function addOrigin(string $url = 'git@github.com:pestphp/tia-fixture.git'): void
{
$this->run(['remote', 'add', 'origin', $url]);
}
public function removeOrigin(): void
{
$this->run(['remote', 'remove', 'origin']);
}
/**
* Points `refs/remotes/origin/HEAD` at a local branch — what
* `git remote set-head` would write, without a remote to talk to.
*/
public function setOriginHead(string $branch): void
{
$this->run(['update-ref', 'refs/remotes/origin/'.$branch, 'HEAD']);
$this->run(['symbolic-ref', 'refs/remotes/origin/HEAD', 'refs/remotes/origin/'.$branch]);
}
/**
* Drops `refs/remotes/origin/HEAD` while keeping the remote-tracking branch
* — what a CI checkout looks like. `actions/checkout` builds its working
* copy with `git init` plus a single-ref `fetch` rather than a `clone`, and
* only a `clone` writes that symbolic ref.
*/
public function unsetOriginHead(): void
{
$this->run(['symbolic-ref', '--delete', 'refs/remotes/origin/HEAD']);
}
public function config(string $key, string $value): void
{
$this->run(['config', '--local', $key, $value]);
}
/**
* Adds a worktree for a new branch and returns its path.
*/
public function worktree(string $path, string $branch): string
{
$this->run(['worktree', 'add', '--quiet', '-b', $branch, $path]);
return $path;
}
public function sha(): string
{
return $this->output(['rev-parse', 'HEAD']);
}
public function currentBranch(): string
{
return $this->output(['rev-parse', '--abbrev-ref', 'HEAD']);
}
/**
* @return array<int, string>
*/
public function branchNames(): array
{
$names = explode("\n", $this->output(['for-each-ref', '--format=%(refname:short)', 'refs/heads']));
return array_values(array_filter($names, fn (string $name): bool => $name !== ''));
}
/**
* @param array<int, string> $arguments
*/
public function output(array $arguments): string
{
return trim($this->process($arguments, mustSucceed: true)->getOutput());
}
/**
* @param array<int, string> $arguments
*/
public function run(array $arguments): void
{
$this->process($arguments, mustSucceed: true);
}
/**
* @param array<int, string> $arguments
*/
private function process(array $arguments, bool $mustSucceed): Process
{
$process = new Process(['git', ...$arguments], $this->path, self::ENV);
$process->setTimeout(30.0);
$process->run();
if ($mustSucceed && ! $process->isSuccessful()) {
throw new RuntimeException(sprintf(
"git %s failed in [%s]:\n%s",
implode(' ', $arguments),
$this->path,
$process->getErrorOutput().$process->getOutput(),
));
}
return $process;
}
}
+339
View File
@@ -0,0 +1,339 @@
<?php
declare(strict_types=1);
namespace Tests\Fixtures\Tia;
/**
* What one run did to the graph.
*
* The three tiers a run may respect, from the conformance matrix:
*
* - COMPLETE — may change everything.
* - RESULTS-ONLY — may change only `baselines[<branch>].results` for tests that
* actually ran. It may never remove an entry, nor move `sha`, `tree`,
* `edges`, `files` or `fingerprint`.
* - HARD-SUPPRESSED — may change nothing at all.
*
* {@see self::writtenCount()} is the load-bearing measurement, and the reason
* {@see Project::sentinel()} exists: without falsified cached values there is no
* way to tell "wrote the same values back" from "wrote nothing".
*
* @internal
*/
final readonly class GraphDelta
{
/**
* @param array<string, mixed>|null $before
* @param array<string, mixed>|null $after
*/
public function __construct(
private ?array $before,
private ?array $after,
) {}
public function graphWasCreated(): bool
{
return $this->before === null && $this->after !== null;
}
public function graphWasDeleted(): bool
{
return $this->before !== null && $this->after === null;
}
/**
* Result entries whose stored values actually moved.
*/
public function writtenCount(): int
{
$written = 0;
foreach ($this->branchKeys() as $branch) {
$before = $this->results($this->before, $branch);
$after = $this->results($this->after, $branch);
foreach ($before as $testId => $entry) {
if (! isset($after[$testId])) {
continue;
}
foreach (['status', 'time', 'assertions', 'message'] as $field) {
if (($entry[$field] ?? null) !== ($after[$testId][$field] ?? null)) {
$written++;
break;
}
}
}
}
return $written;
}
/**
* Result entries that appeared.
*/
public function added(): int
{
$added = 0;
foreach ($this->branchKeys() as $branch) {
$added += count(array_diff(
array_keys($this->results($this->after, $branch)),
array_keys($this->results($this->before, $branch)),
));
}
return $added;
}
/**
* Result entries that were pruned.
*/
public function removed(): int
{
$removed = 0;
foreach ($this->branchKeys() as $branch) {
$removed += count(array_diff(
array_keys($this->results($this->before, $branch)),
array_keys($this->results($this->after, $branch)),
));
}
return $removed;
}
/**
* The baseline keys after the run — the headline signal for the
* default-branch rows, where a phantom key is the defect.
*
* @return array<int, string>
*/
public function branchKeys(): array
{
return array_keys($this->baselines($this->after));
}
/**
* @return array<int, string>
*/
public function branchKeysBefore(): array
{
return array_keys($this->baselines($this->before));
}
public function branchKeysMoved(): bool
{
return $this->branchKeysBefore() !== $this->branchKeys();
}
/**
* Whether the named branch's baseline is byte-identical — how a row proves
* the fallback is read-only.
*/
public function baselineUntouched(string $branch): bool
{
return ($this->baselines($this->before)[$branch] ?? null)
=== ($this->baselines($this->after)[$branch] ?? null);
}
public function shaMoved(): bool
{
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'sha', $this->before) !== $this->baselineField($branch, 'sha', $this->after));
}
public function treeMoved(): bool
{
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'tree', $this->before) !== $this->baselineField($branch, 'tree', $this->after));
}
/**
* Compares edges by the file paths they resolve to, not by file id: ids are
* an implementation detail that shifts whenever `files` is rebuilt in a
* different order.
*/
public function edgesMoved(): bool
{
return $this->edgeSets($this->before) !== $this->edgeSets($this->after);
}
public function filesMoved(): bool
{
return $this->section($this->before, 'files') !== $this->section($this->after, 'files');
}
public function fingerprintMoved(): bool
{
return $this->section($this->before, 'fingerprint') !== $this->section($this->after, 'fingerprint');
}
public function structureMoved(): bool
{
if ($this->edgesMoved()) {
return true;
}
if ($this->filesMoved()) {
return true;
}
if ($this->fingerprintMoved()) {
return true;
}
return $this->branchKeysMoved();
}
/**
* Nothing moved at all.
*/
public function isHardSuppressed(): bool
{
return $this->before === $this->after;
}
/**
* Results may have moved for tests that ran; nothing structural did.
*/
public function isResultsOnly(): bool
{
return ! $this->graphWasCreated()
&& ! $this->graphWasDeleted()
&& ! $this->structureMoved()
&& ! $this->shaMoved()
&& ! $this->treeMoved()
&& $this->removed() === 0
&& $this->added() === 0;
}
/**
* A one-line verdict, for failure messages.
*/
public function summary(): string
{
if ($this->graphWasCreated()) {
return 'graph created';
}
if ($this->graphWasDeleted()) {
return 'graph deleted';
}
$moved = [];
foreach (['edges', 'files', 'fingerprint'] as $section) {
if ($this->{$section.'Moved'}()) {
$moved[] = $section;
}
}
if ($this->branchKeysMoved()) {
$moved[] = sprintf(
'branchkeys(%s->%s)',
implode('|', $this->branchKeysBefore()),
implode('|', $this->branchKeys()),
);
}
return sprintf(
'w=%d +%d -%d %s%s%s',
$this->writtenCount(),
$this->added(),
$this->removed(),
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
$this->shaMoved() ? ' sha:changed' : '',
$this->treeMoved() ? ' tree:changed' : '',
);
}
/**
* @param array<string, mixed>|null $graph
* @return array<string, array<int, string>>
*/
private function edgeSets(?array $graph): array
{
$files = $this->section($graph, 'files');
$sets = [];
foreach ($this->section($graph, 'edges') as $test => $ids) {
if (! is_string($test)) {
continue;
}
if (! is_array($ids)) {
continue;
}
$paths = array_map(
fn (mixed $id): string => is_int($id) && isset($files[$id]) && is_string($files[$id])
? $files[$id]
: '?'.json_encode($id),
$ids,
);
sort($paths);
$sets[$test] = $paths;
}
ksort($sets);
return $sets;
}
/**
* @param array<string, mixed>|null $graph
* @return array<mixed>
*/
private function section(?array $graph, string $key): array
{
$section = $graph[$key] ?? null;
return is_array($section) ? $section : [];
}
/**
* @param array<string, mixed>|null $graph
* @return array<string, mixed>
*/
private function baselines(?array $graph): array
{
$baselines = [];
foreach ($this->section($graph, 'baselines') as $branch => $baseline) {
if (is_string($branch)) {
$baselines[$branch] = $baseline;
}
}
return $baselines;
}
/**
* @param array<string, mixed>|null $graph
* @return array<string, array<string, mixed>>
*/
private function results(?array $graph, string $branch): array
{
$results = $this->baselines($graph)[$branch]['results'] ?? null;
if (! is_array($results)) {
return [];
}
$entries = [];
foreach ($results as $testId => $entry) {
if (is_string($testId) && is_array($entry)) {
$entries[$testId] = $entry;
}
}
return $entries;
}
/**
* @param array<string, mixed>|null $graph
*/
private function baselineField(string $branch, string $field, ?array $graph): mixed
{
return $this->baselines($graph)[$branch][$field] ?? null;
}
}
+102
View File
@@ -0,0 +1,102 @@
<?php
declare(strict_types=1);
namespace Tests\Fixtures\Tia;
/**
* The outcome of one `pest` invocation against a fixture project.
*
* @internal
*/
final readonly class PestResult
{
/**
* The run's output, with the terminal's escape sequences taken back out.
*/
public string $output;
/**
* @param array<int, string> $arguments
*/
public function __construct(
public array $arguments,
string $output,
public int $exitCode,
) {
$this->output = (string) preg_replace([
'#\x1b[[][^A-Za-z]*[A-Za-z]#', // colours, cursor moves
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#', // hyperlinks
], '', $output);
}
/**
* Tests whose cached result was replayed instead of executed.
*/
public function replayed(): int
{
return $this->recapFragment('replayed');
}
/**
* Tests that ran because the graph held nothing for them — the count that
* betrays a fallback which never resolved.
*/
public function uncached(): int
{
return $this->recapFragment('uncached');
}
/**
* Tests that ran because a file they depend on changed.
*/
public function affected(): int
{
return $this->recapFragment('affected');
}
/**
* The `Tests:` summary line, without its label or leading whitespace.
*/
public function tally(): string
{
if (preg_match('/^\s*Tests:\s+(.+)$/m', $this->output, $matches) !== 1) {
return '';
}
return trim($matches[1]);
}
public function contains(string $needle): bool
{
return str_contains($this->output, $needle);
}
/**
* A description of the run, for failure messages that would otherwise say
* only that 0 !== 6.
*/
public function describe(): string
{
return sprintf(
"pest %s exited %d:\n%s",
implode(' ', $this->arguments),
$this->exitCode,
$this->output,
);
}
/**
* Read off the `Tests:` line rather than the whole output: the TIA headline
* counts affected *files*, and matching that instead would be a quietly
* wrong number.
*/
private function recapFragment(string $label): int
{
if (preg_match('/(\d+) '.preg_quote($label, '/').'/', $this->tally(), $matches) !== 1) {
return 0;
}
return (int) $matches[1];
}
}
+645
View File
@@ -0,0 +1,645 @@
<?php
declare(strict_types=1);
namespace Tests\Fixtures\Tia;
use FilesystemIterator;
use Pest\Factories\TestCaseFactory;
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\ChangedFiles;
use Pest\Plugins\Tia\FileState;
use Pest\Plugins\Tia\Fingerprint;
use Pest\Plugins\Tia\Graph;
use Pest\Plugins\Tia\Storage;
use Pest\Support\Str;
use RecursiveDirectoryIterator;
use RecursiveIteratorIterator;
use RuntimeException;
use Symfony\Component\Process\Process;
/**
* A throwaway Pest project the TIA scenario tests drive.
*
* Two facts about Pest shape everything here:
*
* 1. `bin/pest` derives the project root from **the autoloader it finds**, not
* from the working directory — `dirname($autoloadPath, 2)`. So the project
* owns a real `vendor/autoload.php` and a real copy of `bin/pest` at the path
* a composer install would have put them. A symlinked `vendor` would resolve
* `__DIR__` straight back to the Pest repository, and every scenario would
* silently measure the wrong project.
* 2. TIA cannot *record* without pcov or Xdebug, and CI has neither. So a
* scenario never records: {@see self::seed()} writes the graph a recording
* run would have written, and the run under test exercises the read path.
*
* @internal
*/
final class Project
{
/**
* Test file → the source files a recording run would have linked it to. The
* self-edge every test file gets is added on top of these.
*
* @var array<string, array<int, string>>
*/
public const array EDGES = [
'tests/Unit/CalculatorTest.php' => ['app/Calculator.php'],
'tests/Unit/GreeterTest.php' => ['app/Greeter.php'],
'tests/Feature/CoversCalculatorTest.php' => ['app/Calculator.php'],
];
/**
* Test file → the descriptions it declares, in declaration order.
*
* @var array<string, array<int, string>>
*/
public const array TESTS = [
'tests/Unit/CalculatorTest.php' => ['adds two numbers', 'subtracts two numbers'],
'tests/Unit/GreeterTest.php' => ['greets a person', 'greets the world'],
'tests/Feature/CoversCalculatorTest.php' => ['adds within a feature test', 'subtracts within a feature test'],
];
/**
* Every test in the fixture suite.
*/
public const int TOTAL_TESTS = 6;
/**
* Every project scaffolded so far, so a row cannot leak one by failing
* before its own cleanup.
*
* @var array<int, self>
*/
private static array $created = [];
private ?GitRepo $repo = null;
/**
* @var array<string, mixed>|null
*/
private ?array $snapshot = null;
/**
* @var array<int, string>
*/
private array $extraPaths = [];
/**
* The root whose graph this project reads and writes.
*
* Its own, except where a row seeds a worktree: a worktree's `.git` is a
* file rather than a directory, so {@see Storage} cannot read the remote
* from it and resolves a storage key of its own.
*/
private string $graphRoot;
private function __construct(public readonly string $path)
{
$this->graphRoot = $path;
}
/**
* Scaffolds a project whose default branch is `$branch`, and hands it back
* checked out there.
*
* The repository is realistic on purpose: it has an `origin`, and an
* `origin/HEAD` naming `$branch`, which is what a checkout of a real project
* looks like and what the default branch is autodetected from.
*/
public static function make(string $branch = 'master', ?string $overlay = null): self
{
$project = self::scaffold($overlay);
$project->git()->init($branch);
$project->git()->addOrigin();
$project->git()->setOriginHead($branch);
return $project;
}
/**
* Destroys every project scaffolded so far. Belongs in an `afterEach`.
*/
public static function destroyAll(): void
{
while (self::$created !== []) {
array_pop(self::$created)->destroy();
}
}
/**
* A project that is not a git repository at all — for the rows that assert
* TIA still demands git, and that a plain run does not care.
*
* Lives in the system temp directory, so it is outside any enclosing
* repository, and owns a real `vendor` rather than a symlinked one, so its
* baseline key cannot collide with another fixture's.
*/
public static function withoutGit(?string $overlay = null): self
{
return self::scaffold($overlay);
}
public function git(): GitRepo
{
return $this->repo ??= new GitRepo($this->path);
}
public function path(string $relative = ''): string
{
return $relative === '' ? $this->path : $this->path.DIRECTORY_SEPARATOR.$relative;
}
/**
* Overwrites a file in the project.
*
* Edits must be semantic: TIA hashes PHP at the AST level, so a
* comment-only change is not a change at all.
*/
public function write(string $relative, string $contents): void
{
$path = $this->path($relative);
$directory = dirname($path);
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
}
if (file_put_contents($path, $contents) === false) {
throw new RuntimeException(sprintf('Unable to write [%s].', $path));
}
}
/**
* Adds a worktree for a new branch, scaffolded so `pest` can run in it, and
* returns its path.
*
* The graph is shared with the main checkout, which is the whole point: both
* resolve the same storage key, because {@see Storage} prefers the `origin`
* identity over the path.
*/
public function worktree(string $branch): string
{
$path = $this->path.'-worktree-'.preg_replace('/[^a-z0-9]+/i', '-', $branch);
$this->git()->worktree($path, $branch);
$this->scaffoldVendor($path);
$this->extraPaths[] = $path;
return $path;
}
/**
* Runs `pest` in the project and returns what happened.
*/
public function pest(string ...$arguments): PestResult
{
return $this->pestIn($this->path, ...$arguments);
}
/**
* Runs `pest` in `$directory` — a worktree, say — against this project's
* graph.
*/
public function pestIn(string $directory, string ...$arguments): PestResult
{
return $this->pestWithEnvironment($directory, [], ...$arguments);
}
/**
* @param array<string, string> $environment
*/
public function pestWithEnvironment(string $directory, array $environment, string ...$arguments): PestResult
{
$process = new Process(
[PHP_BINARY, $directory.'/vendor/pestphp/pest/bin/pest', ...$arguments],
$directory,
[
...GitRepo::ENV,
'COLLISION_PRINTER' => 'DefaultPrinter',
'COLLISION_IGNORE_DURATION' => 'true',
'PARATEST' => '0',
'PAO_DISABLE' => '1',
'HOME' => $this->home(),
// Blanked for the same reason `GitRepo::ENV` blanks git's own
// config: the default branch a CI provider reports is the one
// *its* build is for. Pest's suite runs on GitHub Actions, so
// without this every scenario would autodetect Pest's default
// branch instead of the fixture's.
'GITHUB_EVENT_PATH' => '',
'CI_DEFAULT_BRANCH' => '',
...$environment,
],
);
$process->setTimeout(180.0);
$process->run();
return new PestResult(
array_values($arguments),
$process->getOutput().$process->getErrorOutput(),
(int) $process->getExitCode(),
);
}
/**
* Writes the graph a clean, green recording run on `$branch` would have
* written, and remembers it as the snapshot the next {@see self::delta()}
* compares against.
*
* Sentinelled by default: a row almost always wants to know which entries
* were written, and only a real recording run's values can answer that.
*
* `$failing` names descriptions to record as failures — a clean green run
* can never cache one, so a row that needs a cached failure to re-run has to
* be handed it.
*
* @param array<int, string> $failing
*/
public function seed(string $branch, bool $sentinel = true, array $failing = []): void
{
$this->seedFor($this->path, $branch, $sentinel, $failing);
}
/**
* Seeds the graph belonging to `$root` — a worktree, say, which resolves a
* storage key of its own.
*
* @param array<int, string> $failing
*/
public function seedFor(string $root, string $branch, bool $sentinel = true, array $failing = []): void
{
$this->graphRoot = $root;
$changedFiles = new ChangedFiles($root);
$sha = new GitRepo($root)->sha();
$graph = new Graph($root);
$graph->setFingerprint(Fingerprint::compute($root));
$graph->setRecordedAtSha($branch, $sha);
// Hashes the tree as it stands, so the run under test sees nothing as
// changed — the same call the recording path makes.
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
$graph->markKnownTestFiles(array_keys(self::EDGES));
foreach (self::EDGES as $testFile => $sourceFiles) {
$graph->link($testFile, $testFile);
foreach ($sourceFiles as $sourceFile) {
$graph->link($testFile, $sourceFile);
}
}
foreach (self::TESTS as $testFile => $descriptions) {
foreach ($descriptions as $description) {
$failed = in_array($description, $failing, true);
$graph->setResult(
$branch,
self::testId($testFile, $description),
$failed ? 7 : 0,
$failed ? 'cached failure' : '',
0.05,
1,
$testFile,
);
}
}
$json = $graph->encode();
if ($json === null) {
throw new RuntimeException('Unable to encode the seeded graph.');
}
if (! $this->state()->write(Tia::KEY_GRAPH, $json)) {
throw new RuntimeException('Unable to persist the seeded graph.');
}
$sentinel ? $this->sentinel() : $this->snapshot();
}
/**
* The id PHPUnit reports for a test in the fixture suite.
*
* Mirrors how Pest names generated test classes
* ({@see TestCaseFactory}): a wrong id here shows up as
* `0 replayed`, which every scenario asserts against.
*/
public static function testId(string $testFile, string $description): string
{
$basename = basename($testFile, '.php');
$dotPosition = strpos($basename, '.');
if ($dotPosition !== false) {
$basename = substr($basename, 0, $dotPosition);
}
$relative = dirname(ucfirst($testFile)).DIRECTORY_SEPARATOR.$basename;
return 'P\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relative).'::'.Str::evaluable($description);
}
/**
* Falsifies every cached value, so the next {@see self::delta()} can tell
* "wrote the same values back" from "wrote nothing".
*
* `assertions` is only ever falsified where it is already non-zero: risky,
* skipped and incomplete statuses are *derived* from "performed no
* assertions", so patching those would rewrite the status on replay and
* destroy the very discriminator this exists to provide.
*/
public function sentinel(): void
{
$graph = $this->graph();
if ($graph === null) {
throw new RuntimeException('There is no graph to sentinel.');
}
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
foreach (array_keys($baseline['results'] ?? []) as $testId) {
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
}
}
}
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($graph, JSON_UNESCAPED_SLASHES));
$this->snapshot();
}
/**
* The decoded graph, or `null` when there is none.
*
* @return array<string, mixed>|null
*/
public function graph(): ?array
{
$json = $this->state()->read(Tia::KEY_GRAPH);
if ($json === null) {
return null;
}
$graph = json_decode($json, true);
return is_array($graph) ? $graph : null;
}
/**
* @return array<int, string>
*/
public function branchKeys(): array
{
$baselines = $this->graph()['baselines'] ?? [];
return is_array($baselines) ? array_keys($baselines) : [];
}
public function graphDir(): string
{
return $this->withHome(fn (): string => Storage::tempDir($this->graphRoot));
}
public function graphExists(): bool
{
return is_file($this->graphDir().DIRECTORY_SEPARATOR.Tia::KEY_GRAPH);
}
/**
* Remembers the graph as it stands now.
*/
public function snapshot(): void
{
$this->snapshot = $this->graph();
}
/**
* What has happened to the graph since the last snapshot.
*/
public function delta(): GraphDelta
{
return new GraphDelta($this->snapshot, $this->graph());
}
/**
* Writes the `vendor` a composer install would have produced.
*
* Pest is mirrored in at `vendor/pestphp/pest` rather than pointed at,
* because Pest locates things from where its own files sit:
* `bin/pest` finds the project root by walking up from the autoloader it
* loads, and a parallel run picks its worker binary — and with it the
* worker's project root — from the directory the runner class was loaded
* from. Deferring to the repository's copy would resolve both back to the
* Pest repository, and every scenario would quietly measure that instead.
*
* Hardlinked where the filesystem allows it, so the mirror costs almost
* nothing and can never drift from the working tree.
*/
public function scaffoldVendor(string $directory): void
{
$pestRoot = dirname(__DIR__, 3);
$pest = $directory.'/vendor/pestphp/pest';
// `overrides`, `resources` and `stubs` come along because Pest loads
// them relative to `src` — the same list `BootExcludeList` walks.
foreach (['src', 'overrides', 'resources', 'stubs'] as $tree) {
self::mirror($pestRoot.'/'.$tree, $pest.'/'.$tree);
}
foreach (['pest', 'worker.php'] as $binary) {
self::mirror($pestRoot.'/bin/'.$binary, $pest.'/bin/'.$binary);
}
self::mirror($pestRoot.'/composer.json', $pest.'/composer.json');
// Pest's own autoloader, with the mirrored copy taking precedence: the
// repository's `vendor` supplies PHPUnit, Symfony and the plugin
// packages, none of which care where they are loaded from.
file_put_contents($directory.'/vendor/autoload.php', sprintf(
"<?php\n\n\$loader = require %s;\n\$loader->addPsr4('Pest\\\\', __DIR__.'/pestphp/pest/src', true);\n\nreturn \$loader;\n",
var_export($pestRoot.'/vendor/autoload.php', true),
));
// Invoking the binary directly skips composer's bin proxy, which is
// what would otherwise define `$GLOBALS['_composer_bin_dir']`. Without
// it `Pest\Plugin\Loader` looks for `vendor/bin/../pest-plugins.json`
// relative to the working directory — so that is where the plugin list
// goes, and mirroring the repository's keeps it in step with
// composer.json. `vendor/bin` has to exist for the `..` in that path to
// resolve, empty though it is.
if (! is_dir($directory.'/vendor/bin') && ! @mkdir($directory.'/vendor/bin', 0755, true)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $directory.'/vendor/bin'));
}
self::mirror($pestRoot.'/vendor/pest-plugins.json', $directory.'/vendor/pest-plugins.json');
}
public function destroy(): void
{
foreach ([...$this->extraPaths, $this->path] as $path) {
$this->remove($path);
}
}
private function home(): string
{
return $this->path.DIRECTORY_SEPARATOR.'.home';
}
private function state(): FileState
{
return new FileState($this->graphDir());
}
/**
* `Storage` reads `HOME` from the environment, so the graph lands inside the
* throwaway project instead of the developer's real `~/.pest`.
*
* @template TReturn
*
* @param callable(): TReturn $callback
* @return TReturn
*/
private function withHome(callable $callback): mixed
{
$original = getenv('HOME');
putenv('HOME='.$this->home());
try {
return $callback();
} finally {
putenv($original === false ? 'HOME' : 'HOME='.$original);
}
}
private static function scaffold(?string $overlay): self
{
$path = sys_get_temp_dir().DIRECTORY_SEPARATOR.'pest-tia-'.bin2hex(random_bytes(8));
if (! @mkdir($path, 0755, true) && ! is_dir($path)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
}
// Realpathed because Pest realpaths its own project root, and macOS
// hands out a symlinked temp directory.
$real = realpath($path);
$project = new self($real === false ? $path : $real);
self::$created[] = $project;
self::copy(__DIR__.'/app', $project->path);
if ($overlay !== null) {
self::copy(__DIR__.'/overlays/'.$overlay, $project->path);
}
// `ChangedFiles` asks git what changed, so anything the scaffold writes
// but the project does not own has to be invisible to it.
$project->write('.gitignore', implode("\n", ['/vendor/', '/.home/', '/.phpunit.cache/', '']));
$project->scaffoldVendor($project->path);
@mkdir($project->home(), 0755, true);
return $project;
}
/**
* Mirrors a file or directory, hardlinking where the filesystem allows it
* and copying where it does not.
*/
private static function mirror(string $from, string $to): void
{
if (is_dir($from)) {
foreach (self::contentsOf($from) as $path) {
self::mirror($path->getPathname(), $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1));
}
return;
}
$directory = dirname($to);
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
}
if (@link($from, $to) || @copy($from, $to)) {
return;
}
throw new RuntimeException(sprintf('Unable to mirror [%s] into [%s].', $from, $to));
}
/**
* @return iterable<\SplFileInfo>
*/
private static function contentsOf(string $directory): iterable
{
$paths = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($directory, FilesystemIterator::SKIP_DOTS),
RecursiveIteratorIterator::SELF_FIRST,
);
foreach ($paths as $path) {
if (! $path->isDir()) {
yield $path;
}
}
}
private static function copy(string $from, string $to): void
{
$paths = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($from, FilesystemIterator::SKIP_DOTS),
RecursiveIteratorIterator::SELF_FIRST,
);
foreach ($paths as $path) {
$target = $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1);
if ($path->isDir()) {
if (! is_dir($target) && ! @mkdir($target, 0755, true) && ! is_dir($target)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $target));
}
continue;
}
if (! is_dir(dirname($target)) && ! @mkdir(dirname($target), 0755, true) && ! is_dir(dirname($target))) {
throw new RuntimeException(sprintf('Unable to create [%s].', dirname($target)));
}
if (! @copy($path->getPathname(), $target)) {
throw new RuntimeException(sprintf('Unable to copy [%s].', $path->getPathname()));
}
}
}
private function remove(string $path): void
{
if (! is_dir($path)) {
return;
}
$paths = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($path, FilesystemIterator::SKIP_DOTS),
RecursiveIteratorIterator::CHILD_FIRST,
);
foreach ($paths as $entry) {
$entry->isDir() && ! $entry->isLink() ? @rmdir($entry->getPathname()) : @unlink($entry->getPathname());
}
@rmdir($path);
}
}
+18
View File
@@ -0,0 +1,18 @@
<?php
declare(strict_types=1);
namespace Fixture\App;
final class Calculator
{
public function add(int $a, int $b): int
{
return $a + $b;
}
public function subtract(int $a, int $b): int
{
return $a - $b;
}
}
+13
View File
@@ -0,0 +1,13 @@
<?php
declare(strict_types=1);
namespace Fixture\App;
final class Greeter
{
public function greet(string $name): string
{
return sprintf('Hello, %s!', $name);
}
}
+9
View File
@@ -0,0 +1,9 @@
{
"name": "pest/tia-fixture",
"description": "Throwaway project the TIA scenario tests scaffold into a temp directory.",
"license": "MIT",
"require": {},
"config": {
"lock": true
}
}
+17
View File
@@ -0,0 +1,17 @@
{
"_readme": [
"Hand-written: the fixture project installs nothing. It exists so the TIA",
"fingerprint has a composer.lock to hash, the way a real project does."
],
"content-hash": "0000000000000000000000000000000000",
"packages": [],
"packages-dev": [],
"aliases": [],
"minimum-stability": "stable",
"stability-flags": {},
"prefer-stable": true,
"prefer-lowest": false,
"platform": {},
"platform-dev": {},
"plugin-api-version": "2.6.0"
}
+20
View File
@@ -0,0 +1,20 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
bootstrap="vendor/autoload.php"
cacheDirectory=".phpunit.cache"
colors="true"
failOnRisky="true"
failOnWarning="false"
>
<testsuites>
<testsuite name="default">
<directory suffix="Test.php">./tests</directory>
</testsuite>
</testsuites>
<source>
<include>
<directory suffix=".php">./app</directory>
</include>
</source>
</phpunit>
@@ -0,0 +1,15 @@
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
// A second test file over the same source file, so a `Calculator` edit narrows
// to two of the three test files rather than to one.
test('adds within a feature test', function (): void {
expect((new Calculator)->add(10, 5))->toBe(15);
});
test('subtracts within a feature test', function (): void {
expect((new Calculator)->subtract(10, 5))->toBe(5);
});
+9
View File
@@ -0,0 +1,9 @@
<?php
declare(strict_types=1);
// The fixture project has no composer autoloader of its own — its `vendor/`
// holds nothing but a shim pointing back at Pest's. Requiring the two classes
// here is enough for every test file in the suite.
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
@@ -0,0 +1,16 @@
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
// `test()` rather than `it()`: the harness seeds results by test id, and `it()`
// would prefix every description with `it `, leaving Project::TESTS a step away
// from what is written here.
test('adds two numbers', function (): void {
expect((new Calculator)->add(1, 2))->toBe(3);
});
test('subtracts two numbers', function (): void {
expect((new Calculator)->subtract(3, 1))->toBe(2);
});
@@ -0,0 +1,13 @@
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
});
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
@@ -0,0 +1,11 @@
<?php
declare(strict_types=1);
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
// Declared default branch. Wins over whatever the repository autodetects — the
// escape hatch for a checkout with no `origin/HEAD` and a misleading
// `init.defaultBranch`.
pest()->tia()->defaultBranch('master');
@@ -0,0 +1,10 @@
<?php
declare(strict_types=1);
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
// A branch the repository does not have. Accepted as configured — a name that
// resolves to no baseline degrades to a full run, which is safe.
pest()->tia()->defaultBranch('nope');
+4
View File
@@ -19,6 +19,10 @@ pest()->in('PHPUnit/GlobPatternTests/SubFolder2/*AsPattern.php')->use(CustomTest
pest()->in('Visual')->group('integration');
// Every row scaffolds a throwaway project and runs `pest` in it as a
// subprocess, which is far too slow for the unit suite.
pest()->in('Features/Tia')->group('integration');
// NOTE: global test value container to be mutated and checked across files, as needed
$_SERVER['globalHook'] = (object) ['calls' => (object) ['beforeAll' => 0, 'afterAll' => 0]];
+20 -2
View File
@@ -66,8 +66,26 @@ describe('applyMigrationChanges()', function (): void {
});
describe('rerun tracking', function (): void {
beforeEach(function (): void {
// `hasUnlocatedTestsToRerun()` stats each recorded file to tell a
// deleted test apart from a live one, so the files have to exist.
$this->projectRoot = sys_get_temp_dir().'/pest-tia-rerun-'.bin2hex(random_bytes(4));
mkdir($this->projectRoot.'/tests/Feature', 0755, true);
touch($this->projectRoot.'/tests/Feature/FooTest.php');
touch($this->projectRoot.'/tests/Feature/BarTest.php');
});
afterEach(function (): void {
@unlink($this->projectRoot.'/tests/Feature/FooTest.php');
@unlink($this->projectRoot.'/tests/Feature/BarTest.php');
@rmdir($this->projectRoot.'/tests/Feature');
@rmdir($this->projectRoot.'/tests');
@rmdir($this->projectRoot);
});
it('reruns cached failures via their file', function (): void {
$graph = new Graph(sys_get_temp_dir());
$graph = new Graph($this->projectRoot);
$graph->setResult('main', 'Tests\FooTest::it fails', 7, 'boom', 0.1, 1, 'tests/Feature/FooTest.php');
$graph->setResult('main', 'Tests\BarTest::it passes', 0, '', 0.1, 1, 'tests/Feature/BarTest.php');
@@ -76,7 +94,7 @@ describe('rerun tracking', function (): void {
});
it('flags cached failures whose file is unknown', function (): void {
$graph = new Graph(sys_get_temp_dir());
$graph = new Graph($this->projectRoot);
$graph->setResult('main', 'Tests\EvalTest::it fails', 7, 'boom', 0.1, 1);
expect($graph->testFilesToRerun('main'))->toBeEmpty()
+114
View File
@@ -0,0 +1,114 @@
<?php
declare(strict_types=1);
use Pest\Plugins\Tia\ResultCollector;
use Pest\Subscribers\EnsureTiaAssertionsAreRecordedOnFinished;
use Pest\Subscribers\EnsureTiaResultsAreCollected;
use PHPUnit\Event\Code\TestDox;
use PHPUnit\Event\Code\TestMethod;
use PHPUnit\Event\Telemetry\Duration;
use PHPUnit\Event\Telemetry\Info;
use PHPUnit\Event\Telemetry\MemoryUsage;
use PHPUnit\Event\Telemetry\System;
use PHPUnit\Event\Telemetry\SystemCpuTimeMeter;
use PHPUnit\Event\Telemetry\SystemGarbageCollectorStatusProvider;
use PHPUnit\Event\Telemetry\SystemMemoryMeter;
use PHPUnit\Event\Telemetry\SystemStopWatch;
use PHPUnit\Event\Test\Finished;
use PHPUnit\Event\Test\PreparationStarted;
use PHPUnit\Event\TestData\DataFromDataProvider;
use PHPUnit\Event\TestData\TestDataCollection;
use PHPUnit\Metadata\MetadataCollection;
function tiaResultKeyTestMethod(?string $dataSetName): TestMethod
{
$testData = $dataSetName === null
? TestDataCollection::fromArray([])
: TestDataCollection::fromArray([DataFromDataProvider::from($dataSetName, '', '')]);
return new TestMethod(
'Tests\Feature\OrderTest',
'it prices an order',
'/project/tests/Feature/OrderTest.php',
1,
new TestDox('Order', 'it prices an order', 'it prices an order'),
MetadataCollection::fromArray([]),
$testData,
);
}
function tiaResultKeyTelemetryInfo(): Info
{
$system = new System(
new SystemStopWatch,
new SystemMemoryMeter,
new SystemGarbageCollectorStatusProvider,
new SystemCpuTimeMeter,
);
$zeroDuration = Duration::fromSecondsAndNanoseconds(0, 0);
$zeroMemory = MemoryUsage::fromBytes(0);
$zeroCpuTime = $system->snapshot()->userCpuTime();
return new Info(
$system->snapshot(),
$zeroDuration,
$zeroMemory,
$zeroDuration,
$zeroMemory,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
);
}
it('keys a result per dataset row rather than per method', function (): void {
$collector = new ResultCollector;
$subscriber = new EnsureTiaResultsAreCollected($collector);
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')));
$collector->testPassed();
$collector->finishTest();
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('fake')));
$collector->testSkipped('the fake driver does not report balances');
$collector->finishTest();
// Without the dataset in the key both rows write to `Class::method`, so the
// second one overwrites the first and a replay hands every row the same
// status — a passing row reported as skipped, or a failing one as passed.
expect(array_keys($collector->all()))->toBe([
'Tests\Feature\OrderTest::it prices an order#opp',
'Tests\Feature\OrderTest::it prices an order#fake',
]);
});
it('records assertions against the same per-dataset key', function (): void {
$collector = new ResultCollector;
new EnsureTiaResultsAreCollected($collector)->notify(
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')),
);
$collector->testPassed();
new EnsureTiaAssertionsAreRecordedOnFinished($collector)->notify(
new Finished(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp'), 7),
);
expect($collector->all()['Tests\Feature\OrderTest::it prices an order#opp']['assertions'])->toBe(7);
});
it('leaves a test without a dataset keyed by class and method', function (): void {
$collector = new ResultCollector;
new EnsureTiaResultsAreCollected($collector)->notify(
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod(null)),
);
$collector->testPassed();
expect(array_keys($collector->all()))->toBe(['Tests\Feature\OrderTest::it prices an order']);
});
+17
View File
@@ -47,6 +47,15 @@ describe('fromSql()', function (): void {
->and(TableExtractor::fromSql('select * from information_schema.tables'))->toBeEmpty();
});
it('does not leak int keys for numeric identifiers', function (): void {
// `substring(x FROM 1 FOR 3)` is standard SQL, and the `1` matches the
// FROM pattern. Collecting names as array keys makes PHP coerce the
// numeric string to an int, which then violates the declared
// list<string> and blows up Recorder::linkTable(string).
expect(TableExtractor::fromSql('select substring(name from 1 for 3) from users'))
->each->toBeString();
});
it('returns nothing for non-DML statements', function (): void {
expect(TableExtractor::fromSql('PRAGMA foreign_keys = ON'))->toBeEmpty()
->and(TableExtractor::fromSql(''))->toBeEmpty()
@@ -89,6 +98,14 @@ describe('fromMigrationSource()', function (): void {
->toBe(['audits', 'events', 'sessions', 'settings', 'users']);
});
it('does not leak int keys for numeric table names', function (): void {
// A table named `123` is a legal quoted identifier. Collecting names as
// array keys makes PHP coerce it to an int, breaking the declared
// list<string>, so it must survive as a string rather than be dropped.
expect(TableExtractor::fromMigrationSource("DB::table('123')->insert([]);"))
->toBe(['123']);
});
it('extracts tables from DB::table calls', function (): void {
expect(TableExtractor::fromMigrationSource("DB::table('permissions')->insert([]);"))
->toBe(['permissions']);
+1 -1
View File
@@ -15,7 +15,7 @@ it('does not allow to add the same test description twice', function (): void {
$testSuite->tests->set($method);
})->throws(
TestAlreadyExist::class,
sprintf('A test with the description `%s` already exists in the filename `%s`.', 'bar', 'foo'),
sprintf('A test with the description [%s] already exists in the filename [%s].', 'bar', 'foo'),
);
it('does not allow static closures', function (): void {
+8 -8
View File
@@ -29,30 +29,30 @@ $run = function () {
$normalizedPath = (fn (string $path): string => str_replace('/', DIRECTORY_SEPARATOR, $path));
test('junit output', function () use ($normalizedPath, $run): void {
$result = $run('tests/.tests/SuccessOnly.php');
$result = $run('tests/Fixtures/Suites/SuccessOnly.php');
expect($result['testsuite']['@attributes'])
->name->toBe('Tests\tests\SuccessOnly')
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
->tests->toBe('4')
->assertions->toBe('4')
->errors->toBe('0')
->failures->toBe('0')
->skipped->toBe('0')
->and($result['testsuite']['testcase'])->toHaveCount(2)
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
});
test('junit with parallel', function () use ($normalizedPath, $run): void {
$result = $run('tests/.tests/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
$result = $run('tests/Fixtures/Suites/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
expect($result['testsuite']['@attributes'])
->name->toBe('Tests\tests\SuccessOnly')
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
->tests->toBe('1')
->assertions->toBe('1')
->errors->toBe('0')
->failures->toBe('0')
->skipped->toBe('0')
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
});
+12 -4
View File
@@ -24,13 +24,13 @@ test('parallel', function () use ($run): void {
$file = file_get_contents(__FILE__);
$file = preg_replace(
'/\$expected = \'.*?\';/',
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';",
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';",
$file,
);
file_put_contents(__FILE__, $file);
}
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';
expect($output)
->toContain("Tests: {$expected}")
@@ -42,8 +42,8 @@ test('a parallel test can extend another test with same name', function () use (
})->skipOnWindows();
test('parallel reports invalid datasets as failures', function () use ($run): void {
expect($run('tests/.tests/ParallelInvalidDataset'))
->toContain("A dataset with the name `missing.dataset` does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
expect($run('tests/Fixtures/Suites/ParallelInvalidDataset'))
->toContain("A dataset with the name [missing.dataset] does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
->toContain('Tests: 1 failed, 1 passed (1 assertions)')
->toContain('Parallel: 3 processes');
})->skipOnWindows();
@@ -58,3 +58,11 @@ test('parallel can have multiple exclude-groups', function () use ($run): void {
expect((int) $doubleMatch[1])->toBeLessThan((int) $singleMatch[1])
->and($doubleExclude)->toContain('Parallel: 3 processes');
})->skipOnWindows();
test('parallel can have multiple groups', function () use ($run): void {
$output = $run('tests/Fixtures/Suites/MultipleGroups', '--group=one', '--group=two');
expect($output)
->toContain('Tests: 2 passed (2 assertions)')
->toContain('Parallel: 3 processes');
})->skipOnWindows();
+1 -1
View File
@@ -27,7 +27,7 @@ test('allows to run a single test', function () use ($run, $snapshot): void {
})->skipOnWindows();
test('allows to run a directory', function () use ($run, $snapshot): void {
expect($run('tests/Fixtures'))->toContain($snapshot('allows-to-run-a-directory'));
expect($run('tests/Fixtures/DirectoryWithTests'))->toContain($snapshot('allows-to-run-a-directory'));
})->skipOnWindows();
it('disable decorating printer when colors is set to never', function () use ($snapshot): void {
+1 -1
View File
@@ -10,7 +10,7 @@ function normalize_windows_os_output(string $text): string
}
test('visual snapshot of team city', function (string $testFile): void {
$testsPath = dirname(__DIR__)."/.tests/$testFile";
$testsPath = dirname(__DIR__)."/Fixtures/Suites/$testFile";
$snapshot = implode(DIRECTORY_SEPARATOR, [
dirname(__DIR__),
+2 -2
View File
@@ -8,7 +8,7 @@ test('filter works with unicode characters in filename', function (): void {
$process = new Process([
'php',
'bin/pest',
'tests/.tests/StraßenTest.php',
'tests/Fixtures/Suites/StraßenTest.php',
'--colors=never',
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);
@@ -26,7 +26,7 @@ test('filter with unicode regex matches unicode filename', function (): void {
'php',
'bin/pest',
'--filter=.*Straß.*',
'tests/.tests/',
'tests/Fixtures/Suites/',
'--colors=never',
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);