mirror of
https://github.com/pestphp/pest.git
synced 2026-09-05 14:23:34 +02:00
Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 nuno maduro
|
d0572ce138 | ||
|
nuno maduro
|
b0d1fc589d | ||
|
nuno maduro
|
5a4f124199 | ||
|
nuno maduro
|
c3ceb7d0c7 | ||
|
nuno maduro
|
16d1208344 | ||
|
nuno maduro
|
1cebc84c2d | ||
|
nuno maduro
|
1f79660add | ||
|
nuno maduro
|
37bdf60a1c | ||
|
nuno maduro
|
cc7acfe485 | ||
|
nuno maduro
|
7bfb2a6185 | ||
|
nuno maduro
|
ca56ca5216 | ||
|
nuno maduro
|
b49ba062d5 | ||
|
nuno maduro
|
09699847a2 | ||
|
nuno maduro
|
545d0c2784 | ||
|
nuno maduro
|
b795af3b1b | ||
|
nuno maduro
|
a40cc6bc0e | ||
|
nuno maduro
|
4d3d0105b7 |
@@ -16,3 +16,19 @@ composer test:integration # visual and snapshot tests
|
||||
composer test # everything CI runs, in CI's order
|
||||
composer update:snapshots # only when a test was added or removed
|
||||
```
|
||||
|
||||
## TIA scenario tests
|
||||
|
||||
`tests/Features/Tia/*` scaffold a throwaway git project, run a real `pest` subprocess against it, and diff the TIA graph it wrote. They exist because TIA's contract is about what a run *writes* — replay, branch keys, and the COMPLETE / RESULTS-ONLY / HARD-SUPPRESSED tiers are invisible to ordinary assertions, and every case used to be measured by hand against a playground app.
|
||||
|
||||
Add one whenever a change touches branch resolution, replay, filtered mode, or the write tiers. How:
|
||||
|
||||
- `Project::make('master')` scaffolds; `seed('master')` writes a graph and sentinels every cached result (`time=9.999`, `assertions=42`) so any rewrite shows up.
|
||||
- `$project->pest('--tia', …)` runs it; `$project->delta()` compares against that snapshot. `writtenCount()` is the discriminator — `0` means "replayed", not "wrote the same values". `mutateGraph()` bends one entry; overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`.
|
||||
- Keep expectations driver-independent: a cold recording run needs pcov/xdebug and behaves differently without one. Seed a graph instead of recording one.
|
||||
|
||||
Run them by file (a directory argument finds nothing) or by `--filter`:
|
||||
|
||||
```bash
|
||||
php bin/pest tests/Features/Tia/PartialRunWriteTier.php
|
||||
```
|
||||
|
||||
@@ -1,342 +0,0 @@
|
||||
# TIA write-tier conformance — plan
|
||||
|
||||
Baseline sweep: **147 cases, 144 PASS / 2 FAIL / 1 SKIP** against `dev-fix/tia-filtered` (`db70017`)
|
||||
on `playground/laravel` with pcov enabled.
|
||||
|
||||
This file records (1) what is left to fix, and (2) the full case matrix with the target
|
||||
outcome for every row, so the sweep can be re-run as a conformance check after the fixes land.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — What's left
|
||||
|
||||
### 1. `--parallel` silently under-records `edges` (G12, I5) — **high**
|
||||
|
||||
Any `--tia --parallel` complete run writes `files=4` where sequential writes `files=25`. Only
|
||||
`app/**` and blade views survive; every `config/`, `routes/`, `bootstrap/` edge and every test
|
||||
self-edge is lost. A `routes/web.php` edit that sequential TIA catches, parallel TIA reports as
|
||||
`No affected tests found`.
|
||||
|
||||
Root cause — `src/Restarters/PcovRestarter.php:33`:
|
||||
|
||||
```php
|
||||
if (! Tia::isEnabledForRun($arguments)) {
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
`Tia::isEnabledForRun()` (`src/Plugins/Tia.php:275-288`) returns true only for `--tia` in argv or
|
||||
the `PEST_TIA` env flag. A paratest worker's argv carries neither, so workers never re-exec with
|
||||
`-d pcov.directory=<projectRoot>` and record under pcov's empty default scope.
|
||||
|
||||
Proven both directions:
|
||||
- `PEST_PCOV_RESTARTER_RESTARTED=1 pest --tia` (sequential, restart suppressed) reproduces
|
||||
`files=4` with edge sets identical to the parallel graph.
|
||||
- `PEST_TIA=1 pest --parallel` restores the full `files=25` graph. **Working workaround today.**
|
||||
|
||||
Coverage config is irrelevant: widening `phpunit.xml` `<include>` to `app|config|routes|bootstrap|tests`
|
||||
still yields `files=4`.
|
||||
|
||||
Sticky, not just per-run: `pest --tia --filtered --parallel` re-records only the affected test
|
||||
through a worker and strips that entry's edges from an otherwise-healthy sequential graph, so one
|
||||
parallel run degrades a good graph incrementally.
|
||||
|
||||
**Fix direction:** propagate the TIA-enabled decision into workers (e.g. set `PEST_TIA` in the
|
||||
worker env, or have the parent stamp a recording global the restarter also consults) so
|
||||
`PcovRestarter` restarts them. Then assert `edges` equivalence between sequential and parallel.
|
||||
|
||||
### 2. Deleted test file with a cached failure wedges `--filtered` forever (I9) — **high**
|
||||
|
||||
Expected a WARN and a full-suite fallback. Actual: `--filtered` selects the phantom file, runs
|
||||
nothing, exits **0 green**, on every subsequent invocation.
|
||||
|
||||
Root cause — `src/Plugins/Tia/Graph.php:1523-1525`:
|
||||
|
||||
```php
|
||||
if ($real === false) {
|
||||
$real = $path;
|
||||
}
|
||||
```
|
||||
|
||||
`relative()` falls back to the raw path when `realpath()` fails, so a deleted-but-in-project test
|
||||
file is never "unlocated", `Graph::hasUnlocatedTestsToRerun()` never fires, and the WARN branch at
|
||||
`src/Plugins/Tia.php:982-986` is unreachable for the deletion case. `pruneMissingTests()` only runs
|
||||
under `--fresh`, so nothing clears it.
|
||||
|
||||
**Fix direction:** distinguish "path could not be resolved" from "path resolved outside the root" in
|
||||
`relative()`, or have `hasUnlocatedTestsToRerun()` stat the file directly.
|
||||
|
||||
### 3. `--coverage-text` does not disable filtered mode (I8) — **low**
|
||||
|
||||
`coverageReportActive()` (`src/Plugins/Tia.php:1705-1710`) reads only Pest's own `--coverage`, so
|
||||
raw PHPUnit coverage flags (`--coverage-text`, `--coverage-html`, `--coverage-clover`, …) leave
|
||||
filtered mode on. `Tia::COVERAGE_FLAGS` at `:113-115` already enumerates them — that list is not
|
||||
consulted here.
|
||||
|
||||
### 4. `--coverage` narrows edges like parallel does — **medium, needs triage**
|
||||
|
||||
`pest --tia --coverage` prints `fresh graph (recording coverage baseline)` even when a graph
|
||||
exists, and writes `Feature/ExampleTest` with 2 files instead of 16, dropping self-edges. Same
|
||||
observable shape as G12, likely a different mechanism (piggyback collector scoped by
|
||||
`phpunit.xml <include>` rather than the pcov-restarted recorder). Theme: **edges silently narrow to
|
||||
`app/**` whenever recording does not go through the pcov-restarted TIA recorder.** Confirm whether
|
||||
these are one fix or two.
|
||||
|
||||
### 5. Smaller items
|
||||
|
||||
| Item | Where | Note |
|
||||
|---|---|---|
|
||||
| Warning/deprecation recorded as `status=0` | write path | Codes `6` and `4` appear unreachable; risky/skipped/incomplete record faithfully |
|
||||
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
|
||||
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
|
||||
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
|
||||
| Replay clobbers cached `time` | write path | `0.058` → `0.001`; timing data degrades toward zero across runs |
|
||||
| `pest --parallel` without `--tia` writes nothing | G3, G6 | Sequential `pest --filter=…` does record. Parallel CI contributes nothing to the cached-failure replay path |
|
||||
| `--tia --parallel --filter`/`--shard` record nothing | G2, G8 | More conservative than the results-only contract requires |
|
||||
| `--min=50` without `--coverage` is a silent no-op | — | — |
|
||||
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
|
||||
|
||||
### 6. Test-harness gaps to close before re-running
|
||||
|
||||
- The playground has no `UsesClass`, `->note()`, `->flaky()`, `->issue()`, `->pr()`, `->ticket()` or
|
||||
`->assignee()` annotations, so **C19, C23–C28, C37–C39 matched zero tests** and their
|
||||
graph-invariant assertions proved nothing. Add annotated fixtures to make those rows load-bearing.
|
||||
- Use **sentinel patching** (rewrite every cached entry to `time=9.999 assertions=42`, then see which
|
||||
entries get overwritten) as the discriminator. It is the only way to tell "wrote identical values"
|
||||
from "wrote nothing". A canary test absent from `edges` is unreliable under `--filter` because it
|
||||
never matches the filter and so never runs.
|
||||
- `--tia` on a clean green tree can never cache a failure (unchanged tests replay rather than
|
||||
execute); seeding one requires `--fresh` or an env-driven flaky fixture.
|
||||
- Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix (target outcomes)
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Status column: `PASS` = conforming today · `FIX` = must pass once the item above lands ·
|
||||
`SKIP` = case not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | PASS |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | PASS |
|
||||
| A4 | `git status` after reset ritual | clean | PASS |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | PASS |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | PASS |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | PASS |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | PASS |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS |
|
||||
| B10 | `pest --tia` twice | `time` differs; statuses stable | PASS |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key + new result appear | PASS |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written | PASS |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS |
|
||||
| D11 | `pest --tia --bail` | RO | PASS |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran | PASS |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | PASS |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | PASS |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written | PASS |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | PASS (but see G12 — it strips edges) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | PASS |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | PASS |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** (`files=25` both) | **FIX (item 1)** |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | PASS |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | PASS |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **FIX (item 1)** |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report | **FIX (item 3)** |
|
||||
| I9 | cached failure whose test file was deleted | WARN `could not be located on disk`; falls back to the full suite with replay | **FIX (item 2)** |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild (and the drift reason should be printed — item 5) | PASS |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE | PASS |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (signal must go to the re-exec'd child — item 5) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS |
|
||||
|
||||
---
|
||||
|
||||
## Re-run procedure
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json"
|
||||
|
||||
# per case
|
||||
git checkout -q . && git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
Every pest invocation must be prefixed with `PAO_DISABLE=1` (the app has `laravel/pao`, which emits
|
||||
JSON when it detects an agent). The shell is zsh — build commands with arrays or `eval`; unquoted
|
||||
`$args` does not word-split.
|
||||
@@ -1,455 +0,0 @@
|
||||
# TIA default-branch fallback — phase three
|
||||
|
||||
## Your task
|
||||
|
||||
Fix [pestphp/pest#1823](https://github.com/pestphp/pest/issues/1823) — TIA's cached results never
|
||||
hit for repos whose default branch is not literally `main` — then re-run the **whole** conformance
|
||||
matrix against the playground app, plus the new section **L** that covers the fix.
|
||||
|
||||
Work in this order, and **stop where the plan says stop**:
|
||||
|
||||
1. **Part 1** — read the diagnosis. It is already measured; do not re-derive it, but do re-confirm
|
||||
the two reproductions in Part 1.3 take ~2 minutes and prove your environment is sane.
|
||||
2. **Part 2** — implement the fix in the `pestphp/pest` repo. **Do not commit. Do not touch the
|
||||
playground's `vendor/`.**
|
||||
3. **HARD STOP → Part 3.** Report the diff to Nuno and wait. He validates, commits, and syncs it
|
||||
into the playground himself. You must not proceed until he confirms.
|
||||
4. **Part 4** — verify the sync landed, then run section **L** (new) and the full phase-two matrix
|
||||
(**A–K**, all 156 rows) against the playground.
|
||||
5. **Part 5** — report in the given format.
|
||||
|
||||
Per `CLAUDE.md`: **do not write new `pestphp/pest` unit tests and do not run `composer test`.** Make
|
||||
the change, report it, and ask whether repo tests should be added. The section-L rows are *playground
|
||||
invocations*, not repo tests — those are the deliverable and are always in scope.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — The diagnosis (already measured; commit `bfd5b756`)
|
||||
|
||||
### 1.1 Root cause — two independent hardcoded `'main'` literals
|
||||
|
||||
**(a) The read fallback.** `src/Plugins/Tia/Graph.php` — seven methods default the fallback to the
|
||||
literal `'main'`:
|
||||
|
||||
| line | method |
|
||||
|---|---|
|
||||
| 579 | `recordedAtSha(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 614 | `getAssertions(…, string $fallbackBranch = 'main')` |
|
||||
| 625 | `getTime(…, string $fallbackBranch = 'main')` |
|
||||
| 636 | `getResult(…, string $fallbackBranch = 'main')` |
|
||||
| 663 | `testFilesToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 700 | `hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 811 | `lastRunTree(string $branch, string $fallbackBranch = 'main')` |
|
||||
|
||||
They all funnel into `Graph::baselineFor()` (line 819), which *does* implement a real cross-branch
|
||||
fallback:
|
||||
|
||||
```php
|
||||
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
|
||||
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
|
||||
return ['sha' => null, 'tree' => [], 'results' => []];
|
||||
```
|
||||
|
||||
The mechanism is deliberate. The problem is that **no caller ever passes `$fallbackBranch`** — all
|
||||
nine call sites in `src/Plugins/Tia.php` (lines 419, 429, 432, 785, 857, 1026, 1031, 1048, 1056) pass
|
||||
only `$this->branch`. So on a `master`-named repo the second branch of `baselineFor()` can never
|
||||
fire, and the whole mechanism is dead code.
|
||||
|
||||
`grep -rniE 'defaultBranch|symbolic-ref|init\.defaultBranch|origin/HEAD' src/` returns **nothing** —
|
||||
no default-branch resolution exists anywhere.
|
||||
|
||||
**(b) The detached-HEAD default.** `src/Plugins/Tia.php:206` declares `private string $branch = 'main';`
|
||||
and `ChangedFiles::currentBranch()` (`ChangedFiles.php:208`) returns `null` for detached HEAD. So on
|
||||
detached HEAD `$this->branch` stays the literal `'main'` and is used for **both reads and writes** —
|
||||
minting a baseline key for a branch that does not exist. This is a *write*-side bug and a separate fix
|
||||
from (a).
|
||||
|
||||
### 1.2 Not a regression
|
||||
|
||||
`git log -S"fallbackBranch = 'main'"` bottoms out at `c7e32f5d feat(tia): continues to work on poc`.
|
||||
This is original PoC code, untouched by phase one. Phase one's change 1 modified
|
||||
`hasUnlocatedTestsToRerun()`'s file-existence check — one of the seven methods — without going near
|
||||
the fallback. Do not report it as a phase-one regression.
|
||||
|
||||
No test anywhere exercises the mechanism: `tests/Unit/Plugins/Tia/Graph.php` uses `'main'` as the
|
||||
*actual* branch name, so those assertions pass whether or not the fallback exists. A branch-name
|
||||
mismatch is never tested.
|
||||
|
||||
### 1.3 The two reproductions (re-confirm these before you start)
|
||||
|
||||
Both on the playground, `master`-named default branch, zero local changes:
|
||||
|
||||
```
|
||||
default = master default = main
|
||||
1. record on default → full run (cold) → full run (cold)
|
||||
2. 1st run on feature-x → 25 UNCACHED → 25 replayed ← (a)
|
||||
3. 2nd run on feature-x → 25 replayed → 25 replayed
|
||||
4. back on default → 25 replayed → 25 replayed
|
||||
5. 1st run on feature-y → 25 UNCACHED → 25 replayed
|
||||
```
|
||||
|
||||
```
|
||||
master-only graph, then `git checkout --detach`, then `pest --tia`:
|
||||
→ 25 uncached, and keys become [master,main] ← (b) spurious key
|
||||
```
|
||||
|
||||
The cost is **one full run per new branch, forever**, with no output explaining why — the only clue
|
||||
is the `N uncached` count; the headline is just `─ Experimental TIA mode enabled.`
|
||||
|
||||
### 1.4 Correction to phase two
|
||||
|
||||
Phase two reported **H6 and H7 as passing. Both were false passes.** They ran after H5, which had
|
||||
renamed `master`→`main` and left a `main` key in the graph, so the hardcoded fallback resolved by
|
||||
accident. Re-measured against a clean `master`-only graph, both fail. Section L replaces them as the
|
||||
load-bearing rows; H6/H7 must be re-run **from a cold graph** this time (see Part 4.2).
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — The fix to implement
|
||||
|
||||
### 2.1 Recommended design
|
||||
|
||||
**Step 1 — add a non-throwing resolver** to `src/Plugins/Tia/ChangedFiles.php`, next to
|
||||
`currentBranch()`:
|
||||
|
||||
```php
|
||||
public function defaultBranch(): ?string
|
||||
```
|
||||
|
||||
Resolution order, each step failing soft to the next:
|
||||
|
||||
1. `git symbolic-ref --short refs/remotes/origin/HEAD` → strip a leading `origin/`
|
||||
2. `git config --get init.defaultBranch`
|
||||
3. `null`
|
||||
|
||||
Unlike `currentBranch()`, this must **never throw** `MissingDependency` — it is advisory. Return
|
||||
`null` on any non-zero exit or empty output.
|
||||
|
||||
**Step 2 — add a config surface.** `src/Plugins/Tia/Configuration.php` already exposes `always()`,
|
||||
`locally()`, `filtered()`, `baselined()`, `watch()`. Add:
|
||||
|
||||
```php
|
||||
public function defaultBranch(string $branch): self
|
||||
```
|
||||
|
||||
so `pest()->tia()->defaultBranch('master')` works in `tests/Pest.php`. Explicit config **always
|
||||
wins** over autodetection — that is the escape hatch when `origin/HEAD` is unset.
|
||||
|
||||
**Step 3 — resolve once, in `Tia.php`.** The read path is hot (`getResult()` is called per test at
|
||||
line 419), so resolution must not shell out per call. Resolve alongside `$this->branch` at
|
||||
`Tia.php:1910`, under the existing `$branchResolved` guard:
|
||||
|
||||
```php
|
||||
$this->fallbackBranch = $configuredDefaultBranch
|
||||
?? $changedFiles->defaultBranch()
|
||||
?? 'main';
|
||||
```
|
||||
|
||||
**Step 4 — thread it into `Graph`.** Prefer a `Graph`-level property over editing nine call sites:
|
||||
add `Graph::setFallbackBranch(string $branch)`, change the seven signatures to
|
||||
`?string $fallbackBranch = null`, and resolve inside each with
|
||||
`$fallbackBranch ??= $this->fallbackBranch;`. `baselineFor()` itself needs no change. This keeps the
|
||||
public signatures backward-compatible and minimises blast radius.
|
||||
|
||||
**Step 5 — fix the detached-HEAD write.** `Tia.php:206`'s `= 'main'` default must become the
|
||||
resolved default branch, so detached HEAD stops minting a phantom key.
|
||||
|
||||
### 2.2 Invariants the fix must not break
|
||||
|
||||
These are all covered by existing matrix rows — the fix is wrong if any of them moves:
|
||||
|
||||
- **Read-only.** The fallback must affect *reads* only. Writes go through `ensureBaseline($branch)`
|
||||
and must keep using the real current branch. Otherwise H1–H4/H8 ("no baseline key other than the
|
||||
real branch") break.
|
||||
- **H9** — a non-git dir with `--tia` must still raise
|
||||
`MissingDependency: The feature "Tia mode" requires "git".` Adding a soft resolver must not
|
||||
swallow that.
|
||||
- **H10** — plain `pest` in a non-git dir must still run and create no baseline dir.
|
||||
- **A2/A3** — cold-graph recording unchanged.
|
||||
- **I1/E3** — clean+green `--tia --filtered` must still be a true zero-delta run.
|
||||
- Filtered mode reads `testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so the fallback must
|
||||
reach those two as well, not just `getResult()`.
|
||||
|
||||
### 2.3 Decisions for Nuno (raise these at the Part 3 stop)
|
||||
|
||||
- **D1** — Consult `origin/HEAD` at all? It requires `git remote set-head` and is absent in many CI
|
||||
checkouts and all remote-less repos. Config + `init.defaultBranch` only is simpler but helps fewer
|
||||
people out of the box. *Recommendation: keep it, first in the chain, since it fails soft.*
|
||||
- **D2** — Single-key heuristic: if the graph holds exactly one baseline key, use it as the fallback?
|
||||
Fixes the issue with zero git calls, but is implicit and surprising when several keys exist.
|
||||
*Recommendation: no.*
|
||||
- **D3** — Should detached HEAD write a baseline at all, or be read-only? Current behaviour mints a
|
||||
key. *Recommendation: read-only.*
|
||||
- **D4** — Should `pest()->tia()->defaultBranch()` validate that the branch exists, or accept any
|
||||
string? *Recommendation: accept any string; a nonexistent name degrades to a full run, which is
|
||||
safe.*
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — HARD STOP
|
||||
|
||||
When the code is written:
|
||||
|
||||
1. Show Nuno the diff (`git -C /Users/nunomaduro/Work/projects/pestphp/pest diff`) and a one-paragraph
|
||||
summary of each file's change.
|
||||
2. Answer/raise the D1–D4 decisions.
|
||||
3. State explicitly that you have **not** committed and have **not** synced `vendor/`.
|
||||
4. Ask whether repo unit tests should be added (per `CLAUDE.md`), describing the tests you have in
|
||||
mind — do not write them yet.
|
||||
5. **Wait.** Nuno commits and applies the change to the playground.
|
||||
|
||||
**Never sync the playground's `vendor/` yourself.** `vendor/pestphp/pest` there is a dist copy, not a
|
||||
symlink (composer installed `dev-fix/tia-filtered as 5.2.0`), so pest-repo edits do not reach it. Say
|
||||
what is stale and wait. This applies to before/after contrasts too.
|
||||
|
||||
Once he confirms, verify the sync actually landed before measuring anything:
|
||||
|
||||
```bash
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
grep -c 'defaultBranch' "$V/src/Plugins/Tia/ChangedFiles.php" # must be ≥ 1
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php src/Plugins/Tia/ChangedFiles.php \
|
||||
src/Plugins/Tia/Configuration.php; do
|
||||
diff -q "/Users/nunomaduro/Work/projects/pestphp/pest/$f" "$V/$f" >/dev/null \
|
||||
&& echo "SAME $f" || echo "STALE $f"
|
||||
done
|
||||
```
|
||||
|
||||
If anything reports `STALE`, stop and tell him. State in your final report which pest commit produced
|
||||
the playground numbers.
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Measurement
|
||||
|
||||
### 4.1 Environment and traps
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel` (branch `master`).
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON when
|
||||
it detects an agent.
|
||||
2. **Pin the interpreter to `php85`.** The playground requires PHP `>= 8.4.1` *and* pcov. Locally only
|
||||
`php85` (8.5.8) has both — `php84` (8.4.23) has no pcov, and the bare Herd `php` shim has been
|
||||
observed drifting to 8.3.32 mid-session, which kills every run in
|
||||
`vendor/composer/platform_check.php`. Also put a `php` → `php85` symlink first on `PATH`: the
|
||||
`--shard` list-tests probe spawns a subprocess via bare `php`, not `PHP_BINARY`.
|
||||
3. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`. Scope
|
||||
resets to `git checkout -- tests app` / `git clean -fd tests app`. For `phpunit.xml` and
|
||||
`composer.lock`, copy aside and copy back, verifying with `shasum`.
|
||||
4. **zsh does not word-split unquoted parameters.** A `$PEST` string containing a space becomes one
|
||||
command name. Route every invocation through `eval` (the `pest()` helper below does this).
|
||||
5. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
6. **The sentinel technique must not falsify `assertions` on zero-assertion tests.** A
|
||||
risky/skipped/incomplete status is *derived* from "performed no assertions", so patching those to
|
||||
`42` rewrites the status on replay and destroys the discriminator — it shows up as a phantom
|
||||
`status 5→0` defect. The `sentinel.php` below only patches `assertions` where it is already
|
||||
non-zero. With that, a full replay gives a clean `rewritten=0`.
|
||||
7. Restore branch state after every L row. Several rows rename or detach; leaving a stray `main` key
|
||||
in the graph is exactly what produced phase two's false H6/H7 passes.
|
||||
|
||||
### 4.2 Reference numbers
|
||||
|
||||
The playground already carries the phase-two fixtures at commit `fb2e77e` — **do not rebuild them.**
|
||||
A healthy sequential `--tia` graph is:
|
||||
|
||||
> **`files=27`, 10 edge keys (one per test file), self-edge on all 10, `n=25` results.**
|
||||
|
||||
`sha` will differ once Nuno commits the vendor sync — re-derive it once and use it throughout. The
|
||||
`files`/`edges`/`n` numbers hold as long as no test fixture changes. Suite shape: 10 test files, 25
|
||||
tests, including six deliberate status fixtures (skipped, todo, incomplete, risky, warning,
|
||||
deprecation), a 3-row dataset, a `smoke` group, an env-driven flaky test (green unless
|
||||
`FLAKY_FAIL=1`), and the annotation set (`covers`/`note`/`flaky`/`issue`/`pr`/`ticket`/`assignee`).
|
||||
|
||||
**Re-run H6 and H7 from a cold graph** (`rm -rf` the graph dir, record on `master` only, *then*
|
||||
branch/detach). Their phase-two results are void.
|
||||
|
||||
### 4.3 Harness
|
||||
|
||||
Write these to your scratchpad. `$SP` is your own scratchpad dir.
|
||||
|
||||
<details>
|
||||
<summary><code>lib.sh</code></summary>
|
||||
|
||||
```bash
|
||||
#!/bin/zsh
|
||||
export PAO_DISABLE=1
|
||||
PG=/Users/nunomaduro/Work/projects/playground/laravel
|
||||
SP="<your scratchpad>"
|
||||
PHPBIN="php85"
|
||||
PEST="$PHPBIN $PG/vendor/bin/pest"
|
||||
cd "$PG" || exit 1
|
||||
pest() { eval "$PEST $*"; } # zsh: no word-splitting, must eval
|
||||
GRAPHDIR="$(pest --baseline)"
|
||||
GRAPH="$GRAPHDIR/graph.json"
|
||||
mkdir -p "$SP/bin" && ln -sf "$(command -v php85)" "$SP/bin/php"
|
||||
export PATH="$SP/bin:$PATH" # --shard spawns bare `php`
|
||||
reset_tree() { git checkout -- tests app 2>/dev/null; git clean -qfd tests app 2>/dev/null; }
|
||||
seed() { rm -rf "$GRAPHDIR"; pest --tia >/dev/null 2>&1; $PHPBIN "$SP/sentinel.php" "$GRAPH" >/dev/null; cp "$GRAPH" "$SP/before.json"; }
|
||||
snap() { cp "$GRAPH" "$SP/before.json"; }
|
||||
delta() { $PHPBIN "$SP/cmp.php" "$SP/before.json" "$GRAPH"; }
|
||||
keys() { $PHPBIN -r '$g=json_decode(file_get_contents($argv[1]),true);echo "[".implode(",",array_keys($g["baselines"]??[]))."]";' "$GRAPH"; }
|
||||
tally() { sed -E $'s/\x1b\\[[0-9;]*[a-zA-Z]//g' <"$SP/out.txt" | grep -E 'Tests:' | sed -E 's/^ +//;s/Tests: +//'; }
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>sentinel.php</code> — the write discriminator</summary>
|
||||
|
||||
```php
|
||||
<?php // sentinel.php <graph.json>
|
||||
$p = $argv[1];
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR);
|
||||
$n = 0;
|
||||
foreach ($g['baselines'] ?? [] as $br => $b) {
|
||||
foreach (array_keys($b['results'] ?? []) as $id) {
|
||||
$g['baselines'][$br]['results'][$id]['time'] = 9.999;
|
||||
// Only falsify a non-zero assertion count: risky/skipped/incomplete are
|
||||
// DERIVED from "performed no assertions", so patching those to 42 would
|
||||
// rewrite the status on replay and destroy the discriminator.
|
||||
if ((int) ($b['results'][$id]['assertions'] ?? 0) > 0) {
|
||||
$g['baselines'][$br]['results'][$id]['assertions'] = 42;
|
||||
}
|
||||
$n++;
|
||||
}
|
||||
}
|
||||
file_put_contents($p, json_encode($g, JSON_THROW_ON_ERROR));
|
||||
echo "sentinelled $n results\n";
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>oneline.php</code> — one compact tier verdict per row</summary>
|
||||
|
||||
```php
|
||||
<?php // oneline.php <before.json> <after.json>
|
||||
function load(string $p): ?array {
|
||||
return is_file($p) ? json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR) : null;
|
||||
}
|
||||
$a = load($argv[1]); $b = load($argv[2]);
|
||||
if ($a === null || $b === null) { echo 'GRAPH '.($b === null ? 'DELETED' : 'CREATED'); exit; }
|
||||
function edgeSets(array $g): array {
|
||||
$out = [];
|
||||
foreach ($g['edges'] ?? [] as $t => $ids) { $s = array_map(fn($i) => $g['files'][$i] ?? "?$i", (array) $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$moved = [];
|
||||
if (edgeSets($a) !== edgeSets($b)) $moved[] = 'edges';
|
||||
if (($a['files'] ?? []) !== ($b['files'] ?? [])) $moved[] = 'files';
|
||||
if (($a['fingerprint'] ?? null) !== ($b['fingerprint'] ?? null)) $moved[] = 'fingerprint';
|
||||
$brA = array_keys($a['baselines'] ?? []); $brB = array_keys($b['baselines'] ?? []);
|
||||
if ($brA !== $brB) $moved[] = 'branchkeys('.implode('|', $brA).'->'.implode('|', $brB).')';
|
||||
$add = $rem = $wr = 0; $shaMoved = $treeMoved = false;
|
||||
foreach ($brB as $br) {
|
||||
$ra = $a['baselines'][$br]['results'] ?? []; $rb = $b['baselines'][$br]['results'] ?? [];
|
||||
if (($a['baselines'][$br]['sha'] ?? null) !== ($b['baselines'][$br]['sha'] ?? null)) $shaMoved = true;
|
||||
if (($a['baselines'][$br]['tree'] ?? null) !== ($b['baselines'][$br]['tree'] ?? null)) $treeMoved = true;
|
||||
$add += count(array_diff(array_keys($rb), array_keys($ra)));
|
||||
$rem += count(array_diff(array_keys($ra), array_keys($rb)));
|
||||
foreach ($ra as $id => $x) {
|
||||
if (! isset($rb[$id])) continue;
|
||||
foreach (['status','time','assertions','message'] as $f) {
|
||||
if (($x[$f] ?? null) !== ($rb[$id][$f] ?? null)) { $wr++; break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
$n = 0; foreach ($brB as $br) $n = max($n, count($b['baselines'][$br]['results'] ?? []));
|
||||
printf('n=%d w=%-2d +%d -%d %s%s%s', $n, $wr, $add, $rem,
|
||||
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
|
||||
$shaMoved ? ' sha:CHANGED' : '', $treeMoved ? ' tree:chg' : '');
|
||||
```
|
||||
</details>
|
||||
|
||||
`cmp.php` (verbose per-entry version of the same) and `summarise.php` / `edgediff.php` are in
|
||||
`PLAN_PHASE_TWO.md` §"Graph summariser" — reuse them for drill-downs. Reading the verdict:
|
||||
|
||||
- `w=` — entries actually **written**. Under sentinel patching this is the only reliable way to tell
|
||||
"wrote identical values" from "wrote nothing". A full replay must give `w=0`.
|
||||
- `struct:ok` + `+0 -0` — no prune, no edges/files/fingerprint movement. Required by RESULTS-ONLY.
|
||||
- `STRUCT:branchkeys(...)` — a new baseline key appeared. For section L this is the headline signal.
|
||||
|
||||
### 4.4 Section L — new rows for this fix
|
||||
|
||||
Tiers, unchanged from phase two: **COMPLETE** may change everything · **RESULTS-ONLY (RO)** may
|
||||
change only `baselines[<branch>].results` for tests that ran, and must never remove an entry, add a
|
||||
result for a test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Every L row starts from a **cold graph recorded on the named default branch only** — verify
|
||||
`keys=[<default>]` before branching. Restore branch state afterwards.
|
||||
|
||||
| # | Case | Target outcome |
|
||||
|---|---|---|
|
||||
| L1 | default `master`, record, `git switch -c feature-x`, `pest --tia`, zero changes | **all 25 replayed** (`w=0`), not `25 uncached`. The headline fix. |
|
||||
| L2 | as L1 but default `main` | still all replayed — regression guard, this already worked |
|
||||
| L3 | default `trunk`, then `develop` | replayed for both; the fix must not special-case two names |
|
||||
| L4 | L1, then a *second* new branch `feature-y` | replayed too — the toll must not return per branch |
|
||||
| L5 | L1 then edit `app/Services/Calculator.php` on `feature-x` | narrows to the 2 affected files (`CalculatorTest` + `AnnotationsTest`, which `covers` it); the other 17 replay |
|
||||
| L6 | L1 with `--tia --filtered` | filtered mode reads the fallback too (`testFilesToRerun`, `hasUnlocatedTestsToRerun`) → `No affected tests found`, zero delta |
|
||||
| L7 | L1 with `--tia --parallel` | fallback works in workers as well as the parent |
|
||||
| L8 | L1, then `pest --tia` twice on `feature-x` | idempotent; second run also `w=0` |
|
||||
| L9 | detached HEAD on a `master`-only graph | replays, and **no `main` key minted** — `keys` stays `[master]`. Bug (b). |
|
||||
| L10 | `pest()->tia()->defaultBranch('master')` in `tests/Pest.php`, repo default renamed away | config wins over autodetect |
|
||||
| L11 | config set to a nonexistent branch (`defaultBranch('nope')`) | degrades to a full run; no crash, no phantom key |
|
||||
| L12 | no remote at all (`git remote remove origin` if present) | still resolves (via `init.defaultBranch`) or degrades safely — must not throw |
|
||||
| L13 | branch name with a slash (`feature/x/y`) | replayed; no key-splitting bugs |
|
||||
| L14 | L1, then confirm writes | `feature-x` gets its **own** key; the `master` key is **not** written to (fallback is read-only) |
|
||||
| L15 | git worktree on a new branch (the issue's scenario) | replays from the default-branch baseline |
|
||||
| L16 | non-git dir, `pest --tia` | still `MissingDependency: The feature "Tia mode" requires "git".` — the soft resolver must not swallow it |
|
||||
| L17 | non-git dir, plain `pest` | runs normally; no baseline dir created |
|
||||
| L18 | count `git` subprocesses during one `--tia` run | default-branch resolution is cached, not one call per test. Probe by shimming `git` on `PATH` to a logging wrapper. |
|
||||
|
||||
L10–L11 need a `tests/Pest.php` edit — that file is tracked and **outside** the `tests app` reset
|
||||
scope in practice (it lives in `tests/`, so `git checkout -- tests` does restore it; verify with
|
||||
`git status` after).
|
||||
|
||||
For L16/L17, build a throwaway non-git project — and note the trap that burned phase two: a
|
||||
**symlinked** `vendor` makes Pest resolve the project root back to the playground (identical baseline
|
||||
hash), silently invalidating the test. Use a hardlinked copy:
|
||||
|
||||
```bash
|
||||
NG="$SP/nogit"; rm -rf "$NG"; mkdir -p "$NG"
|
||||
cp -R composer.json composer.lock phpunit.xml artisan tests app bootstrap config routes resources storage "$NG/"
|
||||
[ -f .env ] && cp .env "$NG/"
|
||||
cp -Rl vendor "$NG/vendor" || cp -R vendor "$NG/vendor"
|
||||
```
|
||||
|
||||
Confirm the baseline path differs (`nogit-<hash>`, not `laravel-4a455a95622ac0ec`), and delete both
|
||||
the temp project and its `~/.pest/tia/nogit-*` dir afterwards.
|
||||
|
||||
### 4.5 Re-run the phase-two matrix (A–K, 156 rows)
|
||||
|
||||
Re-run every row of `PLAN_PHASE_TWO.md` Part 2 against the fixed build. No row's status is trusted
|
||||
until re-measured — the fix touches `Graph`'s read path, which nearly every row exercises. Sections
|
||||
**A, B, H, I** are the load-bearing ones here (H is branch-key resolution; I is filtered mode; both
|
||||
consume the changed methods directly). **H6 and H7 must be re-derived from a cold graph** (Part 1.4).
|
||||
|
||||
Most rows batch cheaply — phase two ran C1–C20 in one call at roughly one line of output each. Use
|
||||
`oneline.php` for the sweep and `cmp.php` only to drill into anomalies.
|
||||
|
||||
### 4.6 Known pre-existing failures — do not report as regressions
|
||||
|
||||
| item | status |
|
||||
|---|---|
|
||||
| **G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests. `mergeWorkerReplayPartials()` takes `$result['time']` verbatim at `Tia.php:1451`, never routing through `resultTime()` as the sequential sites (1681, 1762) do. Assertions survive because workers replay those themselves. | Pre-existing (pre-fix had no preservation at all), **out of scope for phase three**. Report it as still-present; do not fix it unless Nuno asks. |
|
||||
| **C19** — `--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
|
||||
| **J11** — `--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
|
||||
| **J10** — `--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Reporting
|
||||
|
||||
Per row: **tier respected (yes/no)**, the **graph delta** under sentinel patching, and for any failure
|
||||
the **pre-fix contrast** so a regression is told apart from a pre-existing defect. For a pre-fix
|
||||
contrast you need `db70017c` (or `bfd5b756` for "before phase three") files swapped into the
|
||||
playground's `vendor/` — that is a sync, so **ask Nuno first** and always restore afterwards.
|
||||
|
||||
State which pest commit produced the numbers. Close with:
|
||||
|
||||
1. Whether L1–L18 all pass (the fix works).
|
||||
2. Whether A–K regressed anywhere relative to phase two's 154/156.
|
||||
3. The D1–D4 decisions as implemented.
|
||||
4. Anything still open — including G4, which will still be failing.
|
||||
|
||||
Leave the playground on `master` with only the four user-modified files dirty, no stray branches, and
|
||||
no leftover `~/.pest/tia/*` dirs beyond `laravel-4a455a95622ac0ec`.
|
||||
@@ -1,466 +0,0 @@
|
||||
# TIA write-tier conformance — phase two
|
||||
|
||||
## Your task
|
||||
|
||||
**Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app** at
|
||||
`/Users/nunomaduro/Work/projects/playground/laravel`. Every row, including the ones already marked
|
||||
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
|
||||
re-measured against the current code. The `Phase 1` column is prior evidence and a hint at what to
|
||||
watch, never a reason to skip a row.
|
||||
|
||||
These are **not** the `pestphp/pest` repo's own tests (`composer test`) — do not run those. Each row is
|
||||
a `pest` invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
|
||||
|
||||
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any
|
||||
failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do **not**
|
||||
stop at reading this file or summarising it; the deliverable is executed results.
|
||||
|
||||
Work in this order: **Part 1 (environment traps) → Part 1b (build the fixtures — the playground has
|
||||
none of them) → Part 2 (the matrix) → Part 3 (priorities)**. The matrix is too large for one context;
|
||||
take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are
|
||||
the load-bearing ones — do those first if you run short.
|
||||
|
||||
Phase one implemented `PLAN.md` Part 1 items 1–4 plus three §5 items. This file turns the matrix into
|
||||
a conformance check rather than a bug list.
|
||||
|
||||
Code under test: `pestphp/pest` at `/Users/nunomaduro/Work/projects/pestphp/pest`, branch
|
||||
`fix/tia-filtered`, commit **`bfd5b756`** or later. Verify with
|
||||
`grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php` → at least 3 hits. Pre-fix
|
||||
baseline for every contrast is **`db70017c`**.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — What phase one changed
|
||||
|
||||
| # | Change | Files |
|
||||
|---|---|---|
|
||||
| 1 | `hasUnlocatedTestsToRerun()` stats the file, so a deleted test file is "unlocated" | `src/Plugins/Tia/Graph.php` |
|
||||
| 2 | `enterReplayMode()` uses `activateLinkTracking()` under piggyback coverage | `src/Plugins/Tia.php` |
|
||||
| 3 | `enterReplayMode()` stamps `TIA_PIGGYBACK_COVERAGE` for workers | `src/Plugins/Tia.php` |
|
||||
| 4 | `replaceEdges(…, keepExisting:)` — piggyback edges seed empty sets, never overwrite populated ones | `Graph.php`, `Tia.php` |
|
||||
| 5 | `renderFreshGraph()` stops claiming "fresh graph" when the graph is kept; reason reworded to `recording a coverage baseline` | `Tia.php` |
|
||||
| 6 | `COVERAGE_REPORT_FLAGS` + `coverageReportActive()` union over `originalArguments`; new `pestCoverageActive()` keeps the coverage-cache marker/hijack on Pest's own `--coverage` | `Tia.php` |
|
||||
| 7 | `Tia::recordsEdgesInWorkers()` + `WrapperRunner::handleTia()` inject `-d pcov.directory=<root>` into worker argv | `Tia.php`, `src/Plugins/Parallel/Paratest/WrapperRunner.php` |
|
||||
| 8 | Sequential record runs announce structural drift via `renderFreshGraph()` | `Tia.php` |
|
||||
| 9 | `Graph::getTime()` + `cachedTimeByTestId` + `resultTime()` preserve replayed durations; edge-less write guard is now `$recordsEdges = $complete && ($markKnownTestFiles \|\| $this->recordingActive)` | `Graph.php`, `Tia.php` |
|
||||
|
||||
Deliberately **not** done: `PLAN.md` §5 SIGINT propagation, §5 warning/deprecation `status=0`
|
||||
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
|
||||
|
||||
### Target-outcome changes this forces
|
||||
|
||||
Two rows in the original matrix asserted the **old**, buggy behaviour. Their targets are updated
|
||||
below — do not report them as regressions:
|
||||
|
||||
- **B10** was "`time` differs; statuses stable". Change 9 means replayed entries now **keep** their
|
||||
recorded `time`. New target: `time` differs only for tests that actually executed.
|
||||
- **B12** gains an edge-preservation assertion it never had (see K1).
|
||||
|
||||
### One known-failing repo test
|
||||
|
||||
`tests/Unit/Plugins/Tia/Graph.php:69-76` asserts `hasUnlocatedTestsToRerun('main')` is `false` for
|
||||
`tests/Feature/FooTest.php` under `new Graph(sys_get_temp_dir())` — a path that does not exist. That
|
||||
assertion encodes the I9 bug and **will fail** under change 1. It needs re-pointing at a root/file
|
||||
that exists (e.g. `dirname(__DIR__, 4)` + `'tests/Unit/Plugins/Tia/Graph.php'`). Left untouched by
|
||||
request; it is a repo-test matter, not a playground one.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Environment, and the traps in it
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel`
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
|
||||
```
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON
|
||||
when it detects an agent.
|
||||
2. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock` —
|
||||
that a blanket reset would destroy. `PLAN.md`'s "reset ritual" is unsafe as written. Scope resets
|
||||
to what you touched: `git checkout -- tests app` / `git clean -fd tests app`, and for
|
||||
`composer.lock` (needed for the drift rows) **copy it aside and copy it back**, verifying with
|
||||
`shasum`.
|
||||
3. **`vendor/pestphp/pest` is a dist copy, not a symlink.** Composer installed
|
||||
`dev-fix/tia-filtered as 5.2.0`, so edits in the pest repo do **not** reach the playground.
|
||||
|
||||
**Tell Nuno whenever a sync is needed to move forward — do not sync silently.** Say what is stale
|
||||
and what the sync would be, then wait. This includes temporarily swapping in `db70017c` files for a
|
||||
before/after contrast. When you report any playground result, state which commit produced it.
|
||||
The sync itself, once he agrees:
|
||||
```bash
|
||||
PEST=/Users/nunomaduro/Work/projects/pestphp/pest
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \
|
||||
src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; done
|
||||
```
|
||||
Verify with `grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"` → `1`. **Always restore the
|
||||
current version before continuing** after a pre-fix contrast.
|
||||
4. **Coverage driver:** pcov only, no xdebug. `ini_get('pcov.directory')` is `''` by default — that
|
||||
emptiness is the entire mechanism behind G12.
|
||||
5. **Suite shape as found:** 7 tests in 5 files — `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,
|
||||
TrioTest}.php`, `tests/Feature/ExampleTest.php`. A healthy sequential graph is **`files=22`,
|
||||
5 edge keys, self-edge on every test** (`PLAN.md`'s `files=25` is stale). Pre-fix parallel gives
|
||||
`files=4` with zero self-edges. **These numbers shift the moment you add the Part 1b fixtures** —
|
||||
re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants
|
||||
that do *not* shift: sequential and parallel must agree, and every test must have a self-edge.
|
||||
6. **Sentinel patching is the only reliable discriminator** between "wrote identical values" and
|
||||
"wrote nothing": rewrite every cached result to `time=9.999 assertions=42`, snapshot, run the
|
||||
case, diff. A canary test absent from `edges` is unreliable under `--filter` because it never
|
||||
matches the filter and so never runs.
|
||||
7. Seeding a cached failure needs `--fresh` (or an env-driven flaky fixture): `--tia` on a clean
|
||||
green tree replays rather than executes, so it can never cache a failure. Working recipe — break
|
||||
an assertion, `pest --tia --fresh`, then restore the source.
|
||||
8. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
9. The shell is zsh — build commands with arrays or `eval`; unquoted `$args` does not word-split.
|
||||
|
||||
### Graph summariser
|
||||
|
||||
Write this to a scratch path and use it for every diff.
|
||||
|
||||
```php
|
||||
<?php // summarise.php <graph.json> [label]
|
||||
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
|
||||
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
|
||||
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
|
||||
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
|
||||
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
|
||||
ksort($edges);
|
||||
foreach ($edges as $test => $ids) {
|
||||
$self = 'no';
|
||||
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
|
||||
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
|
||||
}
|
||||
foreach ($g['baselines'] ?? [] as $branch => $b) {
|
||||
$r = $b['results'] ?? [];
|
||||
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
|
||||
foreach ($r as $id => $x) {
|
||||
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
|
||||
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Normalised edge-set equality (for G12 / I5 / K1):
|
||||
|
||||
```php
|
||||
<?php // edgediff.php <a.json> <b.json>
|
||||
function edges(string $p): array {
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
|
||||
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$a = edges($argv[1]); $b = edges($argv[2]);
|
||||
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
|
||||
foreach ($a as $t => $s) {
|
||||
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
|
||||
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
|
||||
}
|
||||
```
|
||||
|
||||
### Per-case loop
|
||||
|
||||
```bash
|
||||
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 1b — Fixtures you must build first
|
||||
|
||||
**The playground has none of the fixtures the matrix depends on.** Verified inventory: the only test
|
||||
files are `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php` and
|
||||
`tests/Feature/ExampleTest.php` — 7 plain tests, zero occurrences of `->group()`, `->only()`,
|
||||
`->skip()`, `->todo()`, `->note()`, `->flaky()`, `->covers()`, `->uses()`, `->issue()`, `->pr()`,
|
||||
`->ticket()`, `->assignee()`, datasets, or any `FLAKY_OK`-style env hook. `phpunit.xml` defines only
|
||||
the `Unit` and `Feature` testsuites.
|
||||
|
||||
So roughly 30 rows below cannot run as written until you create the fixtures. Build them **all up
|
||||
front, in one commit**, then re-derive the baseline graph shape once and use those numbers for the
|
||||
whole sweep — every fixture you add changes `files`, the `edges` key count and `n`, so adding them
|
||||
piecemeal invalidates earlier rows.
|
||||
|
||||
| Fixture to create | Rows that need it |
|
||||
|---|---|
|
||||
| a test with `->group('smoke')` | C7, C8, C9, C10 |
|
||||
| an env-driven flaky test (passes iff `FLAKY_OK=1`) | E1, E2, E3, E4 |
|
||||
| a `->skip()`ed test | E9, D8 |
|
||||
| a `->todo()` test | E10, C23, C39 |
|
||||
| a risky test (no assertions; pair with `--disallow-test-output`) | E11, D7 |
|
||||
| a test that triggers a PHPUnit warning | D6 |
|
||||
| a test calling `markTestIncomplete()` | D9 |
|
||||
| a test that triggers a deprecation | D10 |
|
||||
| a dataset test with ≥3 rows | E13, E14 |
|
||||
| `->only()` — added and removed per case, not left in | C31, C32, G7 |
|
||||
| `->covers(App\Services\Calculator::class)` | C18 |
|
||||
| `->uses(...)` / `UsesClass` annotation | C19 |
|
||||
| `->note(...)` | C24 |
|
||||
| `->flaky()` **annotation** (distinct from the env-driven flaky test above) | C25 |
|
||||
| `->issue(123)`, `->pr(1)`, `->ticket('X')`, `->assignee('X')` | C26, C27, C28, C37, C38 |
|
||||
|
||||
This is `PLAN.md` §6's "test-harness gaps to close before re-running", now itemised: without these,
|
||||
the listed rows' graph-invariant assertions match zero tests and **prove nothing** — they pass
|
||||
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous *only because* its fixture is
|
||||
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
|
||||
|
||||
Rows needing an *action* rather than a fixture — breaking `trio one` for the D rows, an uncommitted
|
||||
test edit for `--dirty`, branch renames and a non-git dir for H5–H10, `--mutate` against
|
||||
`app/Services` for the F rows — are fine as written; `pestphp/pest-plugin-mutate` is installed.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Phase-1 column: **VERIFIED** = re-run against the fixed code in the phase-one session, pre-fix
|
||||
contrast captured · **PASS (sweep)** = passed in the original `db70017` sweep and *not* re-checked
|
||||
since the changes — these are the bulk of phase two's work · **SKIP** = not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS (sweep) |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | VERIFIED |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | VERIFIED |
|
||||
| A4 | `git status` after reset ritual | clean *except the four user-modified files* (see trap 2) | VERIFIED |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | VERIFIED |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS (sweep) |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS (sweep) |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | VERIFIED |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | VERIFIED |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS (sweep) — recheck under change 9 |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS (sweep) — recheck under change 9 |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | VERIFIED |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS (sweep) |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS (sweep) |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS (sweep) |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS (sweep) |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS (sweep) |
|
||||
| B10 | `pest --tia` twice | **UPDATED TARGET:** `time` differs only for tests that executed; replayed entries keep their recorded `time`; statuses stable | VERIFIED |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key **and** new result appear in the **same** run | VERIFIED (regression guard for change 9) |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written; coverage report prints | VERIFIED |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS (sweep) |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS (sweep) |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS (sweep) |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS (sweep) |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS (sweep) |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS (sweep) |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS (sweep) |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS (sweep) |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS (sweep) |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS (sweep) |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS (sweep) |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS (sweep) |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS (sweep) |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS (sweep) |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS (sweep) |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS (sweep) |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS (sweep) |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS (sweep) |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS (sweep) |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS (sweep) |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS (sweep) |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS (sweep) |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS (sweep) |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS (sweep) |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS (sweep) |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS (sweep) |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS (sweep) |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS (sweep) |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS (sweep) |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS (sweep) |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS (sweep) |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS (sweep) |
|
||||
| D11 | `pest --tia --bail` | RO | PASS (sweep) |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS (sweep) |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS (sweep) |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS (sweep) |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS (sweep) |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS (sweep) |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran (it executed, so change 9 does not apply) | PASS (sweep) |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS (sweep) |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS (sweep) |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS (sweep) |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS (sweep) |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS (sweep) |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS (sweep) |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS (sweep) |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS (sweep) |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | VERIFIED |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | VERIFIED |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS (sweep) |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS (sweep) |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS (sweep) |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS (sweep) |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS (sweep) |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written; all 7 results present | VERIFIED |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS (sweep) |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS (sweep) |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS (sweep) |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | VERIFIED (see I5) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | VERIFIED |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS (sweep) |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS (sweep) |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS (sweep) |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS (sweep) |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | VERIFIED |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** — `files=22`, self-edge on all 5 tests, both | **VERIFIED (fixed)** — pre-fix: `files=4`, 0 self-edges |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | VERIFIED |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS (sweep) |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS (sweep) |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS (sweep) |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS (sweep) |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS (sweep) |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS (sweep) |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | VERIFIED |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS (sweep) |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **VERIFIED (fixed)** — 1 affected test, edges identical |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS (sweep) |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS (sweep) |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report → full suite runs | **VERIFIED (fixed)** — pre-fix: `No affected tests found` |
|
||||
| I9 | cached failure whose test file was deleted | WARN `Some cached tests due a re-run could not be located on disk` + `Running the full suite with replay instead of a filtered run` | **VERIFIED (fixed)** — pre-fix: `No tests found`, exit 0 green |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS (sweep) |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild, **and the reason is printed sequentially**: `fresh graph (composer.lock changed)` | **VERIFIED (fixed)** — pre-fix: bare `Running in TIA mode.` |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS (sweep) |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS (sweep) |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS (sweep) |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS (sweep) |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS (sweep) |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS (sweep) |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS (sweep) |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS (sweep) |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS (sweep) |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS (sweep) |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE (silent no-op without `--coverage`) | PASS (sweep) |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS (sweep) |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS (sweep) |
|
||||
|
||||
### K — New rows for the phase-one fixes
|
||||
|
||||
These assert behaviour no original row covered. All were verified in phase one; re-run them as
|
||||
regression guards.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| K1 | healthy `pest --tia` graph, then `pest --tia --coverage` | `edges` **byte-identical** — piggyback data may seed empty sets, never narrow populated ones; `files=22` stays `22` | **VERIFIED (fixed)** — pre-fix: `ExampleTest` 16→2 edges, self-edges lost |
|
||||
| K2 | same run's headline | `Experimental TIA mode enabled / recording a coverage baseline.` — no false `fresh graph` | **VERIFIED (fixed)** |
|
||||
| K3 | `pest --tia` ×3 on a clean tree | replayed entries keep their recorded `time` across all three | **VERIFIED (fixed)** — pre-fix: `0.053 → 0.001 → 0.001 → 0.001` |
|
||||
| K4 | add a test file, one `pest --tia` (graph exists, fingerprint matches → replay+refresh) | result **and** edges appear in that same run; `n` grows by 1 | **VERIFIED** — regression guard for change 9 |
|
||||
| K5 | add a test file, plain `pest --no-tia` | no result written for it; `n` unchanged; no edges | **VERIFIED (fixed)** — pre-fix: edge-less result inflated `n` |
|
||||
| K6 | `pest --tia --parallel` worker argv | carries `-d pcov.directory=<projectRoot>`; `PEST_TIA` unset; not re-exec'd | **VERIFIED** — probe `bin/worker.php` |
|
||||
| K7 | second `pest --tia --coverage` (cache primed → replay) | recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — **new work** |
|
||||
| K8 | `pest --tia --parallel --coverage` | workers read `TIA_PIGGYBACK_COVERAGE`; no widened pcov scope; report intact | Not yet measured — **new work** |
|
||||
| K9 | `pest --tia --filtered --coverage-html=<dir>` / `--coverage-clover=<file>` | filtered mode off, same as I8, for every flag in `COVERAGE_REPORT_FLAGS` | Not yet measured — **new work** |
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Priorities for phase two
|
||||
|
||||
0. **Part 1b fixtures** — nothing in C, D6–D10, or E9–E14 means anything until they exist.
|
||||
1. **K7, K8, K9** — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
|
||||
reasoned about but not observed; K9 covers the seven `COVERAGE_REPORT_FLAGS` beyond
|
||||
`--coverage-text`.
|
||||
2. **B2, B3, F4** and all of **D** and **E** — change 9 touched the shared result-write path, and
|
||||
these are the rows that exercise it hardest. `$recordsEdges` is the thing to falsify: it must be
|
||||
false for every partial and every non-recording run.
|
||||
3. **F3–F8** — change 7 injects a `-d` into worker argv; `--mutate --parallel` is the one place that
|
||||
both spawns workers and must write nothing.
|
||||
4. **H1–H10** — untouched by these changes; cheapest bulk confirmation.
|
||||
5. Raw PHPUnit coverage flags print **no report at all** in Pest, with or without TIA
|
||||
(`--no-tia --coverage-text` is equally silent). Pre-existing, unrelated to change 6 — do not
|
||||
chase it as a regression, but it means I8/K9 can only assert the selection half.
|
||||
|
||||
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
|
||||
failure the pre-fix contrast (write `git show db70017c:<file>` into vendor, re-run, restore) so a
|
||||
regression is told apart from a pre-existing defect.
|
||||
@@ -94,6 +94,7 @@
|
||||
"test:inline": "php bin/pest --configuration=phpunit.inline.xml",
|
||||
"test:parallel": "php bin/pest --exclude-group=integration --parallel --processes=3",
|
||||
"test:integration": "php bin/pest --group=integration -v",
|
||||
"test:tia": "php bin/pest --group=tia -v",
|
||||
"update:snapshots": "REBUILD_SNAPSHOTS=true php bin/pest --update-snapshots",
|
||||
"test": [
|
||||
"@test:lint",
|
||||
|
||||
@@ -28,8 +28,6 @@ return RectorConfig::configure()
|
||||
->withSkip([
|
||||
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
|
||||
__DIR__.'/tests/Fixtures/Arch',
|
||||
// Fixture suites are pinned by the TeamCity / JUnit snapshots, down to
|
||||
// the line numbers — rewriting their source would break them.
|
||||
__DIR__.'/tests/Fixtures/Suites',
|
||||
ReturnNeverTypeRector::class,
|
||||
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
|
||||
|
||||
@@ -35,6 +35,12 @@ final readonly class BootSubscribers implements Bootstrapper
|
||||
Subscribers\EnsureTiaResultIsRecordedOnSkipped::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnIncomplete::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnRisky::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnNoticeTriggered::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnPhpNoticeTriggered::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnDeprecationTriggered::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnPhpDeprecationTriggered::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnWarningTriggered::class,
|
||||
Subscribers\EnsureTiaResultIsRecordedOnPhpWarningTriggered::class,
|
||||
Subscribers\EnsureTiaAssertionsAreRecordedOnFinished::class,
|
||||
];
|
||||
|
||||
|
||||
@@ -286,9 +286,6 @@ trait Testable
|
||||
if ($replay !== ReplayType::None) {
|
||||
assert($status !== null);
|
||||
|
||||
// Marks the replay before the branches below throw, so `tearDown`
|
||||
// short-circuits for every replayed result — the throwing branches
|
||||
// never reach `parent::setUp`, so no user hook may run after them.
|
||||
$this->__replay = $replay;
|
||||
|
||||
match ($replay) {
|
||||
|
||||
@@ -19,6 +19,6 @@ final class MissingDependency extends InvalidArgumentException implements Except
|
||||
*/
|
||||
public function __construct(string $feature, string $dependency)
|
||||
{
|
||||
parent::__construct(sprintf('The feature "%s" requires "%s".', $feature, $dependency));
|
||||
parent::__construct(sprintf('The feature [%s] requires [%s].', $feature, $dependency));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Exceptions;
|
||||
|
||||
use NunoMaduro\Collision\Contracts\RenderlessEditor;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessTrace;
|
||||
use Pest\Contracts\Panicable;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Console\Exception\ExceptionInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class TiaRequiresCommit extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
|
||||
{
|
||||
public function __construct()
|
||||
{
|
||||
parent::__construct(
|
||||
'Tia mode requires a repository with at least one commit, so the baseline it records can be anchored to a revision.',
|
||||
);
|
||||
}
|
||||
|
||||
public function render(OutputInterface $output): void
|
||||
{
|
||||
$output->writeln([
|
||||
'',
|
||||
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires at least one commit.',
|
||||
'',
|
||||
' A baseline is anchored to the revision it was recorded at, and this repository',
|
||||
' has none yet, so there is nothing to record against and nothing to compare a',
|
||||
' later run to.',
|
||||
'',
|
||||
' Commit once, then run again:',
|
||||
'',
|
||||
' <fg=yellow>git add . && git commit -m "Initial commit"</>',
|
||||
'',
|
||||
' Runs without <fg=yellow>--tia</> are unaffected.',
|
||||
'',
|
||||
]);
|
||||
}
|
||||
|
||||
public function exitCode(): int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
@@ -205,15 +205,6 @@ final class WrapperRunner implements RunnerInterface
|
||||
}
|
||||
|
||||
/**
|
||||
* Widens pcov's instrumentation scope to the whole project for workers that
|
||||
* record TIA edges.
|
||||
*
|
||||
* pcov's default scope is a single source directory it auto-detects, so
|
||||
* `config/`, `routes/`, `bootstrap/` and every test's own file never reach
|
||||
* the recorder — a worker-recorded graph selects a fraction of what a
|
||||
* sequential one does. `pcov.directory` is only settable at startup, hence
|
||||
* the command line rather than an `ini_set()` inside the worker.
|
||||
*
|
||||
* @param array<int, non-empty-string> $parameters
|
||||
* @return array<int, non-empty-string>
|
||||
*/
|
||||
|
||||
+215
-340
@@ -12,6 +12,7 @@ use Pest\Contracts\Plugins\Terminable;
|
||||
use Pest\Exceptions\InvalidOption;
|
||||
use Pest\Exceptions\MissingDependency;
|
||||
use Pest\Exceptions\NoAffectedTestsFound;
|
||||
use Pest\Exceptions\TiaRequiresCommit;
|
||||
use Pest\Exceptions\TiaRequiresDefaultBranch;
|
||||
use Pest\Exceptions\TiaRequiresRemote;
|
||||
use Pest\Exceptions\TiaRequiresRepositoryRoot;
|
||||
@@ -32,13 +33,13 @@ use Pest\Plugins\Tia\Storage;
|
||||
use Pest\Plugins\Tia\TableExtractor;
|
||||
use Pest\Plugins\Tia\WatchPatterns;
|
||||
use Pest\Support\Container;
|
||||
use Pest\Support\Git;
|
||||
use Pest\Support\View;
|
||||
use Pest\TestCaseFilters\TiaTestCaseFilter;
|
||||
use Pest\TestSuite;
|
||||
use PHPUnit\Framework\TestStatus\TestStatus;
|
||||
use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
@@ -63,11 +64,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
private const string BASELINE_PATH_OPTION = '--baseline';
|
||||
|
||||
/**
|
||||
* Set by the mutation plugin on the subprocess running a single mutant,
|
||||
* and nowhere else. Its own `--mutate` flag is popped before the argv is
|
||||
* handed to that subprocess, so the flag cannot be matched instead.
|
||||
*/
|
||||
private const string ENV_MUTATION_TESTING = 'PEST_MUTATION_TESTING';
|
||||
|
||||
private const string ENV_TIA = 'PEST_TIA';
|
||||
@@ -100,21 +96,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
private const string FILTERED_GLOBAL = 'TIA_FILTERED';
|
||||
|
||||
private const string WORKER_RESULTS_GLOBAL = 'TIA_WORKER_RESULTS';
|
||||
|
||||
private const string PIGGYBACK_COVERAGE_GLOBAL = 'TIA_PIGGYBACK_COVERAGE';
|
||||
|
||||
/**
|
||||
* The parent's resolved fallback branch, handed to the workers.
|
||||
*
|
||||
* A worker cannot resolve it for itself: the restarters run before
|
||||
* `tests/Pest.php` is loaded, so a `defaultBranch()` declared there is
|
||||
* invisible to it — and autodetecting again would spend a git call per
|
||||
* worker to reach the answer the parent already has.
|
||||
*/
|
||||
private const string FALLBACK_BRANCH_GLOBAL = 'TIA_FALLBACK_BRANCH';
|
||||
|
||||
/**
|
||||
* The branch assumed when a repository cannot name its own default.
|
||||
*/
|
||||
private const string DEFAULT_BRANCH = 'main';
|
||||
|
||||
/**
|
||||
@@ -137,49 +124,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
'--compact', '--ci-build-id', '--min',
|
||||
];
|
||||
|
||||
/**
|
||||
* PHPUnit flags that make this run produce a coverage report.
|
||||
*
|
||||
* Pest's own `--coverage` is tracked by the Coverage plugin, but a raw
|
||||
* PHPUnit report flag never reaches it. A run that reports coverage must
|
||||
* not be narrowed to the affected tests — the report would then describe a
|
||||
* subset of the suite — and must let PHPUnit own the coverage driver rather
|
||||
* than have the TIA recorder clear it mid-collection.
|
||||
*
|
||||
* Flags that only shape collection or an existing report — `--coverage-filter`,
|
||||
* `--path-coverage`, `--warm-coverage-cache`, `--only-summary-for-coverage-text`,
|
||||
* `--show-uncovered-for-coverage-text`, `--disable-coverage-ignore` — produce no
|
||||
* report on their own, so they are deliberately absent.
|
||||
*
|
||||
* @var list<string>
|
||||
*/
|
||||
/** @var list<string> */
|
||||
private const array COVERAGE_REPORT_FLAGS = [
|
||||
'--coverage-clover', '--coverage-cobertura', '--coverage-crap4j',
|
||||
'--coverage-html', '--coverage-openclover', '--coverage-php',
|
||||
'--coverage-text', '--coverage-xml',
|
||||
];
|
||||
|
||||
/**
|
||||
* Flags that narrow this run to a subset of the suite.
|
||||
*
|
||||
* Only user-supplied, per-run narrowing belongs here. A filter that is
|
||||
* always in force — `<groups>` in phpunit.xml, or a plugin registering a
|
||||
* test case filter from `boot()` — applies equally to the runs that build
|
||||
* the baseline, so it does not make this run narrower than the baseline
|
||||
* and must not disable baseline writes.
|
||||
*
|
||||
* `--shard` is rewritten to `--filter` before this plugin sees the
|
||||
* arguments, so it is covered here too. The `bin/pest`-only flags are
|
||||
* stripped from the handled arguments, so they are matched against the
|
||||
* original argv instead.
|
||||
*
|
||||
* Flags that cut a run short instead of narrowing it — `--bail`, `--retry`,
|
||||
* `--stop-on-*` — do not belong here either. They only narrow the run when
|
||||
* something actually fails, and that is not known until it is over, so they
|
||||
* are handled by stoppedEarly() from addOutput().
|
||||
*
|
||||
* @var list<string>
|
||||
*/
|
||||
/** @var list<string> */
|
||||
private const array PARTIAL_SELECTION_FLAGS = [
|
||||
'--filter', '--exclude-filter', '--group', '--exclude-group',
|
||||
'--covers', '--uses', '--testsuite', '--exclude-testsuite', '--test-suffix',
|
||||
@@ -187,15 +139,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
|
||||
];
|
||||
|
||||
/**
|
||||
* Options that cannot be combined with Tia mode.
|
||||
*
|
||||
* `--covers` and `--uses` select on coverage metadata Tia does not model,
|
||||
* so they resolve to no tests at all rather than to the ones the user meant.
|
||||
* `--random-order-seed` exits non-zero on its own, with or without Tia.
|
||||
* Either way the run cannot honour both things it was asked for, so it says
|
||||
* so instead of silently dropping Tia and running something else.
|
||||
*/
|
||||
private const array UNSUPPORTED_OPTIONS = [
|
||||
'--covers', '--uses', '--random-order-seed',
|
||||
];
|
||||
@@ -213,47 +156,18 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
/** @var array<string, int> */
|
||||
private array $cachedAssertionsByTestId = [];
|
||||
|
||||
/**
|
||||
* Recorded durations of the tests this run replayed rather than executed.
|
||||
*
|
||||
* A replayed test never runs, so the duration PHPUnit reports for it is the
|
||||
* cost of replaying it — near zero. Writing that back would decay every
|
||||
* cached timing toward zero one run at a time.
|
||||
*
|
||||
* @var array<string, float>
|
||||
*/
|
||||
/** @var array<string, array{status: int, message: string}> */
|
||||
private array $cachedStatusByTestId = [];
|
||||
|
||||
/** @var array<string, float> */
|
||||
private array $cachedTimeByTestId = [];
|
||||
|
||||
private ?Graph $replayGraph = null;
|
||||
|
||||
/**
|
||||
* The baseline this run reads from and writes to.
|
||||
*
|
||||
* The repository's default branch is only the fallback for a checkout whose
|
||||
* branch cannot be read — a detached HEAD. It is also the branch every
|
||||
* other baseline falls back to reading, so writing there by accident
|
||||
* corrupts the shared baseline. Resolved through resolveBranch() rather
|
||||
* than at every use site, because the git call it needs is not free.
|
||||
*/
|
||||
private string $branch = self::DEFAULT_BRANCH;
|
||||
|
||||
/**
|
||||
* The baseline branches with none of their own read from.
|
||||
*
|
||||
* Read-only, and the whole point of the exercise: without it the first run
|
||||
* on every new branch re-runs a suite whose results the default branch
|
||||
* already holds.
|
||||
*/
|
||||
private string $fallbackBranch = self::DEFAULT_BRANCH;
|
||||
|
||||
/**
|
||||
* Whether anything actually named the branch above.
|
||||
*
|
||||
* When nothing did, the value is a guess, and a guess is what the TIA path
|
||||
* refuses to run on: an unresolved fallback reads no baseline at all, which
|
||||
* looks exactly like a hit in the output. Runs that never asked for TIA
|
||||
* still have to write somewhere, so the guess stands for them.
|
||||
*/
|
||||
private bool $fallbackBranchResolved = false;
|
||||
|
||||
private bool $branchResolved = false;
|
||||
@@ -272,28 +186,20 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
private bool $baselineFetchAttemptedForDrift = false;
|
||||
|
||||
private bool $freshRebuild = false;
|
||||
|
||||
private bool $filteredMode = false;
|
||||
|
||||
/**
|
||||
* Bars this run from touching the graph at all, results included.
|
||||
*
|
||||
* Reserved for runs whose results describe something other than the code
|
||||
* in the working tree, which is nothing the baseline can ever use.
|
||||
*/
|
||||
private bool $writesSuppressed = false;
|
||||
|
||||
/**
|
||||
* Narrows this run's writes to the results of the tests it actually ran.
|
||||
*
|
||||
* A run that covered only part of the suite still learns something true
|
||||
* about the tests it did reach. What it cannot do is speak for the rest:
|
||||
* pruning results, advancing the recorded sha and replacing the edge map
|
||||
* all claim the whole suite reported, so they stay behind a complete run.
|
||||
*/
|
||||
private bool $resultsOnlyWrites = false;
|
||||
|
||||
private bool $flushesWorkerResults = false;
|
||||
|
||||
private bool $unreadableGraphReported = false;
|
||||
|
||||
private bool $detachedHead = false;
|
||||
|
||||
private bool $graphUnreachable = false;
|
||||
|
||||
/** @var array<int, string> */
|
||||
private array $originalArguments = [];
|
||||
|
||||
@@ -342,15 +248,52 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
$graph = Graph::decode($json, $projectRoot);
|
||||
|
||||
// Every read of a baseline goes through a graph loaded here, so this is
|
||||
// the one place the resolved fallback has to reach.
|
||||
$graph?->setFallbackBranch($this->fallbackBranch);
|
||||
if (! $graph instanceof Graph) {
|
||||
$this->discardUnreadableGraph();
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
$graph->setFallbackBranch($this->fallbackBranch);
|
||||
|
||||
return $graph;
|
||||
}
|
||||
|
||||
private function discardUnreadableGraph(): void
|
||||
{
|
||||
if (Parallel::isWorker()) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (! $this->deleteState(self::KEY_GRAPH)) {
|
||||
return;
|
||||
}
|
||||
|
||||
if ($this->unreadableGraphReported) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->unreadableGraphReported = true;
|
||||
|
||||
$this->output->writeln('');
|
||||
$this->renderBadge('WARN', 'The dependency graph could not be read — it will be rebuilt.');
|
||||
}
|
||||
|
||||
private function deleteState(string $key): bool
|
||||
{
|
||||
if ($this->detachedHead) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return $this->state->delete($key);
|
||||
}
|
||||
|
||||
private function saveGraph(Graph $graph): bool
|
||||
{
|
||||
if ($this->detachedHead) {
|
||||
return true;
|
||||
}
|
||||
|
||||
$json = $graph->encode();
|
||||
|
||||
if ($json === null) {
|
||||
@@ -388,19 +331,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return ! self::argumentPresent('--ci', $arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the workers of this run record their own coverage edges.
|
||||
*
|
||||
* Stamped by the parent before paratest spawns anything, because a worker
|
||||
* cannot tell on its own: its argv carries no `--tia`, and the restarters
|
||||
* run before `tests/Pest.php` is loaded, so {@see self::isEnabledForRun()}
|
||||
* sees an empty {@see WatchPatterns} too. Left unanswered, pcov keeps its
|
||||
* default scope — a single auto-detected source directory — and every edge
|
||||
* outside it, test self-edges included, is silently dropped.
|
||||
*
|
||||
* Piggyback runs are excluded: their edges come from PHPUnit's own coverage
|
||||
* session, so widening pcov there costs time and buys nothing.
|
||||
*/
|
||||
public static function recordsEdgesInWorkers(): bool
|
||||
{
|
||||
return (string) Parallel::getGlobal(self::RECORDING_GLOBAL) === '1'
|
||||
@@ -483,6 +413,10 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
}
|
||||
|
||||
$this->replayedCount++;
|
||||
$this->cachedStatusByTestId[$testId] = [
|
||||
'status' => $result->asInt(),
|
||||
'message' => $result->message(),
|
||||
];
|
||||
$assertions = $this->replayGraph->getAssertions($this->branch, $testId);
|
||||
$this->cachedAssertionsByTestId[$testId] = $assertions ?? 0;
|
||||
|
||||
@@ -541,11 +475,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
|
||||
$disabled = $disabled || $partial;
|
||||
|
||||
// A mutation subprocess runs the suite against source the mutation
|
||||
// plugin has deliberately broken. Its failures describe the mutant, not
|
||||
// the working tree, so unlike every other narrowed run there is nothing
|
||||
// in its results worth keeping. The parent `--mutate` run is untouched
|
||||
// by this: it runs the whole suite against real source.
|
||||
if (getenv(self::ENV_MUTATION_TESTING) !== false) {
|
||||
$this->writesSuppressed = true;
|
||||
}
|
||||
@@ -565,19 +494,11 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$arguments = $this->popArgument(self::BASELINED_OPTION, $arguments);
|
||||
|
||||
if ($disabled) {
|
||||
$this->requestWorkerResults();
|
||||
|
||||
if ($partial) {
|
||||
// TIA cannot choose what runs here — the user already did — but
|
||||
// the tests they picked still report honestly, so their results
|
||||
// are kept and everything that would speak for the excluded ones
|
||||
// is not. `--no-tia` needs none of this: it still runs the whole
|
||||
// suite, so it remains a complete run.
|
||||
$this->resultsOnlyWrites = true;
|
||||
|
||||
// `$this->filteredMode` counts as asking for it: reaching here
|
||||
// means the narrowing came from the command line while filtered
|
||||
// mode came from the environment or the config, and a run that
|
||||
// silently declines what the config asked for is the one most
|
||||
// in need of the explanation.
|
||||
if ($cliEnabled || $freshRequested || $this->forceRefetch || $this->filteredMode) {
|
||||
$this->output->writeln('');
|
||||
$this->renderChild('TIA does not apply to partial runs — running the selected tests directly.');
|
||||
@@ -586,15 +507,22 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
$this->forceRefetch = false;
|
||||
$this->filteredMode = false;
|
||||
$this->freshRebuild = false;
|
||||
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
if ($isWorker && (string) Parallel::getGlobal(self::WORKER_RESULTS_GLOBAL) === '1') {
|
||||
$this->flushesWorkerResults = true;
|
||||
$this->resultsOnlyWrites = true;
|
||||
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
$forceRebuild = $freshRequested && ($enabled || $recordingGlobal || $replayingGlobal);
|
||||
$this->freshRebuild = $forceRebuild;
|
||||
|
||||
if (! $enabled && ! $this->forceRefetch && ! $recordingGlobal && ! $replayingGlobal) {
|
||||
$this->requestWorkerResults();
|
||||
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
@@ -617,16 +545,11 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return;
|
||||
}
|
||||
|
||||
if (Parallel::isWorker() && ($this->replayGraph instanceof Graph || $this->recordingActive)) {
|
||||
if (Parallel::isWorker() && ($this->replayGraph instanceof Graph || $this->recordingActive || $this->flushesWorkerResults)) {
|
||||
$this->flushWorkerReplay();
|
||||
}
|
||||
|
||||
// Both only ever set for the parent — addOutput() returns early in
|
||||
// workers, whose partials are ephemeral and only reach the baseline if
|
||||
// the parent consumes them. Everything this method goes on to write is
|
||||
// whole-suite by nature — the edge map above all — so a narrowed run
|
||||
// stops here too, its results already persisted by addOutput().
|
||||
if ($this->writesSuppressed || $this->resultsOnlyWrites) {
|
||||
if ($this->writesSuppressed || $this->resultsOnlyWrites || $this->hasUnfinishedTest()) {
|
||||
$this->recorder->reset();
|
||||
$this->coverageCollector->reset();
|
||||
|
||||
@@ -699,10 +622,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$graph->replaceTestInertiaComponents($perTestInertia);
|
||||
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
|
||||
|
||||
if ($this->freshRebuild) {
|
||||
$graph->pruneMissingTests();
|
||||
}
|
||||
|
||||
$this->seedResultsInto($graph);
|
||||
|
||||
if (! $this->saveGraph($graph)) {
|
||||
@@ -722,20 +641,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
// `->only()` narrows the executed set exactly like `--filter` does, but
|
||||
// is only knowable once the suite has been collected — too late to turn
|
||||
// TIA off up front. Sampled in addOutput() because Only's lock file is
|
||||
// already gone by the time terminate() runs (its plugin terminates
|
||||
// first). Whether the run was cut short is likewise only knowable now.
|
||||
if (Only::isEnabled() || $this->stoppedEarly()) {
|
||||
if (Only::isEnabled() || $this->stoppedEarly() || $this->hasUnfinishedTest()) {
|
||||
$this->resultsOnlyWrites = true;
|
||||
}
|
||||
|
||||
$this->reportMissingWorkerDrivers();
|
||||
|
||||
// Runs before the checks below: it is what fills the parent's result
|
||||
// collector in parallel, and a worker that stopped early narrows the
|
||||
// whole run.
|
||||
if (Parallel::isEnabled()) {
|
||||
$this->mergeWorkerReplayPartials();
|
||||
}
|
||||
@@ -750,7 +661,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
if ($this->replayRan) {
|
||||
if ($this->replayRan || $this->graphUnreachable) {
|
||||
$this->bumpRecordedSha();
|
||||
}
|
||||
|
||||
@@ -815,10 +726,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$graph->replaceTestInertiaComponents($finalisedInertia);
|
||||
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
|
||||
|
||||
if ($this->freshRebuild) {
|
||||
$graph->pruneMissingTests();
|
||||
}
|
||||
|
||||
if (! $this->saveGraph($graph)) {
|
||||
$this->renderBadge('ERROR', 'Could not write the dependency graph.');
|
||||
|
||||
@@ -861,8 +768,8 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $this->reconcileFingerprint($rebuilt, $current);
|
||||
}
|
||||
|
||||
$this->state->delete(self::KEY_GRAPH);
|
||||
$this->state->delete(self::KEY_COVERAGE_CACHE);
|
||||
$this->deleteState(self::KEY_GRAPH);
|
||||
$this->deleteState(self::KEY_COVERAGE_CACHE);
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -878,7 +785,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$graph->clearResults($this->branch);
|
||||
$graph->setFingerprint($current);
|
||||
$this->saveGraph($graph);
|
||||
$this->state->delete(self::KEY_COVERAGE_CACHE);
|
||||
$this->deleteState(self::KEY_COVERAGE_CACHE);
|
||||
}
|
||||
|
||||
return $graph;
|
||||
@@ -898,14 +805,18 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
Panic::with(new TiaRequiresRepositoryRoot($subdirectoryPrefix));
|
||||
}
|
||||
|
||||
$this->resolveBranch($projectRoot);
|
||||
try {
|
||||
$this->resolveBranch($projectRoot);
|
||||
} catch (MissingDependency $missingGit) {
|
||||
$repository = new ChangedFiles($projectRoot);
|
||||
|
||||
if ($repository->isRepository() && ! $repository->hasCommits()) {
|
||||
Panic::with(new TiaRequiresCommit);
|
||||
}
|
||||
|
||||
throw $missingGit;
|
||||
}
|
||||
|
||||
// After resolveBranch(), so a directory that is no repository at all
|
||||
// still reports the missing git dependency rather than an unresolved
|
||||
// default branch. Nothing named the branch every other baseline reads
|
||||
// through, so every new branch would re-run the whole suite while the
|
||||
// output called it a hit. A repository with no remote is the likeliest
|
||||
// reason and gets said out loud.
|
||||
if (! $this->fallbackBranchResolved) {
|
||||
Panic::with(new ChangedFiles($projectRoot)->hasRemote()
|
||||
? new TiaRequiresDefaultBranch
|
||||
@@ -915,7 +826,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$fingerprint = Fingerprint::compute($projectRoot);
|
||||
$this->startFingerprint = $fingerprint;
|
||||
|
||||
if ($forceRebuild) {
|
||||
if ($forceRebuild && ! $this->detachedHead) {
|
||||
Storage::purge($projectRoot);
|
||||
}
|
||||
|
||||
@@ -933,6 +844,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
&& $changedFiles->since($branchSha) === null) {
|
||||
$this->renderBadge('WARN', 'Recorded commit is no longer reachable — graph will be rebuilt.');
|
||||
$graph = null;
|
||||
$this->graphUnreachable = true;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -948,19 +860,20 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
}
|
||||
}
|
||||
|
||||
// Both of these belong to the coverage cache, which only Pest's own
|
||||
// `--coverage` ever writes or merges. A raw PHPUnit report flag takes
|
||||
// the piggyback path — it must not drive the driver itself — but must
|
||||
// not leave a marker behind, nor force a recording run to prime a cache
|
||||
// that nothing on its path will fill.
|
||||
$coverageCacheOwned = $this->piggybackCoverage && $this->pestCoverageActive();
|
||||
|
||||
if ($coverageCacheOwned) {
|
||||
$this->state->write(self::KEY_COVERAGE_MARKER, '');
|
||||
}
|
||||
|
||||
if (! $graph instanceof Graph && $this->piggybackCoverage) {
|
||||
$this->emitCoverageScopedRecordSkipped();
|
||||
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
if ($coverageCacheOwned && ! $this->state->exists(self::KEY_COVERAGE_CACHE)) {
|
||||
if ($graph instanceof Graph && $this->driftLabel === null) {
|
||||
if ($this->driftLabel === null) {
|
||||
$this->freshGraphReason = 'recording a coverage baseline';
|
||||
}
|
||||
|
||||
@@ -1115,7 +1028,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
$affectedFromChanges = $changed === [] ? [] : $graph->affected($changed);
|
||||
$affectedFromChanges = $changed === [] ? [] : $graph->testFilesOnDisk($graph->affected($changed));
|
||||
$rerunFromCache = [];
|
||||
|
||||
if ($this->filteredMode && $graph->hasUnlocatedTestsToRerun($this->branch)) {
|
||||
@@ -1157,9 +1070,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
if (! Parallel::isEnabled()) {
|
||||
if ($canRefreshReplayEdges) {
|
||||
// Piggyback runs read PHPUnit's own coverage session. Driving
|
||||
// the driver alongside it would clear the data PHPUnit is about
|
||||
// to read, so only link tracking may run here.
|
||||
if ($this->piggybackCoverage) {
|
||||
$this->recorder->activateLinkTracking();
|
||||
} else {
|
||||
@@ -1329,9 +1239,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$recorder->activate();
|
||||
$this->recordingActive = true;
|
||||
|
||||
// Why this run is rebuilding is worth saying whenever there is a reason
|
||||
// for it — the parallel and piggyback branches above already do. Runs
|
||||
// that are simply recording for the first time have nothing to explain.
|
||||
if ($this->driftLabel !== null || $this->freshGraphReason !== null) {
|
||||
$this->output->writeln('');
|
||||
$this->renderFreshGraph();
|
||||
@@ -1347,8 +1254,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
private function renderFreshGraph(): void
|
||||
{
|
||||
if ($this->driftLabel === null && $this->freshGraphReason !== null) {
|
||||
// The reason is only ever set for a run that keeps its graph and
|
||||
// records alongside it, so "fresh graph" would be a lie here.
|
||||
$headline = sprintf('Experimental TIA mode enabled / %s.', $this->freshGraphReason);
|
||||
} else {
|
||||
$headline = 'Experimental TIA mode enabled / fresh graph';
|
||||
@@ -1376,6 +1281,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$this->renderChild('Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug.');
|
||||
}
|
||||
|
||||
private function emitCoverageScopedRecordSkipped(): void
|
||||
{
|
||||
$this->output->writeln('');
|
||||
|
||||
$this->renderChild('Running in TIA mode, however TIA is skipped as an active coverage report narrows the edges it could record.');
|
||||
$this->renderChild('Record the baseline with a plain --tia run first; coverage runs then reuse it.');
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, array<int, string>> $perTestFiles
|
||||
* @param array<string, array<int, string>> $perTestTables
|
||||
@@ -1423,6 +1336,21 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$this->renderChild('Install / enable pcov or xdebug (mode: coverage) in the worker PHP and rerun.');
|
||||
}
|
||||
|
||||
private function requestWorkerResults(): void
|
||||
{
|
||||
if (Parallel::isWorker() || ! Parallel::isEnabled() || $this->writesSuppressed) {
|
||||
return;
|
||||
}
|
||||
|
||||
if ($this->state->read(self::KEY_GRAPH) === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->purgeWorkerPartials();
|
||||
|
||||
Parallel::setGlobal(self::WORKER_RESULTS_GLOBAL, '1');
|
||||
}
|
||||
|
||||
private function purgeWorkerPartials(): void
|
||||
{
|
||||
foreach ($this->collectWorkerEdgesPartials() as $key) {
|
||||
@@ -1444,14 +1372,8 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return;
|
||||
}
|
||||
|
||||
// A replayed result carries the duration PHPUnit measured for a test
|
||||
// that never ran — near zero. Unlike the assertion count, which the
|
||||
// replay injects into the result itself, the cached duration lives only
|
||||
// in this process: the parent replayed nothing of its own, so once the
|
||||
// partial is written the real value is unrecoverable. Launder it here
|
||||
// and the parent's verbatim read is correct by construction.
|
||||
foreach ($results as $testId => $result) {
|
||||
$results[$testId]['time'] = $this->resultTime($testId, $result['time']);
|
||||
$results[$testId] = $this->replayedAsRecorded($testId, $result);
|
||||
}
|
||||
|
||||
$json = json_encode([
|
||||
@@ -1459,9 +1381,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
'replayed' => $this->replayedCount,
|
||||
'affected' => $this->affectedCount,
|
||||
'executed' => $this->executedCount,
|
||||
// Only the worker knows it stopped early — the parent runs no tests
|
||||
// of its own, so its own check would always come back clean.
|
||||
'truncated' => $this->stoppedEarly(),
|
||||
'truncated' => $this->stoppedEarly() || $collector->hasUnfinishedTest(),
|
||||
], JSON_UNESCAPED_SLASHES);
|
||||
|
||||
if ($json === false) {
|
||||
@@ -1498,9 +1418,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
continue;
|
||||
}
|
||||
|
||||
// One worker stopping early leaves the whole suite incomplete: the
|
||||
// tests it never reached are missing from the merged result set just
|
||||
// as if they had been filtered out.
|
||||
if (($decoded['truncated'] ?? false) === true) {
|
||||
$this->resultsOnlyWrites = true;
|
||||
}
|
||||
@@ -1728,15 +1645,29 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return $coverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* The duration to record for a test: its own, unless it was replayed rather
|
||||
* than executed, in which case the duration it was recorded with stands.
|
||||
*/
|
||||
private function resultTime(string $testId, float $time): float
|
||||
{
|
||||
return $this->cachedTimeByTestId[$testId] ?? $time;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array{status: int, message: string, time: float, assertions: int, file?: string} $result
|
||||
* @return array{status: int, message: string, time: float, assertions: int, file?: string}
|
||||
*/
|
||||
private function replayedAsRecorded(string $testId, array $result): array
|
||||
{
|
||||
$result['time'] = $this->resultTime($testId, $result['time']);
|
||||
|
||||
$cached = $this->cachedStatusByTestId[$testId] ?? null;
|
||||
|
||||
if ($cached !== null) {
|
||||
$result['status'] = $cached['status'];
|
||||
$result['message'] = $cached['message'];
|
||||
}
|
||||
|
||||
return $result;
|
||||
}
|
||||
|
||||
private function seedResultsInto(Graph $graph): void
|
||||
{
|
||||
/** @var ResultCollector $collector */
|
||||
@@ -1756,12 +1687,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$touchedFiles[$file] = true;
|
||||
}
|
||||
|
||||
$result = $this->replayedAsRecorded($testId, $result);
|
||||
|
||||
$graph->setResult(
|
||||
$this->branch,
|
||||
$testId,
|
||||
$result['status'],
|
||||
$result['message'],
|
||||
$this->resultTime($testId, $result['time']),
|
||||
$result['time'],
|
||||
$result['assertions'],
|
||||
$file,
|
||||
);
|
||||
@@ -1769,17 +1702,33 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
$graph->markKnownTestFiles(array_keys($touchedFiles));
|
||||
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
|
||||
$this->reclaim($graph);
|
||||
|
||||
$collector->reset();
|
||||
}
|
||||
|
||||
/**
|
||||
* Folds the run's results into the existing graph.
|
||||
*
|
||||
* An incomplete run passes `$complete: false`, which keeps the additive
|
||||
* half — the results of the tests it did run — and drops the half that
|
||||
* speaks for the suite as a whole.
|
||||
*/
|
||||
private function reclaim(Graph $graph): void
|
||||
{
|
||||
if ($this->branch !== $this->fallbackBranch) {
|
||||
$graph->markBaselineComplete($this->branch);
|
||||
}
|
||||
|
||||
$graph->pruneMissingTests();
|
||||
$graph->pruneResultsForMissingFiles($this->branch);
|
||||
|
||||
$branches = new ChangedFiles(TestSuite::getInstance()->rootPath)->branchNames();
|
||||
|
||||
if ($branches === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (! in_array($this->fallbackBranch, $branches, true)) {
|
||||
return;
|
||||
}
|
||||
|
||||
$graph->pruneMissingBranches([...$branches, $this->branch, $this->fallbackBranch]);
|
||||
}
|
||||
|
||||
private function snapshotTestResults(bool $markKnownTestFiles = false, bool $complete = true): void
|
||||
{
|
||||
/** @var ResultCollector $collector */
|
||||
@@ -1802,23 +1751,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
try {
|
||||
$this->resolveBranch($projectRoot);
|
||||
} catch (MissingDependency) {
|
||||
// This run never asked for TIA, so a missing git must not turn it
|
||||
// into a failure the way it does on the TIA path. Writing to the
|
||||
// fallback baseline is the lesser of the two evils.
|
||||
}
|
||||
|
||||
// The graph above was loaded before the branch was known — this path
|
||||
// only writes, but a graph carrying an unresolved fallback is the exact
|
||||
// bug this whole change is about.
|
||||
$graph->setFallbackBranch($this->fallbackBranch);
|
||||
|
||||
$touchedFiles = [];
|
||||
|
||||
// Whether this run is the one that records the edges its results will be
|
||||
// invalidated through. A recording run's edges are written after this
|
||||
// (terminate() runs last), and a parallel one's arrive with the worker
|
||||
// partials that ask for $markKnownTestFiles — either way the graph on
|
||||
// disk cannot be asked yet, so the run is taken at its word.
|
||||
$recordsEdges = $complete && ($markKnownTestFiles || $this->recordingActive);
|
||||
|
||||
foreach ($results as $testId => $result) {
|
||||
@@ -1832,22 +1770,18 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$touchedFiles[$file] = true;
|
||||
}
|
||||
|
||||
// A result is only ever invalidated through the edges of the test
|
||||
// that produced it, so one recorded for a test the graph has no
|
||||
// edges for could never be invalidated again — it would be replayed
|
||||
// as settled however far the code around it moved. A run that
|
||||
// records no edges leaves such a test exactly as unknown as it
|
||||
// found it, whether or not it ran the whole suite.
|
||||
if (! $recordsEdges && (! is_string($file) || ! $graph->knowsTest($file))) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$result = $this->replayedAsRecorded($testId, $result);
|
||||
|
||||
$graph->setResult(
|
||||
$this->branch,
|
||||
$testId,
|
||||
$result['status'],
|
||||
$result['message'],
|
||||
$this->resultTime($testId, $result['time']),
|
||||
$result['time'],
|
||||
$result['assertions'],
|
||||
$file,
|
||||
);
|
||||
@@ -1857,10 +1791,9 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
$graph->markKnownTestFiles(array_keys($touchedFiles));
|
||||
}
|
||||
|
||||
// Pruning reads the absence of a test from this run as the test being
|
||||
// gone. That only holds if every test was invited to report.
|
||||
if ($complete) {
|
||||
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
|
||||
$this->reclaim($graph);
|
||||
}
|
||||
|
||||
$this->saveGraph($graph);
|
||||
@@ -1898,15 +1831,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this run produces a coverage report, however it was asked for.
|
||||
*
|
||||
* The original argv, not the handled arguments: Pest's own Coverage plugin
|
||||
* appends `--coverage-php <path>` to those and runs before this one, and a
|
||||
* paratest worker's arguments always carry it too. `bin/worker.php` never
|
||||
* hands over the original argv, so a worker sees `[]` here and keeps taking
|
||||
* this from {@see self::PIGGYBACK_COVERAGE_GLOBAL} instead.
|
||||
*/
|
||||
private function coverageReportActive(): bool
|
||||
{
|
||||
if ($this->pestCoverageActive()) {
|
||||
@@ -1916,10 +1840,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return array_any(self::COVERAGE_REPORT_FLAGS, fn (string $flag): bool => $this->hasArgument($flag, $this->originalArguments));
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether Pest's own `--coverage` was given — the only entry point that
|
||||
* writes the coverage cache these two flags read and clean up.
|
||||
*/
|
||||
private function pestCoverageActive(): bool
|
||||
{
|
||||
$coverage = Container::getInstance()->get(Coverage::class);
|
||||
@@ -1929,11 +1849,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
}
|
||||
|
||||
/**
|
||||
* Panics when the run asks for Tia alongside an option Tia cannot honour.
|
||||
*
|
||||
* Checked against the original argv as well, because `bin/pest` consumes
|
||||
* some of these itself before PHPUnit ever sees them.
|
||||
*
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function guardUnsupportedOptions(array $arguments): void
|
||||
@@ -1952,11 +1867,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a selection-narrowing flag was given, either among the arguments
|
||||
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
|
||||
* the original argv. Explicit path arguments and `->only()` are detected
|
||||
* separately.
|
||||
*
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function hasPartialSelection(array $arguments): bool
|
||||
@@ -1974,36 +1884,19 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the run stopped before reaching every test it had queued.
|
||||
*
|
||||
* Covers `--bail`, `--retry` and every `--stop-on-*` flag, the equivalent
|
||||
* `phpunit.xml` attributes, and an interrupted run — none of which narrow
|
||||
* the selection up front, so hasPartialSelection() cannot see them. The
|
||||
* tests queued behind the defect that halted the run never reported, and
|
||||
* folding what did report into the baseline prunes the cached results of
|
||||
* their siblings in every file the run had already entered.
|
||||
*
|
||||
* Deliberately unguarded. Both callers run only once PHPUnit's
|
||||
* configuration is registered — the kernel reads it unguarded itself just
|
||||
* before dispatching addOutput(), and flushWorkerReplay() bails out unless
|
||||
* the worker actually executed something. Swallowing a failure here would
|
||||
* report every truncated run as complete, which is the corruption this
|
||||
* guards against in the first place.
|
||||
*/
|
||||
private function stoppedEarly(): bool
|
||||
{
|
||||
return TestResultFacade::shouldStop();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the baselines this run reads from and writes to, once.
|
||||
*
|
||||
* Results are written on runs where TIA itself took no part, and those
|
||||
* never reach handleParent(). Without this the default would stand and
|
||||
* every such run would write its results to `main`, whatever branch it
|
||||
* actually ran on.
|
||||
*/
|
||||
private function hasUnfinishedTest(): bool
|
||||
{
|
||||
$collector = Container::getInstance()->get(ResultCollector::class);
|
||||
assert($collector instanceof ResultCollector);
|
||||
|
||||
return $collector->hasUnfinishedTest();
|
||||
}
|
||||
|
||||
private function resolveBranch(string $projectRoot): void
|
||||
{
|
||||
if ($this->branchResolved) {
|
||||
@@ -2014,9 +1907,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
$changedFiles = new ChangedFiles($projectRoot);
|
||||
|
||||
// Resolved before the current branch, which throws where git is
|
||||
// missing: the fallback is advisory, so a run that cannot name its
|
||||
// branch at all should still carry the best answer available.
|
||||
$resolved = $this->resolveFallbackBranch($changedFiles);
|
||||
|
||||
$this->fallbackBranchResolved = $resolved !== null;
|
||||
@@ -2024,24 +1914,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
|
||||
Parallel::setGlobal(self::FALLBACK_BRANCH_GLOBAL, $this->fallbackBranch);
|
||||
|
||||
// A detached HEAD has no branch of its own to write to. The default
|
||||
// branch is the honest key there — it is the commit the checkout most
|
||||
// likely sits on, and it keeps a phantom baseline from being minted
|
||||
// under a branch name the repository never had.
|
||||
$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch;
|
||||
$currentBranch = $changedFiles->currentBranch();
|
||||
|
||||
$this->detachedHead = $currentBranch === null;
|
||||
$this->branch = $currentBranch ?? $this->fallbackBranch;
|
||||
}
|
||||
|
||||
/**
|
||||
* The branch every other baseline falls back to reading, or null when
|
||||
* nothing in the checkout can name it.
|
||||
*
|
||||
* Ordered by how much the source actually knows. Configuration first: it is
|
||||
* the escape hatch for a repository whose git-side answers disagree with its
|
||||
* branches. Then the CI provider, which states the answer outright where git
|
||||
* is at its least informed. Then git itself. Then the recorded graph, whose
|
||||
* single baseline can only have come from the branch this repository
|
||||
* integrates on.
|
||||
*/
|
||||
private function resolveFallbackBranch(ChangedFiles $changedFiles): ?string
|
||||
{
|
||||
$inherited = Parallel::getGlobal(self::FALLBACK_BRANCH_GLOBAL);
|
||||
@@ -2056,14 +1934,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
?? $this->soleRecordedBranch();
|
||||
}
|
||||
|
||||
/**
|
||||
* The one branch a recorded graph holds a baseline for.
|
||||
*
|
||||
* Last in the chain and deliberately narrow: with a single baseline on disk
|
||||
* there is only one branch whose results can be read at all, so naming it is
|
||||
* strictly better than resolving to a branch that holds nothing. Two or more
|
||||
* baselines carry no such implication and are left alone.
|
||||
*/
|
||||
private function soleRecordedBranch(): ?string
|
||||
{
|
||||
$json = $this->state->read(self::KEY_GRAPH);
|
||||
@@ -2098,11 +1968,15 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
if (str_starts_with($arg, '-')) {
|
||||
continue;
|
||||
}
|
||||
if ($index > 0) {
|
||||
$previous = $arguments[$index - 1] ?? '';
|
||||
if (in_array($previous, self::VALUE_TAKING_FLAGS, true)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if ($index === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$previous = $arguments[$index - 1] ?? '';
|
||||
|
||||
if (in_array($previous, self::VALUE_TAKING_FLAGS, true)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$candidate = $this->resolveArgumentPath($arg, $projectRoot);
|
||||
@@ -2111,16 +1985,28 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
continue;
|
||||
}
|
||||
|
||||
foreach ($testPaths as $testPath) {
|
||||
if ($candidate === $testPath || str_starts_with($candidate, $testPath.DIRECTORY_SEPARATOR)) {
|
||||
return true;
|
||||
}
|
||||
if ($this->narrowsSuite($candidate, $testPaths)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $testPaths
|
||||
*/
|
||||
private function narrowsSuite(string $candidate, array $testPaths): bool
|
||||
{
|
||||
foreach ($testPaths as $testPath) {
|
||||
if ($candidate === $testPath || str_starts_with($candidate, $testPath.DIRECTORY_SEPARATOR)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return array_all($testPaths, fn (string $testPath): bool => ! str_starts_with($testPath, $candidate.DIRECTORY_SEPARATOR));
|
||||
}
|
||||
|
||||
private function resolveArgumentPath(string $arg, string $projectRoot): ?string
|
||||
{
|
||||
$candidates = [$arg, rtrim($projectRoot, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR.ltrim($arg, DIRECTORY_SEPARATOR)];
|
||||
@@ -2250,16 +2136,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
*/
|
||||
private function gitSubdirectoryPrefix(string $projectRoot): ?string
|
||||
{
|
||||
$process = new Process(['git', 'rev-parse', '--show-prefix'], $projectRoot);
|
||||
$process->run();
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$prefix = trim($process->getOutput());
|
||||
|
||||
return $prefix === '' ? null : rtrim(str_replace(DIRECTORY_SEPARATOR, '/', $prefix), '/');
|
||||
return new Git($projectRoot)->subdirectoryPrefix();
|
||||
}
|
||||
|
||||
private function composerLockDelta(string $projectRoot, string $sha): string
|
||||
@@ -2269,15 +2146,13 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
|
||||
return '';
|
||||
}
|
||||
|
||||
$process = new Process(['git', 'show', $sha.':composer.lock'], $projectRoot);
|
||||
$process->setTimeout(5.0);
|
||||
$process->run();
|
||||
$baseline = new Git($projectRoot)->show($sha, 'composer.lock');
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($baseline === null) {
|
||||
return '';
|
||||
}
|
||||
|
||||
$oldVersions = $this->lockVersions($process->getOutput());
|
||||
$oldVersions = $this->lockVersions($baseline);
|
||||
$newVersions = $this->lockVersions($current);
|
||||
|
||||
if ($oldVersions === [] && $newVersions === []) {
|
||||
|
||||
@@ -5,14 +5,19 @@ declare(strict_types=1);
|
||||
namespace Pest\Plugins\Tia;
|
||||
|
||||
use Pest\Exceptions\MissingDependency;
|
||||
use Symfony\Component\Process\Process;
|
||||
use Pest\Support\Git;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class ChangedFiles
|
||||
{
|
||||
public function __construct(private string $projectRoot) {}
|
||||
private Git $git;
|
||||
|
||||
public function __construct(private string $projectRoot)
|
||||
{
|
||||
$this->git = new Git($projectRoot);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $files project-relative paths.
|
||||
@@ -155,15 +160,7 @@ final readonly class ChangedFiles
|
||||
|
||||
private function contentAtSha(string $sha, string $path): ?string
|
||||
{
|
||||
$process = new Process(['git', 'show', $sha.':'.$path], $this->projectRoot);
|
||||
$process->setTimeout(5.0);
|
||||
$process->run();
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return $process->getOutput();
|
||||
return $this->git->show($sha, $path);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -176,21 +173,17 @@ final readonly class ChangedFiles
|
||||
return $candidates;
|
||||
}
|
||||
|
||||
$process = new Process(
|
||||
['git', 'check-ignore', '--no-index', '-z', '--stdin'],
|
||||
$this->projectRoot,
|
||||
$result = $this->git->result(
|
||||
['check-ignore', '--no-index', '-z', '--stdin'],
|
||||
implode("\x00", array_keys($candidates)),
|
||||
);
|
||||
$process->setTimeout(5.0);
|
||||
$process->setInput(implode("\x00", array_keys($candidates)));
|
||||
$process->run();
|
||||
|
||||
$exitCode = $process->getExitCode();
|
||||
|
||||
if ($exitCode !== 0 && $exitCode !== 1) {
|
||||
// `check-ignore` exits 1 when nothing matched — that is not a failure.
|
||||
if ($result['exitCode'] !== 0 && $result['exitCode'] !== 1) {
|
||||
throw new MissingDependency('Tia mode', 'git');
|
||||
}
|
||||
|
||||
$output = $process->getOutput();
|
||||
$output = $result['output'];
|
||||
|
||||
if ($output === '') {
|
||||
return $candidates;
|
||||
@@ -207,31 +200,20 @@ final readonly class ChangedFiles
|
||||
|
||||
public function currentBranch(): ?string
|
||||
{
|
||||
$process = new Process(['git', 'rev-parse', '--abbrev-ref', 'HEAD'], $this->projectRoot);
|
||||
$process->run();
|
||||
$output = $this->git->raw(['rev-parse', '--abbrev-ref', 'HEAD']);
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($output === null) {
|
||||
throw new MissingDependency('Tia mode', 'git');
|
||||
}
|
||||
|
||||
$branch = trim($process->getOutput());
|
||||
$branch = trim($output);
|
||||
|
||||
return $branch === '' || $branch === 'HEAD' ? null : $branch;
|
||||
}
|
||||
|
||||
/**
|
||||
* The repository's default branch — the one every other branch's baseline
|
||||
* falls back to reading.
|
||||
*
|
||||
* Advisory, unlike {@see self::currentBranch()}: a repository that cannot
|
||||
* answer the question is not a broken repository. A remote-less checkout
|
||||
* has no `origin/HEAD`, and plenty of CI checkouts never run
|
||||
* `git remote set-head`, so every step here fails soft and the caller is
|
||||
* left to pick its own default.
|
||||
*/
|
||||
public function defaultBranch(): ?string
|
||||
{
|
||||
$head = $this->gitOutput(['git', 'symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
|
||||
$head = $this->git->output(['symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
|
||||
|
||||
if ($head !== null) {
|
||||
$branch = preg_replace('#^origin/#', '', $head);
|
||||
@@ -241,60 +223,86 @@ final readonly class ChangedFiles
|
||||
}
|
||||
}
|
||||
|
||||
// `init.defaultBranch` is a setting of the machine, not of the
|
||||
// repository — it names what `git init` would have called the first
|
||||
// branch here, which is worth nothing once the repository disagrees.
|
||||
// Taken only when a branch by that name actually exists.
|
||||
$configured = $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
|
||||
$configured = $this->git->output(['config', '--get', 'init.defaultBranch']);
|
||||
|
||||
if ($configured === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$exists = $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/heads/'.$configured]) !== null
|
||||
|| $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/remotes/origin/'.$configured]) !== null;
|
||||
$exists = $this->git->hasRef('refs/heads/'.$configured)
|
||||
|| $this->git->hasRef('refs/remotes/origin/'.$configured);
|
||||
|
||||
return $exists ? $configured : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the repository has any remote configured.
|
||||
*
|
||||
* Advisory like {@see self::defaultBranch()} — a `git` that cannot answer
|
||||
* is reported as "no remote", and the caller decides what that means.
|
||||
* @return list<string>|null
|
||||
*/
|
||||
public function hasRemote(): bool
|
||||
public function branchNames(): ?array
|
||||
{
|
||||
return $this->gitOutput(['git', 'remote']) !== null;
|
||||
}
|
||||
$output = $this->git->raw(['for-each-ref', '--format=%(refname)', 'refs/heads', 'refs/remotes']);
|
||||
|
||||
/**
|
||||
* @param array<int, string> $command
|
||||
*/
|
||||
private function gitOutput(array $command): ?string
|
||||
{
|
||||
$process = new Process($command, $this->projectRoot);
|
||||
$process->setTimeout(5.0);
|
||||
$process->run();
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($output === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$output = trim($process->getOutput());
|
||||
$names = [];
|
||||
|
||||
return $output === '' ? null : $output;
|
||||
foreach ($this->splitLines($output) as $ref) {
|
||||
if (str_starts_with($ref, 'refs/heads/')) {
|
||||
$names[substr($ref, strlen('refs/heads/'))] = true;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (! str_starts_with($ref, 'refs/remotes/')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$tail = substr($ref, strlen('refs/remotes/'));
|
||||
$slash = strpos($tail, '/');
|
||||
|
||||
if ($slash === false) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$branch = substr($tail, $slash + 1);
|
||||
|
||||
if ($branch !== '' && $branch !== 'HEAD') {
|
||||
$names[$branch] = true;
|
||||
}
|
||||
}
|
||||
|
||||
return array_keys($names);
|
||||
}
|
||||
|
||||
public function hasRemote(): bool
|
||||
{
|
||||
return $this->git->hasRemote();
|
||||
}
|
||||
|
||||
public function isRepository(): bool
|
||||
{
|
||||
return $this->git->isRepository();
|
||||
}
|
||||
|
||||
public function hasCommits(): bool
|
||||
{
|
||||
return $this->git->hasCommits();
|
||||
}
|
||||
|
||||
/**
|
||||
* Working-tree scans get a longer leash than metadata queries — on a large
|
||||
* repository with a cold cache, `status` and `diff` are not instant.
|
||||
*/
|
||||
private function scan(): Git
|
||||
{
|
||||
return $this->git->withTimeout(60.0);
|
||||
}
|
||||
|
||||
private function shaIsReachable(string $sha): bool
|
||||
{
|
||||
$process = new Process(
|
||||
['git', 'merge-base', '--is-ancestor', $sha, 'HEAD'],
|
||||
$this->projectRoot,
|
||||
);
|
||||
$process->run();
|
||||
|
||||
return $process->getExitCode() === 0;
|
||||
return $this->git->succeeds(['merge-base', '--is-ancestor', $sha, 'HEAD']);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -302,17 +310,13 @@ final readonly class ChangedFiles
|
||||
*/
|
||||
private function diffSinceSha(string $sha): array
|
||||
{
|
||||
$process = new Process(
|
||||
['git', 'diff', '--name-only', $sha.'..HEAD'],
|
||||
$this->projectRoot,
|
||||
);
|
||||
$process->run();
|
||||
$output = $this->scan()->raw(['diff', '--name-only', '--no-renames', $sha.'..HEAD']);
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($output === null) {
|
||||
throw new MissingDependency('Tia mode', 'git');
|
||||
}
|
||||
|
||||
return $this->splitLines($process->getOutput());
|
||||
return $this->splitLines($output);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -320,18 +324,12 @@ final readonly class ChangedFiles
|
||||
*/
|
||||
private function workingTreeChanges(): array
|
||||
{
|
||||
$process = new Process(
|
||||
['git', 'status', '--porcelain', '-z', '--untracked-files=all'],
|
||||
$this->projectRoot,
|
||||
);
|
||||
$process->run();
|
||||
$output = $this->scan()->raw(['status', '--porcelain', '-z', '--untracked-files=all']);
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($output === null) {
|
||||
throw new MissingDependency('Tia mode', 'git');
|
||||
}
|
||||
|
||||
$output = $process->getOutput();
|
||||
|
||||
if ($output === '') {
|
||||
return [];
|
||||
}
|
||||
@@ -369,14 +367,13 @@ final readonly class ChangedFiles
|
||||
|
||||
public function currentSha(): ?string
|
||||
{
|
||||
$process = new Process(['git', 'rev-parse', 'HEAD'], $this->projectRoot);
|
||||
$process->run();
|
||||
$output = $this->git->raw(['rev-parse', 'HEAD']);
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($output === null) {
|
||||
throw new MissingDependency('Tia mode', 'git');
|
||||
}
|
||||
|
||||
$sha = trim($process->getOutput());
|
||||
$sha = trim($output);
|
||||
|
||||
return $sha === '' ? null : $sha;
|
||||
}
|
||||
|
||||
@@ -4,74 +4,31 @@ declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia;
|
||||
|
||||
use Pest\Plugins\Tia\Contracts\Ci;
|
||||
|
||||
/**
|
||||
* The default branch as the CI provider itself reports it.
|
||||
*
|
||||
* Worth asking before git: a CI checkout is the one place where git knows the
|
||||
* least. `actions/checkout` builds the working copy with `git init` plus a
|
||||
* single-ref `fetch` rather than a `clone`, so `origin/HEAD` is never set and
|
||||
* `init.defaultBranch` — a setting of the runner image, not of the repository —
|
||||
* is all git has left to offer. The provider, meanwhile, states the answer
|
||||
* outright in the environment it handed us.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class CiDefaultBranch
|
||||
{
|
||||
/**
|
||||
* Advisory, like every other source in the chain: anything unreadable,
|
||||
* unparsable, or simply absent means "no answer", never a failure.
|
||||
* @var array<int, class-string<Ci>>
|
||||
*/
|
||||
private const array CIS = [
|
||||
Cis\GitLab::class,
|
||||
Cis\GitHub::class,
|
||||
];
|
||||
|
||||
public static function detect(): ?string
|
||||
{
|
||||
return self::fromGitLab() ?? self::fromGitHubEvent();
|
||||
}
|
||||
foreach (self::CIS as $class) {
|
||||
$branch = (new $class)->defaultBranch();
|
||||
|
||||
private static function fromGitLab(): ?string
|
||||
{
|
||||
return self::environment('CI_DEFAULT_BRANCH');
|
||||
}
|
||||
|
||||
/**
|
||||
* GitHub publishes no default-branch variable, but every repository-scoped
|
||||
* event payload carries `repository.default_branch`, and the path to that
|
||||
* payload is in the environment.
|
||||
*/
|
||||
private static function fromGitHubEvent(): ?string
|
||||
{
|
||||
$path = self::environment('GITHUB_EVENT_PATH');
|
||||
|
||||
if ($path === null || ! is_file($path) || ! is_readable($path)) {
|
||||
return null;
|
||||
if ($branch !== null) {
|
||||
return $branch;
|
||||
}
|
||||
}
|
||||
|
||||
$contents = @file_get_contents($path);
|
||||
|
||||
if ($contents === false) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$payload = json_decode($contents, true);
|
||||
|
||||
if (! is_array($payload) || ! is_array($payload['repository'] ?? null)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$branch = $payload['repository']['default_branch'] ?? null;
|
||||
|
||||
return is_string($branch) && $branch !== '' ? $branch : null;
|
||||
}
|
||||
|
||||
private static function environment(string $name): ?string
|
||||
{
|
||||
$value = getenv($name);
|
||||
|
||||
if (! is_string($value)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$value = trim($value);
|
||||
|
||||
return $value === '' ? null : $value;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia\Cis\Concerns;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
trait ReadsEnvironment
|
||||
{
|
||||
private function environment(string $name): ?string
|
||||
{
|
||||
$value = getenv($name);
|
||||
|
||||
if (! is_string($value)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$value = trim($value);
|
||||
|
||||
return $value === '' ? null : $value;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia\Cis;
|
||||
|
||||
use Pest\Plugins\Tia\Cis\Concerns\ReadsEnvironment;
|
||||
use Pest\Plugins\Tia\Contracts\Ci;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GitHub implements Ci
|
||||
{
|
||||
use ReadsEnvironment;
|
||||
|
||||
public function defaultBranch(): ?string
|
||||
{
|
||||
$path = $this->environment('GITHUB_EVENT_PATH');
|
||||
|
||||
if ($path === null || ! is_file($path) || ! is_readable($path)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$contents = @file_get_contents($path);
|
||||
|
||||
if ($contents === false) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$payload = json_decode($contents, true);
|
||||
|
||||
if (! is_array($payload) || ! is_array($payload['repository'] ?? null)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$branch = $payload['repository']['default_branch'] ?? null;
|
||||
|
||||
return is_string($branch) && $branch !== '' ? $branch : null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia\Cis;
|
||||
|
||||
use Pest\Plugins\Tia\Cis\Concerns\ReadsEnvironment;
|
||||
use Pest\Plugins\Tia\Contracts\Ci;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GitLab implements Ci
|
||||
{
|
||||
use ReadsEnvironment;
|
||||
|
||||
public function defaultBranch(): ?string
|
||||
{
|
||||
return $this->environment('CI_DEFAULT_BRANCH');
|
||||
}
|
||||
}
|
||||
@@ -61,12 +61,6 @@ final class Configuration
|
||||
}
|
||||
|
||||
/**
|
||||
* The branch whose baseline every other branch falls back to reading.
|
||||
*
|
||||
* Autodetected from the repository when left unset; declare it here when
|
||||
* the repository cannot answer for itself — no `origin/HEAD`, or an
|
||||
* `init.defaultBranch` that disagrees with reality.
|
||||
*
|
||||
* @return $this
|
||||
*/
|
||||
public function defaultBranch(string $branch): self
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia\Contracts;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
interface Ci
|
||||
{
|
||||
/**
|
||||
* The default branch advertised by this CI, or `null` when the run
|
||||
* does not happen on it — or when it exposes no such information.
|
||||
*/
|
||||
public function defaultBranch(): ?string;
|
||||
}
|
||||
@@ -29,7 +29,9 @@ enum ReplayType
|
||||
$status->isRisky() => self::Risky,
|
||||
$status->isSkipped() => self::Skipped,
|
||||
$status->isIncomplete() => self::Incomplete,
|
||||
default => self::Failure,
|
||||
$status->isNotice(), $status->isDeprecation(), $status->isWarning() => self::Pass,
|
||||
$status->isFailure(), $status->isError() => self::Failure,
|
||||
default => self::None,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
+276
-51
@@ -43,20 +43,12 @@ final class Graph
|
||||
* @var array<string, array{
|
||||
* sha: ?string,
|
||||
* tree: array<string, string>,
|
||||
* complete?: bool,
|
||||
* results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
|
||||
* }>
|
||||
*/
|
||||
private array $baselines = [];
|
||||
|
||||
/**
|
||||
* The baseline a branch with none of its own reads from.
|
||||
*
|
||||
* Only ever read from: a branch writes to its own key, so a fallback that
|
||||
* leaked into the write path would corrupt the baseline every other branch
|
||||
* depends on. Resolved once per run by the plugin — see
|
||||
* {@see self::setFallbackBranch()} — because the git calls it takes are not
|
||||
* free and the read path runs per test.
|
||||
*/
|
||||
private string $fallbackBranch = 'main';
|
||||
|
||||
private readonly string $projectRoot;
|
||||
@@ -127,6 +119,24 @@ final class Graph
|
||||
return array_keys($affectedSet);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $testFiles Project-relative paths.
|
||||
* @return list<string>
|
||||
*/
|
||||
public function testFilesOnDisk(array $testFiles): array
|
||||
{
|
||||
$root = rtrim($this->projectRoot, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR;
|
||||
$onDisk = [];
|
||||
|
||||
foreach ($testFiles as $testFile) {
|
||||
if (is_file($root.$testFile)) {
|
||||
$onDisk[] = $testFile;
|
||||
}
|
||||
}
|
||||
|
||||
return $onDisk;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $changedFiles
|
||||
* @return array{0: list<string>, 1: list<string>}
|
||||
@@ -669,7 +679,7 @@ final class Graph
|
||||
6 => TestStatus::warning($r['message']),
|
||||
7 => TestStatus::failure($r['message']),
|
||||
8 => TestStatus::error($r['message']),
|
||||
default => TestStatus::unknown(),
|
||||
default => null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -696,7 +706,7 @@ final class Graph
|
||||
|
||||
$rel = $this->relative($file);
|
||||
|
||||
if ($rel !== null) {
|
||||
if ($rel !== null && is_file($this->projectRoot.'/'.$rel)) {
|
||||
$files[$rel] = true;
|
||||
}
|
||||
}
|
||||
@@ -704,15 +714,6 @@ final class Graph
|
||||
return array_keys($files);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether any cached result due a re-run points at a test file that is not
|
||||
* on disk — deleted, or never locatable in the first place (`eval()`'d code,
|
||||
* a path outside the project).
|
||||
*
|
||||
* A filtered run cannot honour such an entry: it would select a file that
|
||||
* collects no tests, so the run reports green without ever re-running the
|
||||
* failure — and does so again on every subsequent invocation.
|
||||
*/
|
||||
public function hasUnlocatedTestsToRerun(string $branch, ?string $fallbackBranch = null): bool
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
@@ -728,12 +729,7 @@ final class Graph
|
||||
return true;
|
||||
}
|
||||
|
||||
$rel = $this->relative($file);
|
||||
|
||||
// Results are stored relative, so `relative()` answers "is this
|
||||
// inside the project" without ever touching the filesystem. The
|
||||
// stat is what tells a deleted test file apart from a live one.
|
||||
if ($rel === null || ! is_file($this->projectRoot.'/'.$rel)) {
|
||||
if ($this->relative($file) === null) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -757,6 +753,10 @@ final class Graph
|
||||
return true;
|
||||
}
|
||||
|
||||
if ($testStatus->isUnknown()) {
|
||||
return true;
|
||||
}
|
||||
|
||||
$configuration = Registry::get();
|
||||
|
||||
if ($testStatus->isRisky()) {
|
||||
@@ -830,21 +830,64 @@ final class Graph
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array{sha: ?string, tree: array<string, string>, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
|
||||
* @return array{sha: ?string, tree: array<string, string>, complete?: bool, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
|
||||
*/
|
||||
private function baselineFor(string $branch, ?string $fallbackBranch): array
|
||||
{
|
||||
$fallbackBranch ??= $this->fallbackBranch;
|
||||
|
||||
if (isset($this->baselines[$branch])) {
|
||||
return $this->baselines[$branch];
|
||||
$fallback = $branch !== $fallbackBranch ? ($this->baselines[$fallbackBranch] ?? null) : null;
|
||||
$own = $this->baselines[$branch] ?? null;
|
||||
|
||||
if ($own === null) {
|
||||
return $fallback ?? ['sha' => null, 'tree' => [], 'results' => []];
|
||||
}
|
||||
|
||||
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) {
|
||||
return $this->baselines[$fallbackBranch];
|
||||
if ($fallback === null) {
|
||||
return $own;
|
||||
}
|
||||
|
||||
return ['sha' => null, 'tree' => [], 'results' => []];
|
||||
$under = ($own['complete'] ?? false) === true
|
||||
? $this->withoutFilesCoveredBy($fallback['results'], $own['results'])
|
||||
: $fallback['results'];
|
||||
|
||||
return [
|
||||
'sha' => $own['sha'] ?? $fallback['sha'],
|
||||
'tree' => $own['tree'] !== [] ? $own['tree'] : $fallback['tree'],
|
||||
'results' => array_replace($under, $own['results']),
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}> $results
|
||||
* @param array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}> $authoritative
|
||||
* @return array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
|
||||
*/
|
||||
private function withoutFilesCoveredBy(array $results, array $authoritative): array
|
||||
{
|
||||
$covered = [];
|
||||
|
||||
foreach ($authoritative as $entry) {
|
||||
$file = $entry['file'] ?? null;
|
||||
|
||||
if (is_string($file) && $file !== '') {
|
||||
$covered[$file] = true;
|
||||
}
|
||||
}
|
||||
|
||||
if ($covered === []) {
|
||||
return $results;
|
||||
}
|
||||
|
||||
foreach ($results as $testId => $entry) {
|
||||
$file = $entry['file'] ?? null;
|
||||
|
||||
if (is_string($file) && isset($covered[$file])) {
|
||||
unset($results[$testId]);
|
||||
}
|
||||
}
|
||||
|
||||
return $results;
|
||||
}
|
||||
|
||||
private function ensureBaseline(string $branch): void
|
||||
@@ -856,13 +899,7 @@ final class Graph
|
||||
|
||||
/**
|
||||
* @param array<string, array<int, string>> $testToFiles
|
||||
* @param bool $keepExisting Leave already-recorded edge sets alone. For runs
|
||||
* whose edges are piggybacked off a PHPUnit coverage
|
||||
* session: that data is scoped by `<source>`, so it
|
||||
* can only ever be narrower than what the TIA
|
||||
* recorder sees — it never contains the test's own
|
||||
* file, for one — and a narrower edge set silently
|
||||
* stops selecting the tests it used to select.
|
||||
* @param bool $keepExisting Leave already-recorded edge sets alone.
|
||||
*/
|
||||
public function replaceEdges(array $testToFiles, bool $keepExisting = false): void
|
||||
{
|
||||
@@ -873,8 +910,6 @@ final class Graph
|
||||
continue;
|
||||
}
|
||||
|
||||
// An empty set means "known, covers nothing", so piggyback data is
|
||||
// still an improvement there — only a populated set is protected.
|
||||
if ($keepExisting && ($this->edges[$testRel] ?? []) !== []) {
|
||||
continue;
|
||||
}
|
||||
@@ -1437,6 +1472,56 @@ final class Graph
|
||||
}
|
||||
}
|
||||
|
||||
public function markBaselineComplete(string $branch): void
|
||||
{
|
||||
if (isset($this->baselines[$branch])) {
|
||||
$this->baselines[$branch]['complete'] = true;
|
||||
}
|
||||
}
|
||||
|
||||
public function pruneResultsForMissingFiles(string $branch): void
|
||||
{
|
||||
if (! isset($this->baselines[$branch]['results'])) {
|
||||
return;
|
||||
}
|
||||
|
||||
$root = rtrim($this->projectRoot, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR;
|
||||
|
||||
foreach ($this->baselines[$branch]['results'] as $testId => $result) {
|
||||
$file = $result['file'] ?? null;
|
||||
if (! is_string($file)) {
|
||||
continue;
|
||||
}
|
||||
if ($file === '') {
|
||||
continue;
|
||||
}
|
||||
|
||||
$rel = $this->relative($file);
|
||||
if ($rel === null) {
|
||||
continue;
|
||||
}
|
||||
if (is_file($root.$rel)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
unset($this->baselines[$branch]['results'][$testId]);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $keep
|
||||
*/
|
||||
public function pruneMissingBranches(array $keep): void
|
||||
{
|
||||
$survivors = array_fill_keys($keep, true);
|
||||
|
||||
foreach (array_keys($this->baselines) as $branch) {
|
||||
if (! isset($survivors[$branch])) {
|
||||
unset($this->baselines[$branch]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prune baseline result entries whose test files were just executed but whose
|
||||
* test IDs are no longer present (e.g. the test method was removed or renamed).
|
||||
@@ -1483,13 +1568,6 @@ final class Graph
|
||||
}
|
||||
|
||||
/**
|
||||
* The branches a recorded graph holds baselines for, read straight from the
|
||||
* encoded form.
|
||||
*
|
||||
* Answerable before the graph is hydrated because the default branch has to
|
||||
* be resolved first: the fallback is what every hydrated graph reads its
|
||||
* baselines through.
|
||||
*
|
||||
* @return list<string>
|
||||
*/
|
||||
public static function branchesIn(string $json): array
|
||||
@@ -1521,10 +1599,10 @@ final class Graph
|
||||
|
||||
$graph = new self($projectRoot);
|
||||
$graph->fingerprint = is_array($data['fingerprint'] ?? null) ? $data['fingerprint'] : [];
|
||||
$graph->files = is_array($data['files'] ?? null) ? array_values($data['files']) : [];
|
||||
$graph->files = self::decodeFiles($data['files'] ?? null);
|
||||
$graph->fileIds = array_flip($graph->files);
|
||||
$graph->edges = is_array($data['edges'] ?? null) ? $data['edges'] : [];
|
||||
$graph->baselines = is_array($data['baselines'] ?? null) ? $data['baselines'] : [];
|
||||
$graph->edges = self::decodeEdges($data['edges'] ?? null);
|
||||
$graph->baselines = self::decodeBaselines($data['baselines'] ?? null);
|
||||
|
||||
$graph->testTables = self::decodeStringMap($data['test_tables'] ?? null);
|
||||
$graph->testInertiaComponents = self::decodeStringMap($data['test_inertia_components'] ?? null);
|
||||
@@ -1533,6 +1611,153 @@ final class Graph
|
||||
return $graph;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
private static function decodeFiles(mixed $section): array
|
||||
{
|
||||
if (! is_array($section)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$files = [];
|
||||
|
||||
foreach ($section as $path) {
|
||||
if (is_string($path) && $path !== '') {
|
||||
$files[] = $path;
|
||||
}
|
||||
}
|
||||
|
||||
return $files;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<string, array<int, int>>
|
||||
*/
|
||||
private static function decodeEdges(mixed $section): array
|
||||
{
|
||||
if (! is_array($section)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$edges = [];
|
||||
|
||||
foreach ($section as $key => $ids) {
|
||||
$testFile = (string) $key;
|
||||
|
||||
if ($testFile === '') {
|
||||
continue;
|
||||
}
|
||||
if (! is_array($ids)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$clean = [];
|
||||
|
||||
foreach ($ids as $id) {
|
||||
if (is_int($id)) {
|
||||
$clean[] = $id;
|
||||
}
|
||||
}
|
||||
|
||||
$edges[$testFile] = $clean;
|
||||
}
|
||||
|
||||
return $edges;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<string, array{sha: ?string, tree: array<string, string>, complete?: bool, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}>
|
||||
*/
|
||||
private static function decodeBaselines(mixed $section): array
|
||||
{
|
||||
if (! is_array($section)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$baselines = [];
|
||||
|
||||
foreach ($section as $key => $baseline) {
|
||||
$branch = (string) $key;
|
||||
|
||||
if ($branch === '') {
|
||||
continue;
|
||||
}
|
||||
if (! is_array($baseline)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$sha = $baseline['sha'] ?? null;
|
||||
$tree = [];
|
||||
|
||||
if (is_array($baseline['tree'] ?? null)) {
|
||||
foreach ($baseline['tree'] as $path => $hash) {
|
||||
if (is_string($path) && is_string($hash)) {
|
||||
$tree[$path] = $hash;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$baselines[$branch] = [
|
||||
'sha' => is_string($sha) ? $sha : null,
|
||||
'tree' => $tree,
|
||||
'results' => self::decodeResults($baseline['results'] ?? null),
|
||||
];
|
||||
|
||||
if (($baseline['complete'] ?? null) === true) {
|
||||
$baselines[$branch]['complete'] = true;
|
||||
}
|
||||
}
|
||||
|
||||
return $baselines;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
|
||||
*/
|
||||
private static function decodeResults(mixed $section): array
|
||||
{
|
||||
if (! is_array($section)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$results = [];
|
||||
|
||||
foreach ($section as $key => $entry) {
|
||||
$testId = (string) $key;
|
||||
|
||||
if ($testId === '') {
|
||||
continue;
|
||||
}
|
||||
if (! is_array($entry)) {
|
||||
continue;
|
||||
}
|
||||
if (! is_int($entry['status'] ?? null)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$time = $entry['time'] ?? null;
|
||||
|
||||
$result = [
|
||||
'status' => $entry['status'],
|
||||
'message' => is_string($entry['message'] ?? null) ? $entry['message'] : '',
|
||||
'time' => is_int($time) || is_float($time) ? (float) $time : 0.0,
|
||||
];
|
||||
|
||||
if (is_int($entry['assertions'] ?? null)) {
|
||||
$result['assertions'] = $entry['assertions'];
|
||||
}
|
||||
|
||||
if (is_string($entry['file'] ?? null) && $entry['file'] !== '') {
|
||||
$result['file'] = $entry['file'];
|
||||
}
|
||||
|
||||
$results[$testId] = $result;
|
||||
}
|
||||
|
||||
return $results;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<string, list<string>>
|
||||
*/
|
||||
|
||||
@@ -360,7 +360,9 @@ final class Recorder
|
||||
}
|
||||
|
||||
$lineKeys = array_keys($lines);
|
||||
if ($lineKeys !== [] && count($covered) === 1 && $covered[0] === max($lineKeys)) {
|
||||
$reportsUnexecutedLines = count($covered) < count($lines);
|
||||
|
||||
if ($reportsUnexecutedLines && $lineKeys !== [] && count($covered) === 1 && $covered[0] === max($lineKeys)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
@@ -16,6 +16,9 @@ final class ResultCollector
|
||||
*/
|
||||
private array $results = [];
|
||||
|
||||
/** @var array<string, true> */
|
||||
private array $triggered = [];
|
||||
|
||||
private ?string $currentTestId = null;
|
||||
|
||||
private ?string $currentTestFile = null;
|
||||
@@ -35,9 +38,30 @@ final class ResultCollector
|
||||
return;
|
||||
}
|
||||
|
||||
if (isset($this->triggered[$this->currentTestId])) {
|
||||
$this->refreshTime();
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
$this->record(TestStatus::success());
|
||||
}
|
||||
|
||||
public function testTriggeredNotice(string $message): void
|
||||
{
|
||||
$this->recordIssue(TestStatus::notice($message));
|
||||
}
|
||||
|
||||
public function testTriggeredDeprecation(string $message): void
|
||||
{
|
||||
$this->recordIssue(TestStatus::deprecation($message));
|
||||
}
|
||||
|
||||
public function testTriggeredWarning(string $message): void
|
||||
{
|
||||
$this->recordIssue(TestStatus::warning($message));
|
||||
}
|
||||
|
||||
public function testFailed(string $message): void
|
||||
{
|
||||
if ($this->currentTestId === null) {
|
||||
@@ -91,6 +115,11 @@ final class ResultCollector
|
||||
return $this->results;
|
||||
}
|
||||
|
||||
public function hasUnfinishedTest(): bool
|
||||
{
|
||||
return $this->currentTestId !== null;
|
||||
}
|
||||
|
||||
public function recordAssertions(string $testId, int $assertions): void
|
||||
{
|
||||
if (isset($this->results[$testId])) {
|
||||
@@ -111,6 +140,7 @@ final class ResultCollector
|
||||
public function reset(): void
|
||||
{
|
||||
$this->results = [];
|
||||
$this->triggered = [];
|
||||
$this->currentTestId = null;
|
||||
$this->currentTestFile = null;
|
||||
$this->startTime = null;
|
||||
@@ -123,6 +153,38 @@ final class ResultCollector
|
||||
$this->startTime = null;
|
||||
}
|
||||
|
||||
private function recordIssue(TestStatus $status): void
|
||||
{
|
||||
if ($this->currentTestId === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
$existing = $this->results[$this->currentTestId]['status'] ?? null;
|
||||
|
||||
if (is_int($existing) && $existing >= $status->asInt()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->triggered[$this->currentTestId] = true;
|
||||
|
||||
$this->record($status);
|
||||
}
|
||||
|
||||
private function refreshTime(): void
|
||||
{
|
||||
if ($this->currentTestId === null) {
|
||||
return;
|
||||
}
|
||||
if (! isset($this->results[$this->currentTestId])) {
|
||||
return;
|
||||
}
|
||||
if ($this->startTime === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->results[$this->currentTestId]['time'] = round(microtime(true) - $this->startTime, 3);
|
||||
}
|
||||
|
||||
private function record(TestStatus $status): void
|
||||
{
|
||||
if ($this->currentTestId === null) {
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\DeprecationTriggered;
|
||||
use PHPUnit\Event\Test\DeprecationTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnDeprecationTriggered implements DeprecationTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(DeprecationTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredDeprecation($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\NoticeTriggered;
|
||||
use PHPUnit\Event\Test\NoticeTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnNoticeTriggered implements NoticeTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(NoticeTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredNotice($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\PhpDeprecationTriggered;
|
||||
use PHPUnit\Event\Test\PhpDeprecationTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnPhpDeprecationTriggered implements PhpDeprecationTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(PhpDeprecationTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredDeprecation($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\PhpNoticeTriggered;
|
||||
use PHPUnit\Event\Test\PhpNoticeTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnPhpNoticeTriggered implements PhpNoticeTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(PhpNoticeTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredNotice($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\PhpWarningTriggered;
|
||||
use PHPUnit\Event\Test\PhpWarningTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnPhpWarningTriggered implements PhpWarningTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(PhpWarningTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredWarning($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Subscribers;
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use PHPUnit\Event\Test\WarningTriggered;
|
||||
use PHPUnit\Event\Test\WarningTriggeredSubscriber;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class EnsureTiaResultIsRecordedOnWarningTriggered implements WarningTriggeredSubscriber
|
||||
{
|
||||
public function __construct(private ResultCollector $collector) {}
|
||||
|
||||
public function notify(WarningTriggered $event): void
|
||||
{
|
||||
if ($event->wasSuppressed()) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->collector->testTriggeredWarning($event->message());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Support;
|
||||
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final readonly class Git
|
||||
{
|
||||
private const float TIMEOUT = 5.0;
|
||||
|
||||
public function __construct(
|
||||
private ?string $directory = null,
|
||||
private float $timeout = self::TIMEOUT,
|
||||
) {}
|
||||
|
||||
public function withTimeout(float $timeout): self
|
||||
{
|
||||
return new self($this->directory, $timeout);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function raw(array $arguments): ?string
|
||||
{
|
||||
$result = $this->result($arguments);
|
||||
|
||||
return $result['exitCode'] === 0 ? $result['output'] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function output(array $arguments): ?string
|
||||
{
|
||||
$output = $this->raw($arguments);
|
||||
|
||||
if ($output === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$output = trim($output);
|
||||
|
||||
return $output === '' ? null : $output;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function succeeds(array $arguments): bool
|
||||
{
|
||||
return $this->result($arguments)['exitCode'] === 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
* @return array{exitCode: int, output: string}
|
||||
*/
|
||||
public function result(array $arguments, ?string $input = null): array
|
||||
{
|
||||
$process = new Process(['git', ...$arguments], $this->directory);
|
||||
$process->setTimeout($this->timeout);
|
||||
|
||||
if ($input !== null) {
|
||||
$process->setInput($input);
|
||||
}
|
||||
|
||||
$process->run();
|
||||
|
||||
return [
|
||||
'exitCode' => $process->getExitCode() ?? 1,
|
||||
'output' => $process->getOutput(),
|
||||
];
|
||||
}
|
||||
|
||||
public function isRepository(): bool
|
||||
{
|
||||
return $this->succeeds(['rev-parse', '--git-dir']);
|
||||
}
|
||||
|
||||
public function hasCommits(): bool
|
||||
{
|
||||
return $this->succeeds(['rev-parse', '--verify', '--quiet', 'HEAD']);
|
||||
}
|
||||
|
||||
public function hasRemote(): bool
|
||||
{
|
||||
return $this->output(['remote']) !== null;
|
||||
}
|
||||
|
||||
public function hasRef(string $ref): bool
|
||||
{
|
||||
return $this->output(['rev-parse', '--verify', '--quiet', $ref]) !== null;
|
||||
}
|
||||
|
||||
public function show(string $sha, string $path): ?string
|
||||
{
|
||||
return $this->raw(['show', $sha.':'.$path]);
|
||||
}
|
||||
|
||||
public function subdirectoryPrefix(): ?string
|
||||
{
|
||||
$prefix = $this->output(['rev-parse', '--show-prefix']);
|
||||
|
||||
if ($prefix === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return rtrim(str_replace(DIRECTORY_SEPARATOR, '/', $prefix), '/');
|
||||
}
|
||||
}
|
||||
@@ -8,8 +8,8 @@ use Pest\Contracts\TestCaseFilter;
|
||||
use Pest\Exceptions\MissingDependency;
|
||||
use Pest\Exceptions\NoDirtyTestsFound;
|
||||
use Pest\Panic;
|
||||
use Pest\Support\Git;
|
||||
use Pest\TestSuite;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
final class GitDirtyTestCaseFilter implements TestCaseFilter
|
||||
{
|
||||
@@ -52,14 +52,13 @@ final class GitDirtyTestCaseFilter implements TestCaseFilter
|
||||
*/
|
||||
private function loadChangedFiles(): void
|
||||
{
|
||||
$process = new Process(['git', 'status', '--short', '--', '*.php']);
|
||||
$process->run();
|
||||
$status = new Git(timeout: 60.0)->raw(['status', '--short', '--', '*.php']);
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
if ($status === null) {
|
||||
throw new MissingDependency('Filter by dirty files', 'git');
|
||||
}
|
||||
|
||||
$output = preg_split('/\R+/', $process->getOutput(), flags: PREG_SPLIT_NO_EMPTY);
|
||||
$output = preg_split('/\R+/', $status, flags: PREG_SPLIT_NO_EMPTY);
|
||||
assert(is_array($output));
|
||||
|
||||
$dirtyFiles = [];
|
||||
|
||||
@@ -1568,6 +1568,52 @@
|
||||
PASS Tests\Features\Tia
|
||||
✓ it does not run user hooks when replaying cached skipped and incomplete results
|
||||
|
||||
PASS Tests\Features\Tia\BranchShapes
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "slashes"
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "dots"
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "unicode"
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "digits"
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "underscores"
|
||||
✓ a branch name git allows is a branch key TIA can hold with dataset "very long"
|
||||
✓ a branch differing from the default only in case gets its own key
|
||||
✓ a branch that only lives on the remote keeps its baseline
|
||||
✓ a branch checked out in a worktree keeps its baseline
|
||||
✓ deleting many branches reclaims every one of their baselines with dataset "sequential"
|
||||
✓ deleting many branches reclaims every one of their baselines with dataset "parallel"
|
||||
✓ a narrowed run does not reclaim anything
|
||||
✓ a detached HEAD does not reclaim anything either
|
||||
✓ the default branch baseline survives every branch that comes and goes
|
||||
✓ a project below the git repository root refuses to run and writes nothing with dataset "sequential"
|
||||
✓ a project below the git repository root refuses to run and writes nothing with dataset "parallel"
|
||||
✓ a repository with no commits says so, and leaves plain runs alone
|
||||
✓ a directory with no repository at all still asks for git
|
||||
|
||||
PASS Tests\Features\Tia\CompleteRunWriteTier
|
||||
✓ a complete run prunes a deleted test with dataset "sequential"
|
||||
✓ a complete run prunes a deleted test with dataset "parallel"
|
||||
✓ a complete run records nothing for a test file the graph does not know
|
||||
✓ a partial run records nothing for a test file the graph does not know
|
||||
✓ a truncated run does not prune with dataset "sequential"
|
||||
✓ a truncated run does not prune with dataset "parallel"
|
||||
✓ a green bail run is complete
|
||||
✓ --no-tia refreshes results without enabling tia with dataset "sequential"
|
||||
✓ --no-tia refreshes results without enabling tia with dataset "parallel"
|
||||
✓ a plain run refreshes the results it executed with dataset "sequential"
|
||||
✓ a plain run refreshes the results it executed with dataset "parallel"
|
||||
✓ a parallel replay keeps the recorded time of tests that did not run
|
||||
✓ a run that never enables tia creates no graph with dataset "plain"
|
||||
✓ a run that never enables tia creates no graph with dataset "filtered"
|
||||
✓ a run that never enables tia creates no graph with dataset "parallel filtered"
|
||||
✓ a test edit narrows to the affected file and replays the rest
|
||||
✓ a parallel run merges worker results into the parent baseline
|
||||
|
||||
PASS Tests\Features\Tia\CoveragePiggyback
|
||||
✓ a coverage report does not found a dependency graph with dataset "pest coverage"
|
||||
✓ a coverage report does not found a dependency graph with dataset "phpunit coverage report"
|
||||
✓ a coverage report does not found a dependency graph with dataset "parallel"
|
||||
✓ a plain run after a coverage run records the whole project scope
|
||||
✓ a coverage report leaves the edges of an existing graph alone
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchReplay
|
||||
✓ replays the default branch baseline on a new branch
|
||||
✓ replays whatever the default branch is called with ('main')
|
||||
@@ -1577,12 +1623,23 @@
|
||||
✓ replays on a second new branch too
|
||||
✓ writes nothing on a second run on the same branch
|
||||
✓ replays on a branch whose name contains slashes
|
||||
✓ replays again once back on the default branch
|
||||
✓ replays on a new branch when tia is enabled by configuration
|
||||
✓ a narrowed run on a new branch does not cost the fallback with dataset "sequential"
|
||||
✓ a narrowed run on a new branch does not cost the fallback with dataset "parallel"
|
||||
✓ replays inside a worktree on a new branch
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchResolution
|
||||
✓ a declared default branch beats autodetection
|
||||
✓ a declared default branch that does not exist degrades to a full run
|
||||
✓ a renamed default branch replays and writes under its new name
|
||||
✓ the CI provider names the default branch where the checkout cannot
|
||||
✓ GitLab names the default branch through its own variable
|
||||
✓ a lone recorded baseline names the default branch
|
||||
✓ a default branch nothing can name is refused rather than guessed
|
||||
✓ an init.defaultBranch naming a branch that exists is still trusted
|
||||
✓ a repository with no remote is refused rather than silently re-run
|
||||
✓ a remote-less repository holding one baseline is not refused
|
||||
✓ a declared default branch stands in for a missing remote
|
||||
✓ tia still requires git
|
||||
✓ a plain run outside a repository creates no baseline
|
||||
@@ -1592,10 +1649,145 @@
|
||||
✓ narrows to the affected tests on a new branch
|
||||
✓ filtered mode reads the fallback too
|
||||
✓ filtered mode finds nothing to do on a clean green feature branch
|
||||
✓ filtered mode falls back to a full replay when a cached failure cannot be located
|
||||
✓ filtered mode finds nothing to do on the default branch itself
|
||||
✓ a detached HEAD replays without minting a branch key
|
||||
✓ a detached HEAD does not write into the default branch baseline with dataset "sequential"
|
||||
✓ a detached HEAD does not write into the default branch baseline with dataset "parallel"
|
||||
✓ the branch that ran gets its own key and the default branch keeps its baseline
|
||||
✓ the fallback reaches parallel workers
|
||||
|
||||
PASS Tests\Features\Tia\FilteredMode
|
||||
✓ re-runs a cached failure on a clean tree
|
||||
✓ an explicit path turns filtered mode off
|
||||
✓ filtered mode runs the whole suite when there is no baseline
|
||||
✓ filtered mode finds nothing to do in parallel either
|
||||
✓ a corrupt graph is reported and does not crash the run
|
||||
✓ --parallel --retry is refused and leaves the graph alone
|
||||
|
||||
PASS Tests\Features\Tia\HostileState
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "empty"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "truncated"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "not json"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "json scalar"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "json list"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "json null"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "empty object"
|
||||
✓ a graph mangled beyond use still lets the suite run with dataset "nul bytes"
|
||||
✓ a graph whose shape is wrong everywhere is repaired rather than trusted with dataset "sequential"
|
||||
✓ a graph whose shape is wrong everywhere is repaired rather than trusted with dataset "parallel"
|
||||
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "below the range"
|
||||
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "one past the range"
|
||||
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "far past the range"
|
||||
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "huge"
|
||||
✓ a cached status with no replay of its own does not fail the run with dataset "notice"
|
||||
✓ a cached status with no replay of its own does not fail the run with dataset "deprecation"
|
||||
✓ a cached status with no replay of its own does not fail the run with dataset "warning"
|
||||
✓ a cached skip or todo replays with its message intact with dataset "skipped"
|
||||
✓ a cached skip or todo replays with its message intact with dataset "incomplete"
|
||||
✓ a cached failure with a multi-line message re-runs rather than replaying the text
|
||||
✓ a result pointing outside the project is not addressable and widens the run
|
||||
✓ an edge pointing at a file id that does not exist is ignored
|
||||
✓ a graph from a schema this build does not know is rebuilt, not read
|
||||
✓ graph.json being a directory does not stop the run
|
||||
✓ a state dir it cannot write to still replays
|
||||
|
||||
PASS Tests\Features\Tia\IssueStatuses
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "deprecation"
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "notice"
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "warning"
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "deprecation"
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "notice"
|
||||
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "warning"
|
||||
✓ a cached deprecation still fails the run that asked to fail on one with dataset "sequential"
|
||||
✓ a cached deprecation still fails the run that asked to fail on one with dataset "parallel"
|
||||
✓ replaying a cached issue does not downgrade it to a pass with dataset "sequential"
|
||||
✓ replaying a cached issue does not downgrade it to a pass with dataset "parallel"
|
||||
✓ a failure outranks an issue triggered on the way to it
|
||||
✓ a skip outranks an issue triggered on the way to it
|
||||
✓ a suppressed issue is not recorded
|
||||
|
||||
PASS Tests\Features\Tia\PartialRunWriteTier
|
||||
✓ a filtered run rewrites only the test that ran
|
||||
✓ a filtered run under --tia announces that tia does not apply
|
||||
✓ a test suffix narrows the tier even though every test runs
|
||||
✓ a dirty run narrows to the uncommitted test edit
|
||||
✓ filtered mode yields to an explicit filter
|
||||
✓ an env flag narrows exactly like the option it mirrors with ('PEST_TIA')
|
||||
✓ an env flag narrows exactly like the option it mirrors with ('PEST_TIA_FILTERED')
|
||||
✓ a partial run does not purge the graph even with --fresh
|
||||
✓ --no-tia does not stop a partial run from refreshing its own entry
|
||||
✓ two partial runs each keep the other entry
|
||||
✓ a shard is a partial run
|
||||
✓ a parallel partial run records the test that ran, like a sequential one
|
||||
|
||||
PASS Tests\Features\Tia\RemoteBaseline
|
||||
✓ a published baseline is fetched instead of recorded locally
|
||||
✓ a fetched baseline that will not decode is discarded rather than trusted
|
||||
✓ a fetched baseline recorded against another tree is not used
|
||||
✓ an artifact without a graph in it fails loudly
|
||||
✓ a baseline that cannot be authenticated for fails loudly
|
||||
✓ a workflow or artifact that is not there fails loudly
|
||||
✓ a network failure warns and lets the suite run with dataset "querying the runs"
|
||||
✓ a network failure warns and lets the suite run with dataset "downloading the artifact"
|
||||
✓ no published baseline yet starts a cooldown, and a corrupt cooldown does not break the run
|
||||
|
||||
PASS Tests\Features\Tia\SelectionPaths
|
||||
✓ a committed rename selects the tests that depended on the old path with dataset "sequential"
|
||||
✓ a committed rename selects the tests that depended on the old path with dataset "parallel"
|
||||
✓ an affected test file that is gone does not strand a filtered run with dataset "sequential"
|
||||
✓ an affected test file that is gone does not strand a filtered run with dataset "parallel"
|
||||
✓ a plain run reclaims the edge of a test file that is gone
|
||||
✓ a changed view selects the test that rendered it
|
||||
✓ a changed partial selects the test that rendered its ancestor with dataset "direct @include"
|
||||
✓ a changed partial selects the test that rendered its ancestor with dataset "transitive @include"
|
||||
✓ a changed partial selects the test that rendered its ancestor with dataset "x- component"
|
||||
✓ a changed partial selects the test that rendered its ancestor with dataset "include cycle"
|
||||
✓ a changed Inertia page selects the test that rendered its component
|
||||
✓ a changed shared JS module selects the tests of the pages that import it
|
||||
✓ a changed frontend runtime file selects every Inertia test
|
||||
|
||||
PASS Tests\Features\Tia\StateReclamation
|
||||
✓ a detached HEAD does not purge the graph on structural drift with dataset "sequential"
|
||||
✓ a detached HEAD does not purge the graph on structural drift with dataset "parallel"
|
||||
✓ a detached HEAD does not purge the graph with --fresh either with dataset "sequential"
|
||||
✓ a detached HEAD does not purge the graph with --fresh either with dataset "parallel"
|
||||
✓ a detached HEAD leaves an unreadable graph for a checkout that can rebuild it
|
||||
✓ a cached failure whose test file was deleted stops widening later runs with dataset "sequential"
|
||||
✓ a cached failure whose test file was deleted stops widening later runs with dataset "parallel"
|
||||
✓ a complete run reclaims the entry and the edge of a deleted test file with dataset "sequential"
|
||||
✓ a complete run reclaims the entry and the edge of a deleted test file with dataset "parallel"
|
||||
✓ a pruned result does not come back from the fallback with dataset "sequential"
|
||||
✓ a pruned result does not come back from the fallback with dataset "parallel"
|
||||
✓ the fallback still reaches a branch that has never run a test file
|
||||
✓ a branch that git no longer knows loses its baseline
|
||||
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "unknown"
|
||||
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "future"
|
||||
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "garbage"
|
||||
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "notice"
|
||||
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "deprecation"
|
||||
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "warning"
|
||||
✓ a malformed baseline entry cannot break the run with dataset "sequential"
|
||||
✓ a malformed baseline entry cannot break the run with dataset "parallel"
|
||||
✓ a run torn down mid-file does not prune the tests it never reached with dataset "sequential"
|
||||
✓ a run torn down mid-file does not prune the tests it never reached with dataset "parallel"
|
||||
✓ a fatal error mid-file is a test error, not a truncation with dataset "sequential"
|
||||
✓ a fatal error mid-file is a test error, not a truncation with dataset "parallel"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "bail"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "stop-on-failure"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "compact"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel bail"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel one process"
|
||||
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel more processes than files"
|
||||
✓ --tia --no-tia is a plain run that still refreshes what it executed with dataset "sequential"
|
||||
✓ --tia --no-tia is a plain run that still refreshes what it executed with dataset "parallel"
|
||||
✓ --fresh on a partial run neither purges nor prunes with dataset "sequential"
|
||||
✓ --fresh on a partial run neither purges nor prunes with dataset "parallel"
|
||||
✓ a second green run on a feature branch writes nothing at all with dataset "sequential"
|
||||
✓ a second green run on a feature branch writes nothing at all with dataset "parallel"
|
||||
✓ a graph whose recorded commit is gone is re-anchored, not warned about forever with dataset "sequential"
|
||||
✓ a graph whose recorded commit is gone is re-anchored, not warned about forever with dataset "parallel"
|
||||
|
||||
PASS Tests\Features\Ticket
|
||||
✓ it may be associated with an ticket #1, #2
|
||||
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
|
||||
@@ -2225,4 +2417,4 @@
|
||||
✓ pass with dataset with ('my-datas-set-value')
|
||||
✓ within describe → pass with dataset with ('my-datas-set-value')
|
||||
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1571 passed (3446 assertions)
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1743 passed (3953 assertions)
|
||||
@@ -49,7 +49,7 @@ it('does not run user hooks when replaying cached skipped and incomplete results
|
||||
expect($storage->write(Tia::KEY_GRAPH, (string) $json))->toBeTrue();
|
||||
|
||||
$process = new Process(
|
||||
['php', 'bin/pest', $fixture, '--tia'],
|
||||
['php', 'bin/pest', '--configuration', 'tests/Fixtures/Suites/TiaReplayHooks.xml', '--tia'],
|
||||
$projectRoot,
|
||||
[
|
||||
'COLLISION_PRINTER' => 'DefaultPrinter',
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a branch name git allows is a branch key TIA can hold', function (string $branch): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo($branch, new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', $branch])
|
||||
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary());
|
||||
})->with([
|
||||
'slashes' => 'feature/deep/nesting',
|
||||
'dots' => 'release.1.2.x',
|
||||
'unicode' => 'feature-café-日本',
|
||||
'digits' => '12345',
|
||||
'underscores' => 'feature_x_y',
|
||||
'very long' => 'feature-'.str_repeat('x', 180),
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a branch differing from the default only in case gets its own key', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('MASTER-2', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'MASTER-2']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a branch that only lives on the remote keeps its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('remote-only', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['update-ref', 'refs/remotes/origin/remote-only', 'HEAD']);
|
||||
$project->git()->run(['branch', '-D', 'remote-only']);
|
||||
|
||||
$project->pest('--tia');
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'remote-only']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a branch checked out in a worktree keeps its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('wt-branch', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['worktree', 'add', '--quiet', $project->path().'-wt', 'wt-branch']);
|
||||
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->run(['worktree', 'remove', '--force', $project->path().'-wt']);
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'wt-branch']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('deleting many branches reclaims every one of their baselines', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
foreach (range(1, 4) as $index) {
|
||||
$project->git()->switchTo('feature-'.$index, new: true);
|
||||
$project->pest('--tia', ...$arguments);
|
||||
}
|
||||
|
||||
expect($project->branchKeys())->toHaveCount(5);
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
|
||||
foreach (range(1, 4) as $index) {
|
||||
$project->git()->run(['branch', '-D', 'feature-'.$index]);
|
||||
}
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($project->branchKeys())->toBe(['master']);
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a narrowed run does not reclaim anything', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['branch', '-D', 'feature-x']);
|
||||
|
||||
$project->snapshot();
|
||||
$project->pest('--tia', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'feature-x'])
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a detached HEAD does not reclaim anything either', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['branch', '-D', 'feature-x']);
|
||||
$project->git()->detach();
|
||||
|
||||
$project->snapshot();
|
||||
$project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'feature-x'])
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the default branch baseline survives every branch that comes and goes', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['branch', '-D', 'feature-x']);
|
||||
|
||||
$project->snapshot();
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->added())->toBe(0, $delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($project->graph()['baselines']['master']['results'])->toHaveCount(Project::TOTAL_TESTS);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a project below the git repository root refuses to run and writes nothing', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$nested = $project->nested();
|
||||
|
||||
$result = $project->pestIn($nested, '--tia', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->output)->toContain('Tia mode requires the git repository root')
|
||||
->and($project->path('.home/.pest'))->not->toBeDirectory()
|
||||
->and($nested.DIRECTORY_SEPARATOR.'.pest')->not->toBeDirectory();
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a repository with no commits says so, and leaves plain runs alone', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
$project->git()->run(['init', '--quiet']);
|
||||
$project->git()->run(['checkout', '--quiet', '-b', 'master']);
|
||||
$project->git()->addOrigin();
|
||||
|
||||
$tia = $project->pest('--tia');
|
||||
|
||||
expect($tia->exitCode)->toBe(1, $tia->describe())
|
||||
->and($tia->output)->toContain('Tia mode requires at least one commit')
|
||||
->and($tia->output)->not->toContain('requires [git]')
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
|
||||
$plain = $project->pest();
|
||||
|
||||
expect($plain->exitCode)->toBe(0, $plain->describe())
|
||||
->and($plain->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a directory with no repository at all still asks for git', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->output)->toContain('requires "git"')
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,234 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a complete run prunes a deleted test', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest(...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->tally())->toContain('5 passed')
|
||||
->and($delta->removed())->toBe(1, $delta->summary())
|
||||
->and($delta->added())->toBe(0, $delta->summary())
|
||||
->and($delta->structureMoved())->toBeFalse($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a complete run records nothing for a test file the graph does not know', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/BrandNewTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
test('brand new thing', function (): void {
|
||||
expect(true)->toBeTrue();
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest();
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->tally())->toContain('7 passed')
|
||||
->and($delta->added())->toBe(0, $delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a partial run records nothing for a test file the graph does not know', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/BrandNewTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
test('brand new thing', function (): void {
|
||||
expect(true)->toBeTrue();
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--filter=brand new thing');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->tally())->toContain('1 passed')
|
||||
->and($delta->added())->toBe(0, $delta->summary())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a truncated run does not prune', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Goodbye, Nuno!');
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--bail', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->tally())->toContain('1 failed')
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->structureMoved())->toBeFalse($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a green bail run is complete', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--bail');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('--no-tia refreshes results without enabling tia', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--no-tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->not->toContain('Experimental TIA mode enabled')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a plain run refreshes the results it executed', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest(...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a parallel replay keeps the recorded time of tests that did not run', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->pest('--tia', '--parallel', '--processes=2');
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a run that never enables tia creates no graph', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$result = $project->pest(...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->with([
|
||||
'plain' => [[]],
|
||||
'filtered' => [['--filter=adds two numbers']],
|
||||
'parallel filtered' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a test edit narrows to the affected file and replays the rest', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
expect((new Greeter)->greet('Nuno'))->toBeString();
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(2, $delta->summary())
|
||||
->and($delta->edgesMoved())->toBeFalse($delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a parallel run merges worker results into the parent baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
expect((new Greeter)->greet('Nuno'))->toBeString();
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia', '--parallel', '--processes=2');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(2, $delta->summary())
|
||||
->and($delta->edgesMoved())->toBeFalse($delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,53 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a coverage report does not found a dependency graph', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($project->graphExists())->toBeFalse();
|
||||
})->with([
|
||||
'pest coverage' => [['--coverage']],
|
||||
'phpunit coverage report' => [['--coverage-text']],
|
||||
'parallel' => [['--coverage', '--parallel', '--processes=2']],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a plain run after a coverage run records the whole project scope', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->pest('--tia', '--coverage');
|
||||
$project->pest('--tia');
|
||||
|
||||
$graph = $project->graph();
|
||||
|
||||
if ($graph === null) {
|
||||
expect($project->graphExists())->toBeFalse();
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
expect(array_keys($graph['edges']))->toEqualCanonicalizing(array_keys(Project::EDGES))
|
||||
->and($graph['files'])->toContain('tests/Unit/CalculatorTest.php')
|
||||
->and($graph['files'])->toContain('app/Calculator.php');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a coverage report leaves the edges of an existing graph alone', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->pest('--tia', '--coverage');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($delta->edgesMoved())->toBeFalse($delta->summary())
|
||||
->and($delta->filesMoved())->toBeFalse($delta->summary())
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->added())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
@@ -4,15 +4,6 @@ declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* Reading a baseline recorded on another branch.
|
||||
*
|
||||
* Without the default-branch fallback, the first `--tia` run on every new branch
|
||||
* re-runs a whole suite whose results the default branch already holds — once
|
||||
* per branch, forever, on any repository not named `main`.
|
||||
*
|
||||
* @see https://github.com/pestphp/pest/issues/1823
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
@@ -54,8 +45,6 @@ test('replays on a second new branch too', function (): void {
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The toll is one full run per new branch. It must not come back for the
|
||||
// second branch either.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -89,16 +78,58 @@ test('replays on a branch whose name contains slashes', function (): void {
|
||||
->and($project->branchKeys())->toContain('feature/x/y');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays again once back on the default branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
|
||||
$project->snapshot();
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays on a new branch when tia is enabled by configuration', function (): void {
|
||||
$project = Project::make('master', overlay: 'always-enabled');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'feature-x']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a narrowed run on a new branch does not cost the fallback', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest(...$arguments);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->with([
|
||||
'sequential' => [['--filter=adds two numbers']],
|
||||
'parallel' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('replays inside a worktree on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$worktree = $project->worktree('feature-worktree');
|
||||
|
||||
// Seeded against the worktree rather than the main checkout: a worktree's
|
||||
// `.git` is a file, so `Storage::originIdentity()` cannot read the remote
|
||||
// from it and the worktree resolves a storage key of its own. That gap is
|
||||
// separate from the branch fallback, and it is the fallback this row is
|
||||
// about — the worktree is checked out on a branch the baseline does not
|
||||
// name, which is the scenario from the issue.
|
||||
$project->seedFor($worktree, 'master');
|
||||
|
||||
$result = $project->pestIn($worktree, '--tia');
|
||||
|
||||
@@ -5,17 +5,11 @@ declare(strict_types=1);
|
||||
use Symfony\Component\Process\ExecutableFinder;
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* How the default branch gets named: declared in `tests/Pest.php`, autodetected
|
||||
* from the repository, or not answerable at all.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a declared default branch beats autodetection', function (): void {
|
||||
// The repository autodetects `develop`, which holds no baseline. Only the
|
||||
// declaration in `tests/Pest.php` can reach the `master` one.
|
||||
$project = Project::make('develop', overlay: 'configured-default-branch');
|
||||
$project->seed('master');
|
||||
|
||||
@@ -35,20 +29,32 @@ test('a declared default branch that does not exist degrades to a full run', fun
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing to fall back to, so everything runs — and no baseline is minted
|
||||
// under the name that resolved to nothing.
|
||||
expect($result->uncached())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'feature-x']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a renamed default branch replays and writes under its new name', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->rename('master', 'main');
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'main'])
|
||||
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the CI provider names the default branch where the checkout cannot', function (): void {
|
||||
// A CI checkout: `actions/checkout` fetches a single ref instead of cloning,
|
||||
// so there is no `origin/HEAD` for git to read the default branch from. The
|
||||
// event payload GitHub hands the job says it outright.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->seed('master');
|
||||
$project->addBaseline('legacy');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
@@ -68,6 +74,7 @@ test('GitLab names the default branch through its own variable', function (): vo
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->seed('master');
|
||||
$project->addBaseline('legacy');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
@@ -80,10 +87,6 @@ test('GitLab names the default branch through its own variable', function (): vo
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a lone recorded baseline names the default branch', function (): void {
|
||||
// Nothing left to ask: no `origin/HEAD`, no CI provider, and an
|
||||
// `init.defaultBranch` that names a branch this repository does not have.
|
||||
// The graph holds exactly one baseline, and it is the only one any branch
|
||||
// could read — so it is the answer.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'main');
|
||||
@@ -98,9 +101,6 @@ test('a lone recorded baseline names the default branch', function (): void {
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a default branch nothing can name is refused rather than guessed', function (): void {
|
||||
// Same checkout as above, without the graph that answered it. Guessing here
|
||||
// is what made this bug expensive: the guess reads no baseline at all, and
|
||||
// the output calls that a hit.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'main');
|
||||
@@ -116,9 +116,6 @@ test('a default branch nothing can name is refused rather than guessed', functio
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an init.defaultBranch naming a branch that exists is still trusted', function (): void {
|
||||
// The setting is the machine's, not the repository's — worth taking only
|
||||
// where the repository has a branch by that name. It does here, and with no
|
||||
// graph on disk it is the only source left, so the run must not be refused.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'master');
|
||||
@@ -129,7 +126,7 @@ test('an init.defaultBranch naming a branch that exists is still trusted', funct
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->not->toContain('could not determine the default branch')
|
||||
->and($project->branchKeys())->toBe(['feature-x']);
|
||||
->and($project->branchKeys())->not->toContain('master');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a repository with no remote is refused rather than silently re-run', function (): void {
|
||||
@@ -141,16 +138,11 @@ test('a repository with no remote is refused rather than silently re-run', funct
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing can name the default branch, so every new branch would re-run the
|
||||
// whole suite with no explanation. Saying so beats doing that quietly. The
|
||||
// missing remote is the likeliest reason and gets named as such.
|
||||
expect($result->output)->toContain('Tia mode requires a repository with a remote.')
|
||||
->and($result->exitCode)->toBe(1, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a remote-less repository holding one baseline is not refused', function (): void {
|
||||
// The refusal above exists to stop a guess, not to demand a remote for its
|
||||
// own sake. With a baseline on disk there is nothing left to guess at.
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
@@ -165,8 +157,6 @@ test('a remote-less repository holding one baseline is not refused', function ()
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a declared default branch stands in for a missing remote', function (): void {
|
||||
// The escape hatch the refusal above points at: with the branch named by
|
||||
// hand there is nothing left for a remote to answer.
|
||||
$project = Project::make('master', overlay: 'configured-default-branch');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
@@ -185,9 +175,7 @@ test('tia still requires git', function (): void {
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The soft default-branch resolver runs before the branch is named, and it
|
||||
// must not swallow this.
|
||||
expect($result->output)->toContain('The feature "Tia mode" requires "git".')
|
||||
expect($result->output)->toContain('The feature [Tia mode[ requires [git].')
|
||||
->and($result->exitCode)->not->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
@@ -212,8 +200,6 @@ test('the default branch is resolved once per run, not once per test', function
|
||||
|
||||
expect($git)->not->toBeNull();
|
||||
|
||||
// A `git` first on `PATH` that records what it was asked before handing over
|
||||
// to the real one.
|
||||
$log = $project->path('git-calls.log');
|
||||
$project->write('shim/git', implode("\n", [
|
||||
'#!/bin/sh',
|
||||
|
||||
@@ -4,12 +4,6 @@ declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* What the fallback is allowed to touch.
|
||||
*
|
||||
* Reading another branch's baseline must stay a read: writes belong to the
|
||||
* branch that ran, and a branch that has no name of its own must not mint one.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
@@ -20,58 +14,43 @@ test('narrows to the affected tests on a new branch', function (): void {
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
// Semantic, not cosmetic: PHP is hashed at the AST level, so a comment
|
||||
// would not register as a change at all.
|
||||
$project->write('app/Calculator.php', <<<'PHP'
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
final class Calculator
|
||||
{
|
||||
public function add(int $a, int $b): int
|
||||
{
|
||||
return $a + $b;
|
||||
}
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
expect((new Greeter)->greet('Nuno'))->toBeString();
|
||||
});
|
||||
|
||||
public function subtract(int $a, int $b): int
|
||||
{
|
||||
return $a - $b;
|
||||
}
|
||||
|
||||
public function multiply(int $a, int $b): int
|
||||
{
|
||||
return $a * $b;
|
||||
}
|
||||
}
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
// Two of the three test files cover `Calculator`; the third replays from the
|
||||
// default branch's baseline.
|
||||
expect($result->affected())->toBe(4, $result->describe())
|
||||
->and($result->replayed())->toBe(2, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe());
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'feature-x'])
|
||||
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
|
||||
->and($delta->edgesMoved())->toBeFalse($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode reads the fallback too', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
// A failure cached on the default branch. Filtered mode asks the graph which
|
||||
// test files are due a re-run, and that read has to reach the fallback as
|
||||
// well: without it a new branch sees nothing to do, reports green, and never
|
||||
// re-runs the failure — on every subsequent invocation.
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
// Selection is by file, so the failure's two siblings in `CalculatorTest`
|
||||
// come along; the other two test files stay out of the run entirely.
|
||||
expect($result->output)->toContain('from 1 previously unsuccessful test')
|
||||
->and($result->output)->not->toContain('No affected tests found')
|
||||
->and($result->tally())->toContain('2 passed');
|
||||
@@ -91,6 +70,40 @@ test('filtered mode finds nothing to do on a clean green feature branch', functi
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode falls back to a full replay when a cached failure cannot be located', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$testId = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
$graph['baselines']['master']['results'][$testId]['file'] = '/build/agent/tests/Unit/DeletedTest.php';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
expect($result->output)->toContain('Some cached tests due a re-run could not be located on disk.')
|
||||
->and($result->output)->toContain('Running the full suite with replay instead of a filtered run.')
|
||||
->and($result->output)->not->toContain('No affected tests found');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode finds nothing to do on the default branch itself', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('No affected tests found')
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a detached HEAD replays without minting a branch key', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
@@ -99,13 +112,27 @@ test('a detached HEAD replays without minting a branch key', function (): void {
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// A detached HEAD has no branch of its own. The default branch is the
|
||||
// honest key, and no phantom one appears beside it.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a detached HEAD does not write into the default branch baseline', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
$project->pest(...$arguments);
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
|
||||
->and($project->branchKeys())->toBe(['master']);
|
||||
})->with([
|
||||
'sequential' => [['--filter=adds two numbers']],
|
||||
'parallel' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('the branch that ran gets its own key and the default branch keeps its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('re-runs a cached failure on a clean tree', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('from 1 previously unsuccessful test')
|
||||
->and($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->tally())->toContain('2 passed')
|
||||
->and($delta->writtenCount())->toBe(2, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an explicit path turns filtered mode off', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered', 'tests/Unit');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($result->output)->not->toContain('No affected tests found')
|
||||
->and($result->tally())->toContain('4 passed')
|
||||
->and($delta->writtenCount())->toBe(4, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode runs the whole suite when there is no baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->output)->not->toContain('No affected tests found');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode finds nothing to do in parallel either', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered', '--parallel', '--processes=2');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('No affected tests found')
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a corrupt graph is reported and does not crash the run', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$graph = $project->graphDir().'/graph.json';
|
||||
|
||||
file_put_contents($graph, '{not json');
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->output)->toContain('The dependency graph could not be read')
|
||||
->and(is_file($graph) ? file_get_contents($graph) : null)->not->toBe('{not json');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('--parallel --retry is refused and leaves the graph alone', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--parallel', '--retry');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->not->toBe(0)
|
||||
->and($result->output)->toContain('--retry')
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,221 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a graph mangled beyond use still lets the suite run', function (string $contents): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
file_put_contents($project->graphDir().'/graph.json', $contents);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->with([
|
||||
'empty' => '',
|
||||
'truncated' => '{"schema":1,"files":["app/Calculator.php"],"edg',
|
||||
'not json' => '{not json',
|
||||
'json scalar' => '"just a string"',
|
||||
'json list' => '[1,2,3]',
|
||||
'json null' => 'null',
|
||||
'empty object' => '{}',
|
||||
'nul bytes' => "\0\0\0\0",
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a graph whose shape is wrong everywhere is repaired rather than trusted', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
$graph['baselines']['master']['results'][$id] = 'nope';
|
||||
$graph['baselines']['master']['results'][7] = ['status' => 0, 'message' => '', 'time' => 0.1];
|
||||
$graph['baselines']['master']['tree'] = 'nope';
|
||||
$graph['baselines']['master']['sha'] = 42;
|
||||
$graph['baselines'][''] = ['sha' => null, 'tree' => [], 'results' => []];
|
||||
$graph['baselines']['broken'] = 'nope';
|
||||
$graph['edges']['tests/Unit/GreeterTest.php'] = 'nope';
|
||||
$graph['edges'][''] = [0];
|
||||
$graph['files'][] = ['nested'];
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->not->toContain('TypeError')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($project->branchKeys())->toBe(['master']);
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a cached status this build cannot interpret is re-run, not replayed', function (int $status): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($status): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = $status;
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->uncached())->toBe(1, $result->describe());
|
||||
})->with([
|
||||
'below the range' => -1,
|
||||
'one past the range' => 9,
|
||||
'far past the range' => 99,
|
||||
'huge' => PHP_INT_MAX,
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a cached status with no replay of its own does not fail the run', function (int $status): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($status): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = $status;
|
||||
$graph['baselines']['master']['results'][$id]['message'] = 'cached detail';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->with(['notice' => 3, 'deprecation' => 4, 'warning' => 6])->skipOnWindows();
|
||||
|
||||
test('a cached skip or todo replays with its message intact', function (int $status, string $tally): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($status): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = $status;
|
||||
$graph['baselines']['master']['results'][$id]['message'] = 'a recorded reason';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain($tally)
|
||||
->and($result->output)->toContain('a recorded reason')
|
||||
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
|
||||
})->with([
|
||||
'skipped' => [1, '1 skipped'],
|
||||
'incomplete' => [2, '1 incomplete'],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a cached failure with a multi-line message re-runs rather than replaying the text', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = 7;
|
||||
$graph['baselines']['master']['results'][$id]['message'] = "line one\nline two\nline three";
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->uncached())->toBe(1, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a result pointing outside the project is not addressable and widens the run', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['file'] = '/build/agent/tests/Unit/CalculatorTest.php';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->toContain('could not be located on disk')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an edge pointing at a file id that does not exist is ignored', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$graph['edges']['tests/Unit/CalculatorTest.php'] = [0, 999, -5];
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a graph from a schema this build does not know is rebuilt, not read', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$graph['schema'] = 2;
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->replayed())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('graph.json being a directory does not stop the run', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
unlink($project->graphDir().'/graph.json');
|
||||
mkdir($project->graphDir().'/graph.json');
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a state dir it cannot write to still replays', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
chmod($project->graphDir(), 0500);
|
||||
|
||||
try {
|
||||
$result = $project->pest('--tia');
|
||||
} finally {
|
||||
chmod($project->graphDir(), 0700);
|
||||
}
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,136 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
function tiaTriggering(string $call): string
|
||||
{
|
||||
return <<<PHP
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
test('adds two numbers', function (): void {
|
||||
{$call}
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
expect((new Calculator)->subtract(3, 1))->toBe(2);
|
||||
});
|
||||
PHP;
|
||||
}
|
||||
|
||||
test('a triggered issue is recorded as itself, not as a pass', function (array $arguments, string $call, int $status): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering($call));
|
||||
$project->git()->commit('trigger an issue');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe($status);
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->with([
|
||||
'deprecation' => ["trigger_error('legacy adder', E_USER_DEPRECATED);", 4],
|
||||
'notice' => ["trigger_error('a notice', E_USER_NOTICE);", 3],
|
||||
'warning' => ["trigger_error('a warning', E_USER_WARNING);", 6],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a cached deprecation still fails the run that asked to fail on one', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"));
|
||||
$project->git()->commit('trigger a deprecation');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
$result = $project->pest('--tia', '--fail-on-deprecation', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->tally())->toContain('1 deprecated')
|
||||
->and($result->uncached())->toBe(1, $result->describe());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('replaying a cached issue does not downgrade it to a pass', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"));
|
||||
$project->git()->commit('trigger a deprecation');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
foreach (range(1, 3) as $ignored) {
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(4);
|
||||
}
|
||||
|
||||
$result = $project->pest('--tia', '--fail-on-deprecation', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a failure outranks an issue triggered on the way to it', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', str_replace(
|
||||
'toBe(3)',
|
||||
'toBe(999)',
|
||||
tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"),
|
||||
));
|
||||
$project->git()->commit('an issue then a failure');
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
expect($result->exitCode)->not->toBe(0)
|
||||
->and($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(7);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a skip outranks an issue triggered on the way to it', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', str_replace(
|
||||
'expect((new Calculator)->add(1, 2))->toBe(3);',
|
||||
"\$this->markTestSkipped('not today');",
|
||||
tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"),
|
||||
));
|
||||
$project->git()->commit('an issue then a skip');
|
||||
|
||||
$project->pest('--tia');
|
||||
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(1);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a suppressed issue is not recorded', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("@trigger_error('quiet', E_USER_DEPRECATED);"));
|
||||
$project->git()->commit('a suppressed deprecation');
|
||||
|
||||
$project->pest('--tia');
|
||||
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
|
||||
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(0);
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,170 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a filtered run rewrites only the test that ran', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->tally())->toContain('1 passed')
|
||||
->and($result->output)->not->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a filtered run under --tia announces that tia does not apply', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a test suffix narrows the tier even though every test runs', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--test-suffix=Test.php');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a dirty run narrows to the uncommitted test edit', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
|
||||
test('greets again', function (): void {
|
||||
expect((new Greeter)->greet('again'))->toBe('Hello, again!');
|
||||
});
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia', '--dirty');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($result->tally())->toContain('3 passed')
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->structureMoved())->toBeFalse($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode yields to an explicit filter', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($result->tally())->toContain('1 passed')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an env flag narrows exactly like the option it mirrors', function (string $variable): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), [
|
||||
$variable => '1',
|
||||
], '--filter=adds two numbers');
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->with(['PEST_TIA', 'PEST_TIA_FILTERED'])->skipOnWindows();
|
||||
|
||||
test('a partial run does not purge the graph even with --fresh', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--fresh', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('--no-tia does not stop a partial run from refreshing its own entry', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--no-tia', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->not->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('two partial runs each keep the other entry', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->pest('--filter=adds two numbers');
|
||||
$project->pest('--filter=greets a person');
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($delta->writtenCount())->toBe(2, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a shard is a partial run', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--shard=1/2');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a parallel partial run records the test that ran, like a sequential one', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--parallel', '--processes=2', '--filter=adds two numbers');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('TIA does not apply to partial runs')
|
||||
->and($result->tally())->toContain('1 passed')
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,141 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
/**
|
||||
* @param callable(array<string, mixed>): array<string, mixed>|null $mutator
|
||||
* @return array{0: Project, 1: array<string, string>}
|
||||
*/
|
||||
function tiaPublishedBaseline(string $mode = 'ok', ?callable $mutator = null): array
|
||||
{
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$payload = $project->detachGraph();
|
||||
|
||||
if ($mutator !== null) {
|
||||
/** @var array<string, mixed> $decoded */
|
||||
$decoded = json_decode($payload, true);
|
||||
$payload = (string) json_encode($mutator($decoded), JSON_UNESCAPED_SLASHES);
|
||||
}
|
||||
|
||||
return [$project, $project->gh($mode, $payload)];
|
||||
}
|
||||
|
||||
test('a published baseline is fetched instead of recorded locally', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline();
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->toContain('Downloading TIA baseline')
|
||||
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($project->graphExists())->toBeTrue();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a fetched baseline that will not decode is discarded rather than trusted', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('corrupt');
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->toContain('The dependency graph could not be read')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->replayed())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a fetched baseline recorded against another tree is not used', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('ok', function (array $graph): array {
|
||||
$graph['fingerprint']['structural']['composer_lock'] = 'a-lockfile-this-project-never-had';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->replayed())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an artifact without a graph in it fails loudly', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('missing-asset');
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->output)->toContain('the artifact is missing expected files')
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a baseline that cannot be authenticated for fails loudly', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('unauthenticated');
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->output)->toContain('is not authenticated')
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a workflow or artifact that is not there fails loudly', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('list-404');
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($result->output)->toContain('not found in repo')
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a network failure warns and lets the suite run', function (string $mode): void {
|
||||
[$project, $environment] = tiaPublishedBaseline($mode);
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->toContain('network error')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->with([
|
||||
'querying the runs' => ['list-network'],
|
||||
'downloading the artifact' => ['download-network'],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('no published baseline yet starts a cooldown, and a corrupt cooldown does not break the run', function (): void {
|
||||
[$project, $environment] = tiaPublishedBaseline('no-runs');
|
||||
|
||||
$discardGraph = function () use ($project): void {
|
||||
if ($project->graphExists()) {
|
||||
$project->detachGraph();
|
||||
}
|
||||
};
|
||||
|
||||
$first = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($first->exitCode)->toBe(0, $first->describe())
|
||||
->and($first->output)->toContain('No baseline published yet')
|
||||
->and($project->graphDir().DIRECTORY_SEPARATOR.'fetch-cooldown.json')->toBeFile();
|
||||
|
||||
$discardGraph();
|
||||
|
||||
$second = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($second->exitCode)->toBe(0, $second->describe())
|
||||
->and($second->output)->toContain('next auto-retry in');
|
||||
|
||||
file_put_contents($project->graphDir().DIRECTORY_SEPARATOR.'fetch-cooldown.json', 'not json{');
|
||||
|
||||
$discardGraph();
|
||||
|
||||
$third = $project->pestWithEnvironment($project->path(), $environment, '--tia', '--baselined');
|
||||
|
||||
expect($third->exitCode)->toBe(0, $third->describe())
|
||||
->and($third->output)->toContain('No baseline published yet')
|
||||
->and($third->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,218 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
function tiaSeedWithView(Project $project, string $view): void
|
||||
{
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($view): array {
|
||||
$id = count($graph['files']);
|
||||
$graph['files'][$id] = $view;
|
||||
$graph['edges']['tests/Unit/GreeterTest.php'][] = $id;
|
||||
|
||||
return $graph;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $components
|
||||
* @param array<string, array<int, string>> $jsFileToComponents
|
||||
*/
|
||||
function tiaSeedWithInertia(Project $project, array $components, array $jsFileToComponents = []): void
|
||||
{
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($components, $jsFileToComponents): array {
|
||||
$graph['test_inertia_components'] = ['tests/Unit/GreeterTest.php' => $components];
|
||||
$graph['js_file_to_components'] = $jsFileToComponents;
|
||||
|
||||
return $graph;
|
||||
});
|
||||
}
|
||||
|
||||
test('a committed rename selects the tests that depended on the old path', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/views/greeting.blade.php', "<p>Hello</p>\n");
|
||||
$project->git()->commit('add view');
|
||||
|
||||
tiaSeedWithView($project, 'resources/views/greeting.blade.php');
|
||||
|
||||
$project->git()->run(['mv', 'resources/views/greeting.blade.php', 'resources/views/hello.blade.php']);
|
||||
$project->git()->commit('move the view');
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('an affected test file that is gone does not strand a filtered run', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/views/page.blade.php', "<p>one</p>\n");
|
||||
$project->git()->commit('add view');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = count($graph['files']);
|
||||
$graph['files'][$id] = 'resources/views/page.blade.php';
|
||||
$graph['edges']['tests/Unit/GhostTest.php'] = [$id];
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$project->write('resources/views/page.blade.php', "<p>two</p>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia', '--filtered', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->toContain('No affected tests found')
|
||||
->and($result->output)->not->toContain('tests/Unit/GhostTest.php')
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a plain run reclaims the edge of a test file that is gone', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/views/page.blade.php', "<p>one</p>\n");
|
||||
$project->git()->commit('add view');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = count($graph['files']);
|
||||
$graph['files'][$id] = 'resources/views/page.blade.php';
|
||||
$graph['edges']['tests/Unit/GhostTest.php'] = [$id];
|
||||
$graph['baselines']['master']['results']['P\\Tests\\Unit\\GhostTest::ghostly'] = [
|
||||
'status' => 0,
|
||||
'message' => '',
|
||||
'time' => 9.999,
|
||||
'assertions' => 42,
|
||||
'file' => 'tests/Unit/GhostTest.php',
|
||||
];
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$project->write('resources/views/page.blade.php', "<p>two</p>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->removed())->toBe(1, $delta->summary())
|
||||
->and($project->graph()['edges'])->not->toHaveKey('tests/Unit/GhostTest.php');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a changed view selects the test that rendered it', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/views/page.blade.php', "<p>one</p>\n");
|
||||
$project->git()->commit('add view');
|
||||
|
||||
tiaSeedWithView($project, 'resources/views/page.blade.php');
|
||||
|
||||
$project->write('resources/views/page.blade.php', "<p>two</p>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a changed partial selects the test that rendered its ancestor', function (array $views, string $changed): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
foreach ($views as $path => $contents) {
|
||||
$project->write($path, $contents);
|
||||
}
|
||||
|
||||
$project->git()->commit('add views');
|
||||
|
||||
tiaSeedWithView($project, 'resources/views/page.blade.php');
|
||||
|
||||
$project->write($changed, $views[$changed]."<span>edited</span>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->with([
|
||||
'direct @include' => [[
|
||||
'resources/views/page.blade.php' => "@include('partials.nav')\n",
|
||||
'resources/views/partials/nav.blade.php' => "<nav>one</nav>\n",
|
||||
], 'resources/views/partials/nav.blade.php'],
|
||||
'transitive @include' => [[
|
||||
'resources/views/page.blade.php' => "@include('partials.wrapper')\n",
|
||||
'resources/views/partials/wrapper.blade.php' => "@include('partials.nav')\n",
|
||||
'resources/views/partials/nav.blade.php' => "<nav>one</nav>\n",
|
||||
], 'resources/views/partials/nav.blade.php'],
|
||||
'x- component' => [[
|
||||
'resources/views/page.blade.php' => "<x-card>hi</x-card>\n",
|
||||
'resources/views/components/card.blade.php' => "<div>one</div>\n",
|
||||
], 'resources/views/components/card.blade.php'],
|
||||
'include cycle' => [[
|
||||
'resources/views/page.blade.php' => "@include('partials.a')\n",
|
||||
'resources/views/partials/a.blade.php' => "@include('partials.b')\n",
|
||||
'resources/views/partials/b.blade.php' => "@include('partials.a')\n",
|
||||
], 'resources/views/partials/b.blade.php'],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('a changed Inertia page selects the test that rendered its component', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/js/Pages/Foo.vue', "<template>one</template>\n");
|
||||
$project->git()->commit('add page');
|
||||
|
||||
tiaSeedWithInertia($project, ['Foo']);
|
||||
|
||||
$project->write('resources/js/Pages/Foo.vue', "<template>two</template>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a changed shared JS module selects the tests of the pages that import it', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/js/Pages/Foo.vue', "<template>one</template>\n");
|
||||
$project->write('resources/js/Shared/Nav.vue', "<template>nav</template>\n");
|
||||
$project->git()->commit('add pages');
|
||||
|
||||
tiaSeedWithInertia($project, ['Foo'], ['resources/js/Shared/Nav.vue' => ['Foo']]);
|
||||
|
||||
$project->write('resources/js/Shared/Nav.vue', "<template>nav two</template>\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a changed frontend runtime file selects every Inertia test', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->write('resources/js/app.js', "console.log(1)\n");
|
||||
$project->git()->commit('add runtime');
|
||||
|
||||
tiaSeedWithInertia($project, ['Foo']);
|
||||
|
||||
$project->write('resources/js/app.js', "console.log(2)\n");
|
||||
$project->snapshot();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->affected())->toBe(2, $result->describe())
|
||||
->and($result->replayed())->toBe(4, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,360 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a detached HEAD does not purge the graph on structural drift', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
|
||||
$project->write('composer.lock', (string) json_encode([
|
||||
'content-hash' => 'drifted',
|
||||
'packages' => [],
|
||||
'packages-dev' => [],
|
||||
]));
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->graphExists())->toBeTrue('the detached run deleted graph.json')
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a detached HEAD does not purge the graph with --fresh either', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
|
||||
$result = $project->pest('--tia', '--fresh', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->graphExists())->toBeTrue('the detached --fresh run deleted graph.json')
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a detached HEAD leaves an unreadable graph for a checkout that can rebuild it', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
|
||||
file_put_contents($project->graphDir().'/graph.json', '{not json');
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and(file_get_contents($project->graphDir().'/graph.json'))->toBe('{not json');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a cached failure whose test file was deleted stops widening later runs', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
unlink($project->path('tests/Unit/CalculatorTest.php'));
|
||||
$project->git()->commit('drop CalculatorTest');
|
||||
|
||||
$project->pest('--tia', '--filtered', ...$arguments);
|
||||
|
||||
$project->snapshot();
|
||||
$second = $project->pest('--tia', '--filtered', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($second->output)->not->toContain('could not be located on disk')
|
||||
->and($second->output)->toContain('No affected tests found')
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a complete run reclaims the entry and the edge of a deleted test file', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
unlink($project->path('tests/Unit/CalculatorTest.php'));
|
||||
$project->git()->commit('drop CalculatorTest');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
$graph = $project->graph();
|
||||
|
||||
expect($graph['edges'] ?? [])->not->toHaveKey('tests/Unit/CalculatorTest.php')
|
||||
->and(array_column($graph['baselines']['master']['results'] ?? [], 'file'))
|
||||
->not->toContain('tests/Unit/CalculatorTest.php');
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a pruned result does not come back from the fallback', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
test('adds two numbers, renamed', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
expect((new Calculator)->subtract(3, 1))->toBe(2);
|
||||
});
|
||||
PHP);
|
||||
|
||||
$project->git()->commit('rename the test on the branch');
|
||||
|
||||
$project->pest('--tia', '--filtered', ...$arguments);
|
||||
|
||||
$second = $project->pest('--tia', '--filtered', ...$arguments);
|
||||
|
||||
expect($second->output)->not->toContain('previously unsuccessful')
|
||||
->and($second->output)->toContain('No affected tests found');
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('the fallback still reaches a branch that has never run a test file', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$project->pest('--filter=greets a person');
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
expect($result->output)->toContain('previously unsuccessful')
|
||||
->and($result->affected())->toBe(2, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a branch that git no longer knows loses its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'feature-x']);
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->run(['branch', '-D', 'feature-x']);
|
||||
|
||||
$project->pest('--tia');
|
||||
|
||||
expect($project->branchKeys())->toBe(['master']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an unknown cached status is re-run rather than replayed as a failure', function (int $status): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($status): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = $status;
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($result->uncached())->toBe(1, $result->describe());
|
||||
})->with(['unknown' => -1, 'future' => 9, 'garbage' => 99])->skipOnWindows();
|
||||
|
||||
test('a cached notice, deprecation or warning does not replay as a failure', function (int $status): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph) use ($status): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id]['status'] = $status;
|
||||
$graph['baselines']['master']['results'][$id]['message'] = 'cached detail';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->with(['notice' => 3, 'deprecation' => 4, 'warning' => 6])->skipOnWindows();
|
||||
|
||||
test('a malformed baseline entry cannot break the run', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->mutateGraph(function (array $graph): array {
|
||||
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
|
||||
$graph['baselines']['master']['results'][$id] = 'nope';
|
||||
$graph['baselines']['master']['tree'] = 'nope';
|
||||
$graph['edges']['tests/Unit/GreeterTest.php'] = 'nope';
|
||||
|
||||
return $graph;
|
||||
});
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->not->toContain('TypeError')
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a run torn down mid-file does not prune the tests it never reached', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
test('adds two numbers', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
exit(0);
|
||||
});
|
||||
PHP);
|
||||
|
||||
$project->git()->commit('a test that kills its own process');
|
||||
|
||||
$project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->shaMoved())->toBeFalse($delta->summary())
|
||||
->and($delta->structureMoved())->toBeFalse($delta->summary())
|
||||
->and(array_keys($project->graph()['baselines']['master']['results']))
|
||||
->toContain(Project::testId('tests/Unit/CalculatorTest.php', 'subtracts two numbers'));
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a fatal error mid-file is a test error, not a truncation', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
test('adds two numbers', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
undefined_function_here();
|
||||
});
|
||||
PHP);
|
||||
|
||||
$project->git()->commit('a test that fatals');
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->not->toBe(0)
|
||||
->and($result->tally())->toContain('1 failed')
|
||||
->and($delta->removed())->toBe(0, $delta->summary())
|
||||
->and($delta->structureMoved())->toBeFalse($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a green complete run leaves the graph exactly as it found it', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with([
|
||||
'bail' => [['--bail']],
|
||||
'stop-on-failure' => [['--stop-on-failure']],
|
||||
'compact' => [['--compact']],
|
||||
'parallel bail' => [['--parallel', '--processes=2', '--bail']],
|
||||
'parallel one process' => [['--parallel', '--processes=1']],
|
||||
'parallel more processes than files' => [['--parallel', '--processes=8']],
|
||||
])->skipOnWindows();
|
||||
|
||||
test('--tia --no-tia is a plain run that still refreshes what it executed', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--no-tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('--fresh on a partial run neither purges nor prunes', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$result = $project->pest('--tia', '--fresh', '--filter=adds two numbers', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->graphExists())->toBeTrue()
|
||||
->and($delta->writtenCount())->toBe(1, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a second green run on a feature branch writes nothing at all', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia', ...$arguments);
|
||||
|
||||
$project->snapshot();
|
||||
$result = $project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
|
||||
test('a graph whose recorded commit is gone is re-anchored, not warned about forever', function (array $arguments): void {
|
||||
$project = Project::make('master');
|
||||
$project->git()->commit('second');
|
||||
$project->seed('master');
|
||||
|
||||
$recordedSha = $project->graph()['baselines']['master']['sha'];
|
||||
|
||||
$project->git()->run(['reset', '--quiet', '--hard', 'HEAD~1']);
|
||||
$project->snapshot();
|
||||
|
||||
$first = $project->pest('--tia', ...$arguments);
|
||||
|
||||
expect($first->exitCode)->toBe(0, $first->describe())
|
||||
->and($first->output)->toContain('no longer reachable')
|
||||
->and($first->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($project->graph()['baselines']['master']['sha'])->not->toBe($recordedSha)
|
||||
->and($project->graph()['baselines']['master']['sha'])->toBe($project->git()->sha());
|
||||
|
||||
$project->snapshot();
|
||||
$second = $project->pest('--tia', ...$arguments);
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($second->output)->not->toContain('no longer reachable')
|
||||
->and($second->replayed())->toBe(Project::TOTAL_TESTS, $second->describe())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary());
|
||||
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
|
||||
@@ -0,0 +1,17 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
Runs TiaReplayHooks.php as a suite of its own, so `tests/Features/Tia.php` can
|
||||
replay it without narrowing the run to a path. TIA declines to replay a
|
||||
narrowed run, and a single-file invocation is exactly that.
|
||||
-->
|
||||
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
|
||||
bootstrap="../../../vendor/autoload.php"
|
||||
colors="true"
|
||||
>
|
||||
<testsuites>
|
||||
<testsuite name="default">
|
||||
<file>TiaReplayHooks.php</file>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
</phpunit>
|
||||
@@ -4,19 +4,10 @@ declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A git repository the scenario tests drive.
|
||||
*
|
||||
* Hermetic by construction: every invocation neutralises the machine's global
|
||||
* and system config and carries its own committer identity. Without that, an
|
||||
* ambient `init.defaultBranch = main` would answer questions the scenario means
|
||||
* to leave unanswered, and rows like "resolves to nothing, degrades safely"
|
||||
* would pass by accident.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GitRepo
|
||||
@@ -27,9 +18,6 @@ final readonly class GitRepo
|
||||
public const array ENV = [
|
||||
'GIT_CONFIG_GLOBAL' => '/dev/null',
|
||||
'GIT_CONFIG_SYSTEM' => '/dev/null',
|
||||
// `GIT_CONFIG_SYSTEM` does not cover every system-level file git reads:
|
||||
// Apple's git also loads one from inside Xcode, and it sets
|
||||
// `init.defaultBranch`. Only this suppresses all of them.
|
||||
'GIT_CONFIG_NOSYSTEM' => '1',
|
||||
'GIT_AUTHOR_NAME' => 'Pest Fixture',
|
||||
'GIT_AUTHOR_EMAIL' => 'fixture@pestphp.io',
|
||||
@@ -39,9 +27,6 @@ final readonly class GitRepo
|
||||
|
||||
public function __construct(public string $path) {}
|
||||
|
||||
/**
|
||||
* Initialises the repository on `$branch` and commits everything in it.
|
||||
*/
|
||||
public function init(string $branch = 'master'): void
|
||||
{
|
||||
$this->run(['init', '--quiet']);
|
||||
@@ -70,12 +55,6 @@ final readonly class GitRepo
|
||||
$this->run(['checkout', '--quiet', '--detach']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers an `origin`, which also decides the graph's storage key:
|
||||
* {@see Storage::projectKey()} prefers the remote's
|
||||
* identity over the path, so two checkouts of one repository — a worktree,
|
||||
* say — share a single graph.
|
||||
*/
|
||||
public function addOrigin(string $url = 'git@github.com:pestphp/tia-fixture.git'): void
|
||||
{
|
||||
$this->run(['remote', 'add', 'origin', $url]);
|
||||
@@ -86,22 +65,12 @@ final readonly class GitRepo
|
||||
$this->run(['remote', 'remove', 'origin']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Points `refs/remotes/origin/HEAD` at a local branch — what
|
||||
* `git remote set-head` would write, without a remote to talk to.
|
||||
*/
|
||||
public function setOriginHead(string $branch): void
|
||||
{
|
||||
$this->run(['update-ref', 'refs/remotes/origin/'.$branch, 'HEAD']);
|
||||
$this->run(['symbolic-ref', 'refs/remotes/origin/HEAD', 'refs/remotes/origin/'.$branch]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops `refs/remotes/origin/HEAD` while keeping the remote-tracking branch
|
||||
* — what a CI checkout looks like. `actions/checkout` builds its working
|
||||
* copy with `git init` plus a single-ref `fetch` rather than a `clone`, and
|
||||
* only a `clone` writes that symbolic ref.
|
||||
*/
|
||||
public function unsetOriginHead(): void
|
||||
{
|
||||
$this->run(['symbolic-ref', '--delete', 'refs/remotes/origin/HEAD']);
|
||||
@@ -112,9 +81,6 @@ final readonly class GitRepo
|
||||
$this->run(['config', '--local', $key, $value]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch and returns its path.
|
||||
*/
|
||||
public function worktree(string $path, string $branch): string
|
||||
{
|
||||
$this->run(['worktree', 'add', '--quiet', '-b', $branch, $path]);
|
||||
@@ -164,7 +130,7 @@ final readonly class GitRepo
|
||||
private function process(array $arguments, bool $mustSucceed): Process
|
||||
{
|
||||
$process = new Process(['git', ...$arguments], $this->path, self::ENV);
|
||||
$process->setTimeout(30.0);
|
||||
$process->setTimeout(120.0);
|
||||
$process->run();
|
||||
|
||||
if ($mustSucceed && ! $process->isSuccessful()) {
|
||||
|
||||
@@ -5,20 +5,6 @@ declare(strict_types=1);
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* What one run did to the graph.
|
||||
*
|
||||
* The three tiers a run may respect, from the conformance matrix:
|
||||
*
|
||||
* - COMPLETE — may change everything.
|
||||
* - RESULTS-ONLY — may change only `baselines[<branch>].results` for tests that
|
||||
* actually ran. It may never remove an entry, nor move `sha`, `tree`,
|
||||
* `edges`, `files` or `fingerprint`.
|
||||
* - HARD-SUPPRESSED — may change nothing at all.
|
||||
*
|
||||
* {@see self::writtenCount()} is the load-bearing measurement, and the reason
|
||||
* {@see Project::sentinel()} exists: without falsified cached values there is no
|
||||
* way to tell "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GraphDelta
|
||||
@@ -42,9 +28,6 @@ final readonly class GraphDelta
|
||||
return $this->before !== null && $this->after === null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries whose stored values actually moved.
|
||||
*/
|
||||
public function writtenCount(): int
|
||||
{
|
||||
$written = 0;
|
||||
@@ -71,9 +54,6 @@ final readonly class GraphDelta
|
||||
return $written;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that appeared.
|
||||
*/
|
||||
public function added(): int
|
||||
{
|
||||
$added = 0;
|
||||
@@ -88,9 +68,6 @@ final readonly class GraphDelta
|
||||
return $added;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that were pruned.
|
||||
*/
|
||||
public function removed(): int
|
||||
{
|
||||
$removed = 0;
|
||||
@@ -106,14 +83,11 @@ final readonly class GraphDelta
|
||||
}
|
||||
|
||||
/**
|
||||
* The baseline keys after the run — the headline signal for the
|
||||
* default-branch rows, where a phantom key is the defect.
|
||||
*
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeys(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->after));
|
||||
return array_map(strval(...), array_keys($this->baselines($this->after)));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -121,7 +95,7 @@ final readonly class GraphDelta
|
||||
*/
|
||||
public function branchKeysBefore(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->before));
|
||||
return array_map(strval(...), array_keys($this->baselines($this->before)));
|
||||
}
|
||||
|
||||
public function branchKeysMoved(): bool
|
||||
@@ -129,10 +103,6 @@ final readonly class GraphDelta
|
||||
return $this->branchKeysBefore() !== $this->branchKeys();
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the named branch's baseline is byte-identical — how a row proves
|
||||
* the fallback is read-only.
|
||||
*/
|
||||
public function baselineUntouched(string $branch): bool
|
||||
{
|
||||
return ($this->baselines($this->before)[$branch] ?? null)
|
||||
@@ -149,11 +119,6 @@ final readonly class GraphDelta
|
||||
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'tree', $this->before) !== $this->baselineField($branch, 'tree', $this->after));
|
||||
}
|
||||
|
||||
/**
|
||||
* Compares edges by the file paths they resolve to, not by file id: ids are
|
||||
* an implementation detail that shifts whenever `files` is rebuilt in a
|
||||
* different order.
|
||||
*/
|
||||
public function edgesMoved(): bool
|
||||
{
|
||||
return $this->edgeSets($this->before) !== $this->edgeSets($this->after);
|
||||
@@ -184,17 +149,11 @@ final readonly class GraphDelta
|
||||
return $this->branchKeysMoved();
|
||||
}
|
||||
|
||||
/**
|
||||
* Nothing moved at all.
|
||||
*/
|
||||
public function isHardSuppressed(): bool
|
||||
{
|
||||
return $this->before === $this->after;
|
||||
}
|
||||
|
||||
/**
|
||||
* Results may have moved for tests that ran; nothing structural did.
|
||||
*/
|
||||
public function isResultsOnly(): bool
|
||||
{
|
||||
return ! $this->graphWasCreated()
|
||||
@@ -206,9 +165,6 @@ final readonly class GraphDelta
|
||||
&& $this->added() === 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* A one-line verdict, for failure messages.
|
||||
*/
|
||||
public function summary(): string
|
||||
{
|
||||
if ($this->graphWasCreated()) {
|
||||
@@ -298,9 +254,7 @@ final readonly class GraphDelta
|
||||
$baselines = [];
|
||||
|
||||
foreach ($this->section($graph, 'baselines') as $branch => $baseline) {
|
||||
if (is_string($branch)) {
|
||||
$baselines[$branch] = $baseline;
|
||||
}
|
||||
$baselines[(string) $branch] = $baseline;
|
||||
}
|
||||
|
||||
return $baselines;
|
||||
|
||||
@@ -5,15 +5,10 @@ declare(strict_types=1);
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* The outcome of one `pest` invocation against a fixture project.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class PestResult
|
||||
{
|
||||
/**
|
||||
* The run's output, with the terminal's escape sequences taken back out.
|
||||
*/
|
||||
public string $output;
|
||||
|
||||
/**
|
||||
@@ -25,39 +20,26 @@ final readonly class PestResult
|
||||
public int $exitCode,
|
||||
) {
|
||||
$this->output = (string) preg_replace([
|
||||
'#\x1b[[][^A-Za-z]*[A-Za-z]#', // colours, cursor moves
|
||||
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#', // hyperlinks
|
||||
'#\x1b[[][^A-Za-z]*[A-Za-z]#',
|
||||
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#',
|
||||
], '', $output);
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests whose cached result was replayed instead of executed.
|
||||
*/
|
||||
public function replayed(): int
|
||||
{
|
||||
return $this->recapFragment('replayed');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because the graph held nothing for them — the count that
|
||||
* betrays a fallback which never resolved.
|
||||
*/
|
||||
public function uncached(): int
|
||||
{
|
||||
return $this->recapFragment('uncached');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because a file they depend on changed.
|
||||
*/
|
||||
public function affected(): int
|
||||
{
|
||||
return $this->recapFragment('affected');
|
||||
}
|
||||
|
||||
/**
|
||||
* The `Tests:` summary line, without its label or leading whitespace.
|
||||
*/
|
||||
public function tally(): string
|
||||
{
|
||||
if (preg_match('/^\s*Tests:\s+(.+)$/m', $this->output, $matches) !== 1) {
|
||||
@@ -72,10 +54,6 @@ final readonly class PestResult
|
||||
return str_contains($this->output, $needle);
|
||||
}
|
||||
|
||||
/**
|
||||
* A description of the run, for failure messages that would otherwise say
|
||||
* only that 0 !== 6.
|
||||
*/
|
||||
public function describe(): string
|
||||
{
|
||||
return sprintf(
|
||||
@@ -86,11 +64,6 @@ final readonly class PestResult
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read off the `Tests:` line rather than the whole output: the TIA headline
|
||||
* counts affected *files*, and matching that instead would be a quietly
|
||||
* wrong number.
|
||||
*/
|
||||
private function recapFragment(string $label): int
|
||||
{
|
||||
if (preg_match('/(\d+) '.preg_quote($label, '/').'/', $this->tally(), $matches) !== 1) {
|
||||
|
||||
+87
-166
@@ -5,7 +5,6 @@ declare(strict_types=1);
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use FilesystemIterator;
|
||||
use Pest\Factories\TestCaseFactory;
|
||||
use Pest\Plugins\Tia;
|
||||
use Pest\Plugins\Tia\ChangedFiles;
|
||||
use Pest\Plugins\Tia\FileState;
|
||||
@@ -19,28 +18,11 @@ use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A throwaway Pest project the TIA scenario tests drive.
|
||||
*
|
||||
* Two facts about Pest shape everything here:
|
||||
*
|
||||
* 1. `bin/pest` derives the project root from **the autoloader it finds**, not
|
||||
* from the working directory — `dirname($autoloadPath, 2)`. So the project
|
||||
* owns a real `vendor/autoload.php` and a real copy of `bin/pest` at the path
|
||||
* a composer install would have put them. A symlinked `vendor` would resolve
|
||||
* `__DIR__` straight back to the Pest repository, and every scenario would
|
||||
* silently measure the wrong project.
|
||||
* 2. TIA cannot *record* without pcov or Xdebug, and CI has neither. So a
|
||||
* scenario never records: {@see self::seed()} writes the graph a recording
|
||||
* run would have written, and the run under test exercises the read path.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class Project
|
||||
{
|
||||
/**
|
||||
* Test file → the source files a recording run would have linked it to. The
|
||||
* self-edge every test file gets is added on top of these.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array EDGES = [
|
||||
@@ -50,8 +32,6 @@ final class Project
|
||||
];
|
||||
|
||||
/**
|
||||
* Test file → the descriptions it declares, in declaration order.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array TESTS = [
|
||||
@@ -60,15 +40,17 @@ final class Project
|
||||
'tests/Feature/CoversCalculatorTest.php' => ['adds within a feature test', 'subtracts within a feature test'],
|
||||
];
|
||||
|
||||
/**
|
||||
* Every test in the fixture suite.
|
||||
*/
|
||||
public const int TOTAL_TESTS = 6;
|
||||
|
||||
/**
|
||||
* Every project scaffolded so far, so a row cannot leak one by failing
|
||||
* before its own cleanup.
|
||||
*
|
||||
* @var array<string, array<int, array<int, string>>>
|
||||
*/
|
||||
public const array SEQUENTIAL_AND_PARALLEL = [
|
||||
'sequential' => [[]],
|
||||
'parallel' => [['--parallel', '--processes=2']],
|
||||
];
|
||||
|
||||
/**
|
||||
* @var array<int, self>
|
||||
*/
|
||||
private static array $created = [];
|
||||
@@ -85,13 +67,6 @@ final class Project
|
||||
*/
|
||||
private array $extraPaths = [];
|
||||
|
||||
/**
|
||||
* The root whose graph this project reads and writes.
|
||||
*
|
||||
* Its own, except where a row seeds a worktree: a worktree's `.git` is a
|
||||
* file rather than a directory, so {@see Storage} cannot read the remote
|
||||
* from it and resolves a storage key of its own.
|
||||
*/
|
||||
private string $graphRoot;
|
||||
|
||||
private function __construct(public readonly string $path)
|
||||
@@ -99,14 +74,6 @@ final class Project
|
||||
$this->graphRoot = $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scaffolds a project whose default branch is `$branch`, and hands it back
|
||||
* checked out there.
|
||||
*
|
||||
* The repository is realistic on purpose: it has an `origin`, and an
|
||||
* `origin/HEAD` naming `$branch`, which is what a checkout of a real project
|
||||
* looks like and what the default branch is autodetected from.
|
||||
*/
|
||||
public static function make(string $branch = 'master', ?string $overlay = null): self
|
||||
{
|
||||
$project = self::scaffold($overlay);
|
||||
@@ -118,9 +85,6 @@ final class Project
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Destroys every project scaffolded so far. Belongs in an `afterEach`.
|
||||
*/
|
||||
public static function destroyAll(): void
|
||||
{
|
||||
while (self::$created !== []) {
|
||||
@@ -128,14 +92,6 @@ final class Project
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A project that is not a git repository at all — for the rows that assert
|
||||
* TIA still demands git, and that a plain run does not care.
|
||||
*
|
||||
* Lives in the system temp directory, so it is outside any enclosing
|
||||
* repository, and owns a real `vendor` rather than a symlinked one, so its
|
||||
* baseline key cannot collide with another fixture's.
|
||||
*/
|
||||
public static function withoutGit(?string $overlay = null): self
|
||||
{
|
||||
return self::scaffold($overlay);
|
||||
@@ -151,12 +107,6 @@ final class Project
|
||||
return $relative === '' ? $this->path : $this->path.DIRECTORY_SEPARATOR.$relative;
|
||||
}
|
||||
|
||||
/**
|
||||
* Overwrites a file in the project.
|
||||
*
|
||||
* Edits must be semantic: TIA hashes PHP at the AST level, so a
|
||||
* comment-only change is not a change at all.
|
||||
*/
|
||||
public function write(string $relative, string $contents): void
|
||||
{
|
||||
$path = $this->path($relative);
|
||||
@@ -171,14 +121,20 @@ final class Project
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch, scaffolded so `pest` can run in it, and
|
||||
* returns its path.
|
||||
*
|
||||
* The graph is shared with the main checkout, which is the whole point: both
|
||||
* resolve the same storage key, because {@see Storage} prefers the `origin`
|
||||
* identity over the path.
|
||||
*/
|
||||
public function nested(string $directory = 'nested'): string
|
||||
{
|
||||
$path = $this->path($directory);
|
||||
|
||||
if (! is_dir($path) && ! @mkdir($path, 0755, true) && ! is_dir($path)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
|
||||
}
|
||||
|
||||
self::copy(__DIR__.'/app', $path);
|
||||
$this->scaffoldVendor($path);
|
||||
|
||||
return $path;
|
||||
}
|
||||
|
||||
public function worktree(string $branch): string
|
||||
{
|
||||
$path = $this->path.'-worktree-'.preg_replace('/[^a-z0-9]+/i', '-', $branch);
|
||||
@@ -190,18 +146,11 @@ final class Project
|
||||
return $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in the project and returns what happened.
|
||||
*/
|
||||
public function pest(string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestIn($this->path, ...$arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in `$directory` — a worktree, say — against this project's
|
||||
* graph.
|
||||
*/
|
||||
public function pestIn(string $directory, string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestWithEnvironment($directory, [], ...$arguments);
|
||||
@@ -221,12 +170,8 @@ final class Project
|
||||
'COLLISION_IGNORE_DURATION' => 'true',
|
||||
'PARATEST' => '0',
|
||||
'PAO_DISABLE' => '1',
|
||||
'XDEBUG_MODE' => 'coverage',
|
||||
'HOME' => $this->home(),
|
||||
// Blanked for the same reason `GitRepo::ENV` blanks git's own
|
||||
// config: the default branch a CI provider reports is the one
|
||||
// *its* build is for. Pest's suite runs on GitHub Actions, so
|
||||
// without this every scenario would autodetect Pest's default
|
||||
// branch instead of the fixture's.
|
||||
'GITHUB_EVENT_PATH' => '',
|
||||
'CI_DEFAULT_BRANCH' => '',
|
||||
...$environment,
|
||||
@@ -244,17 +189,6 @@ final class Project
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the graph a clean, green recording run on `$branch` would have
|
||||
* written, and remembers it as the snapshot the next {@see self::delta()}
|
||||
* compares against.
|
||||
*
|
||||
* Sentinelled by default: a row almost always wants to know which entries
|
||||
* were written, and only a real recording run's values can answer that.
|
||||
*
|
||||
* `$failing` names descriptions to record as failures — a clean green run
|
||||
* can never cache one, so a row that needs a cached failure to re-run has to
|
||||
* be handed it.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seed(string $branch, bool $sentinel = true, array $failing = []): void
|
||||
@@ -263,9 +197,6 @@ final class Project
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeds the graph belonging to `$root` — a worktree, say, which resolves a
|
||||
* storage key of its own.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seedFor(string $root, string $branch, bool $sentinel = true, array $failing = []): void
|
||||
@@ -279,8 +210,6 @@ final class Project
|
||||
$graph->setFingerprint(Fingerprint::compute($root));
|
||||
$graph->setRecordedAtSha($branch, $sha);
|
||||
|
||||
// Hashes the tree as it stands, so the run under test sees nothing as
|
||||
// changed — the same call the recording path makes.
|
||||
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
|
||||
|
||||
$graph->markKnownTestFiles(array_keys(self::EDGES));
|
||||
@@ -322,13 +251,40 @@ final class Project
|
||||
$sentinel ? $this->sentinel() : $this->snapshot();
|
||||
}
|
||||
|
||||
public function detachGraph(): string
|
||||
{
|
||||
$json = $this->state()->read(Tia::KEY_GRAPH);
|
||||
|
||||
if ($json === null) {
|
||||
throw new RuntimeException('There is no graph to detach.');
|
||||
}
|
||||
|
||||
if (! $this->state()->delete(Tia::KEY_GRAPH)) {
|
||||
throw new RuntimeException('Unable to remove the detached graph.');
|
||||
}
|
||||
|
||||
$this->snapshot();
|
||||
|
||||
return $json;
|
||||
}
|
||||
|
||||
/**
|
||||
* The id PHPUnit reports for a test in the fixture suite.
|
||||
*
|
||||
* Mirrors how Pest names generated test classes
|
||||
* ({@see TestCaseFactory}): a wrong id here shows up as
|
||||
* `0 replayed`, which every scenario asserts against.
|
||||
* @return array<string, string>
|
||||
*/
|
||||
public function gh(string $mode = 'ok', string $payload = '{}'): array
|
||||
{
|
||||
self::mirror(__DIR__.'/stubs/gh', $this->path('stub/gh'));
|
||||
chmod($this->path('stub/gh'), 0755);
|
||||
|
||||
$this->write('payload/graph.json', $payload);
|
||||
|
||||
return [
|
||||
'PATH' => $this->path('stub').PATH_SEPARATOR.getenv('PATH'),
|
||||
'GH_STUB_MODE' => $mode,
|
||||
'GH_STUB_PAYLOAD' => $this->path('payload/graph.json'),
|
||||
];
|
||||
}
|
||||
|
||||
public static function testId(string $testFile, string $description): string
|
||||
{
|
||||
$basename = basename($testFile, '.php');
|
||||
@@ -343,41 +299,49 @@ final class Project
|
||||
return 'P\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relative).'::'.Str::evaluable($description);
|
||||
}
|
||||
|
||||
/**
|
||||
* Falsifies every cached value, so the next {@see self::delta()} can tell
|
||||
* "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* `assertions` is only ever falsified where it is already non-zero: risky,
|
||||
* skipped and incomplete statuses are *derived* from "performed no
|
||||
* assertions", so patching those would rewrite the status on replay and
|
||||
* destroy the very discriminator this exists to provide.
|
||||
*/
|
||||
public function sentinel(): void
|
||||
{
|
||||
$this->mutateGraph(function (array $graph): array {
|
||||
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
|
||||
foreach (array_keys($baseline['results'] ?? []) as $testId) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
|
||||
|
||||
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return $graph;
|
||||
});
|
||||
}
|
||||
|
||||
public function addBaseline(string $branch): void
|
||||
{
|
||||
$this->mutateGraph(function (array $graph) use ($branch): array {
|
||||
$graph['baselines'][$branch] = ['sha' => null, 'tree' => [], 'results' => []];
|
||||
|
||||
return $graph;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* @param callable(array<string, mixed>): array<string, mixed> $callback
|
||||
*/
|
||||
public function mutateGraph(callable $callback): void
|
||||
{
|
||||
$graph = $this->graph();
|
||||
|
||||
if ($graph === null) {
|
||||
throw new RuntimeException('There is no graph to sentinel.');
|
||||
throw new RuntimeException('There is no graph to mutate.');
|
||||
}
|
||||
|
||||
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
|
||||
foreach (array_keys($baseline['results'] ?? []) as $testId) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
|
||||
|
||||
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($graph, JSON_UNESCAPED_SLASHES));
|
||||
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($callback($graph), JSON_UNESCAPED_SLASHES));
|
||||
|
||||
$this->snapshot();
|
||||
}
|
||||
|
||||
/**
|
||||
* The decoded graph, or `null` when there is none.
|
||||
*
|
||||
* @return array<string, mixed>|null
|
||||
*/
|
||||
public function graph(): ?array
|
||||
@@ -400,7 +364,7 @@ final class Project
|
||||
{
|
||||
$baselines = $this->graph()['baselines'] ?? [];
|
||||
|
||||
return is_array($baselines) ? array_keys($baselines) : [];
|
||||
return is_array($baselines) ? array_map(strval(...), array_keys($baselines)) : [];
|
||||
}
|
||||
|
||||
public function graphDir(): string
|
||||
@@ -413,43 +377,21 @@ final class Project
|
||||
return is_file($this->graphDir().DIRECTORY_SEPARATOR.Tia::KEY_GRAPH);
|
||||
}
|
||||
|
||||
/**
|
||||
* Remembers the graph as it stands now.
|
||||
*/
|
||||
public function snapshot(): void
|
||||
{
|
||||
$this->snapshot = $this->graph();
|
||||
}
|
||||
|
||||
/**
|
||||
* What has happened to the graph since the last snapshot.
|
||||
*/
|
||||
public function delta(): GraphDelta
|
||||
{
|
||||
return new GraphDelta($this->snapshot, $this->graph());
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the `vendor` a composer install would have produced.
|
||||
*
|
||||
* Pest is mirrored in at `vendor/pestphp/pest` rather than pointed at,
|
||||
* because Pest locates things from where its own files sit:
|
||||
* `bin/pest` finds the project root by walking up from the autoloader it
|
||||
* loads, and a parallel run picks its worker binary — and with it the
|
||||
* worker's project root — from the directory the runner class was loaded
|
||||
* from. Deferring to the repository's copy would resolve both back to the
|
||||
* Pest repository, and every scenario would quietly measure that instead.
|
||||
*
|
||||
* Hardlinked where the filesystem allows it, so the mirror costs almost
|
||||
* nothing and can never drift from the working tree.
|
||||
*/
|
||||
public function scaffoldVendor(string $directory): void
|
||||
{
|
||||
$pestRoot = dirname(__DIR__, 3);
|
||||
$pest = $directory.'/vendor/pestphp/pest';
|
||||
|
||||
// `overrides`, `resources` and `stubs` come along because Pest loads
|
||||
// them relative to `src` — the same list `BootExcludeList` walks.
|
||||
foreach (['src', 'overrides', 'resources', 'stubs'] as $tree) {
|
||||
self::mirror($pestRoot.'/'.$tree, $pest.'/'.$tree);
|
||||
}
|
||||
@@ -460,21 +402,11 @@ final class Project
|
||||
|
||||
self::mirror($pestRoot.'/composer.json', $pest.'/composer.json');
|
||||
|
||||
// Pest's own autoloader, with the mirrored copy taking precedence: the
|
||||
// repository's `vendor` supplies PHPUnit, Symfony and the plugin
|
||||
// packages, none of which care where they are loaded from.
|
||||
file_put_contents($directory.'/vendor/autoload.php', sprintf(
|
||||
"<?php\n\n\$loader = require %s;\n\$loader->addPsr4('Pest\\\\', __DIR__.'/pestphp/pest/src', true);\n\nreturn \$loader;\n",
|
||||
var_export($pestRoot.'/vendor/autoload.php', true),
|
||||
));
|
||||
|
||||
// Invoking the binary directly skips composer's bin proxy, which is
|
||||
// what would otherwise define `$GLOBALS['_composer_bin_dir']`. Without
|
||||
// it `Pest\Plugin\Loader` looks for `vendor/bin/../pest-plugins.json`
|
||||
// relative to the working directory — so that is where the plugin list
|
||||
// goes, and mirroring the repository's keeps it in step with
|
||||
// composer.json. `vendor/bin` has to exist for the `..` in that path to
|
||||
// resolve, empty though it is.
|
||||
if (! is_dir($directory.'/vendor/bin') && ! @mkdir($directory.'/vendor/bin', 0755, true)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory.'/vendor/bin'));
|
||||
}
|
||||
@@ -500,9 +432,6 @@ final class Project
|
||||
}
|
||||
|
||||
/**
|
||||
* `Storage` reads `HOME` from the environment, so the graph lands inside the
|
||||
* throwaway project instead of the developer's real `~/.pest`.
|
||||
*
|
||||
* @template TReturn
|
||||
*
|
||||
* @param callable(): TReturn $callback
|
||||
@@ -529,8 +458,6 @@ final class Project
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
|
||||
}
|
||||
|
||||
// Realpathed because Pest realpaths its own project root, and macOS
|
||||
// hands out a symlinked temp directory.
|
||||
$real = realpath($path);
|
||||
|
||||
$project = new self($real === false ? $path : $real);
|
||||
@@ -543,8 +470,6 @@ final class Project
|
||||
self::copy(__DIR__.'/overlays/'.$overlay, $project->path);
|
||||
}
|
||||
|
||||
// `ChangedFiles` asks git what changed, so anything the scaffold writes
|
||||
// but the project does not own has to be invisible to it.
|
||||
$project->write('.gitignore', implode("\n", ['/vendor/', '/.home/', '/.phpunit.cache/', '']));
|
||||
|
||||
$project->scaffoldVendor($project->path);
|
||||
@@ -553,10 +478,6 @@ final class Project
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirrors a file or directory, hardlinking where the filesystem allows it
|
||||
* and copying where it does not.
|
||||
*/
|
||||
private static function mirror(string $from, string $to): void
|
||||
{
|
||||
if (is_dir($from)) {
|
||||
|
||||
@@ -4,8 +4,6 @@ declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// A second test file over the same source file, so a `Calculator` edit narrows
|
||||
// to two of the three test files rather than to one.
|
||||
test('adds within a feature test', function (): void {
|
||||
expect((new Calculator)->add(10, 5))->toBe(15);
|
||||
});
|
||||
|
||||
@@ -2,8 +2,5 @@
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// The fixture project has no composer autoloader of its own — its `vendor/`
|
||||
// holds nothing but a shim pointing back at Pest's. Requiring the two classes
|
||||
// here is enough for every test file in the suite.
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
@@ -4,9 +4,6 @@ declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// `test()` rather than `it()`: the harness seeds results by test id, and `it()`
|
||||
// would prefix every description with `it `, leaving Project::TESTS a step away
|
||||
// from what is written here.
|
||||
test('adds two numbers', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
pest()->tia()->always();
|
||||
@@ -5,7 +5,4 @@ declare(strict_types=1);
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// Declared default branch. Wins over whatever the repository autodetects — the
|
||||
// escape hatch for a checkout with no `origin/HEAD` and a misleading
|
||||
// `init.defaultBranch`.
|
||||
pest()->tia()->defaultBranch('master');
|
||||
|
||||
@@ -5,6 +5,4 @@ declare(strict_types=1);
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// A branch the repository does not have. Accepted as configured — a name that
|
||||
// resolves to no baseline degrades to a full run, which is safe.
|
||||
pest()->tia()->defaultBranch('nope');
|
||||
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/bin/sh
|
||||
|
||||
if [ "$1" = "auth" ]; then
|
||||
[ "$GH_STUB_MODE" = "unauthenticated" ] && exit 1
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$1" = "run" ] && [ "$2" = "list" ]; then
|
||||
case "$GH_STUB_MODE" in
|
||||
no-runs) exit 0 ;;
|
||||
list-404) echo "HTTP 404: Not Found" >&2; exit 1 ;;
|
||||
list-network) echo "could not resolve host: api.github.com" >&2; exit 1 ;;
|
||||
esac
|
||||
echo 987654321
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$1" = "api" ]; then
|
||||
echo 2048
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$1" = "run" ] && [ "$2" = "download" ]; then
|
||||
case "$GH_STUB_MODE" in
|
||||
download-403) echo "HTTP 403: Forbidden" >&2; exit 1 ;;
|
||||
download-network) echo "connection refused" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
dir=""
|
||||
previous=""
|
||||
for argument in "$@"; do
|
||||
[ "$previous" = "-D" ] && dir="$argument"
|
||||
previous="$argument"
|
||||
done
|
||||
|
||||
[ -z "$dir" ] && exit 1
|
||||
|
||||
case "$GH_STUB_MODE" in
|
||||
missing-asset) echo "{}" > "$dir/other.json" ;;
|
||||
corrupt) printf 'not json at all' > "$dir/graph.json" ;;
|
||||
*) cp "$GH_STUB_PAYLOAD" "$dir/graph.json" ;;
|
||||
esac
|
||||
|
||||
exit 0
|
||||
fi
|
||||
|
||||
exit 1
|
||||
+1
-3
@@ -19,9 +19,7 @@ pest()->in('PHPUnit/GlobPatternTests/SubFolder2/*AsPattern.php')->use(CustomTest
|
||||
|
||||
pest()->in('Visual')->group('integration');
|
||||
|
||||
// Every row scaffolds a throwaway project and runs `pest` in it as a
|
||||
// subprocess, which is far too slow for the unit suite.
|
||||
pest()->in('Features/Tia')->group('integration');
|
||||
pest()->in('Features/Tia')->group('integration', 'tia');
|
||||
|
||||
// NOTE: global test value container to be mutated and checked across files, as needed
|
||||
$_SERVER['globalHook'] = (object) ['calls' => (object) ['beforeAll' => 0, 'afterAll' => 0]];
|
||||
|
||||
@@ -67,8 +67,6 @@ describe('applyMigrationChanges()', function (): void {
|
||||
|
||||
describe('rerun tracking', function (): void {
|
||||
beforeEach(function (): void {
|
||||
// `hasUnlocatedTestsToRerun()` stats each recorded file to tell a
|
||||
// deleted test apart from a live one, so the files have to exist.
|
||||
$this->projectRoot = sys_get_temp_dir().'/pest-tia-rerun-'.bin2hex(random_bytes(4));
|
||||
mkdir($this->projectRoot.'/tests/Feature', 0755, true);
|
||||
|
||||
|
||||
@@ -9,6 +9,8 @@ $run = function (): ?string {
|
||||
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
|
||||
);
|
||||
|
||||
$process->setTimeout(300.0);
|
||||
|
||||
$process->run();
|
||||
|
||||
return removeAnsiEscapeSequences($process->getOutput());
|
||||
|
||||
@@ -17,6 +17,8 @@ test('visual snapshot of test suite on success', function (): void {
|
||||
['EXCLUDE' => 'integration', '--exclude-group' => 'integration', 'REBUILD_SNAPSHOTS' => false, 'PARATEST' => 0, 'COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
|
||||
));
|
||||
|
||||
$process->setTimeout(300.0);
|
||||
|
||||
$process->run();
|
||||
|
||||
return preg_replace([
|
||||
|
||||
Reference in New Issue
Block a user