mirror of
https://github.com/pestphp/pest.git
synced 2026-09-05 06:13:35 +02:00
Compare commits
22 Commits
v5.0.2
...
e90e4f70fc
| Author | SHA1 | Date | |
|---|---|---|---|
| e90e4f70fc | |||
| d112582857 | |||
| 9f3c4e1e82 | |||
| 71f39366c2 | |||
| bfd5b75677 | |||
| 411b9954b5 | |||
| db70017cb2 | |||
| 86adaedbbd | |||
| 92c7677c6e | |||
| 872f0a50c2 | |||
| 668809bc75 | |||
| 8d8f45c843 | |||
| 086b3e9107 | |||
| 43fe26f324 | |||
| 865c1e5113 | |||
| 19eed8d581 | |||
| ad1850b110 | |||
| 585de259a2 | |||
| 953664dce0 | |||
| 1dd959848c | |||
| 047753c836 | |||
| b63626a94d |
+1
-1
@@ -1,7 +1,7 @@
|
||||
.idea/*
|
||||
.idea/codeStyleSettings.xml
|
||||
.temp/*
|
||||
composer.lock
|
||||
/composer.lock
|
||||
/vendor/
|
||||
coverage.xml
|
||||
.phpunit.result.cache
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
# CLAUDE.md
|
||||
|
||||
**Do not edit this file.** Agents must never add, remove, or reword anything here. If a change seems needed, say so and let a human do it.
|
||||
|
||||
## Ask before testing
|
||||
|
||||
When asked to review code or build a feature, do not run the test suite and do not write new tests. Make the change, report it, then ask the user whether tests should be added — describing the tests you have in mind — and wait for the user to confirm.
|
||||
|
||||
Two reasons this matters here: the suite takes minutes, and `tests/.snapshots/success.txt` plus the tally in `tests/Visual/Parallel.php` encode the whole suite's result, so a single added test breaks both.
|
||||
|
||||
Once the user confirms:
|
||||
|
||||
```bash
|
||||
composer test:unit # fast, excludes the visual group
|
||||
composer test:integration # visual and snapshot tests
|
||||
composer test # everything CI runs, in CI's order
|
||||
composer update:snapshots # only when a test was added or removed
|
||||
```
|
||||
@@ -0,0 +1,342 @@
|
||||
# TIA write-tier conformance — plan
|
||||
|
||||
Baseline sweep: **147 cases, 144 PASS / 2 FAIL / 1 SKIP** against `dev-fix/tia-filtered` (`db70017`)
|
||||
on `playground/laravel` with pcov enabled.
|
||||
|
||||
This file records (1) what is left to fix, and (2) the full case matrix with the target
|
||||
outcome for every row, so the sweep can be re-run as a conformance check after the fixes land.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — What's left
|
||||
|
||||
### 1. `--parallel` silently under-records `edges` (G12, I5) — **high**
|
||||
|
||||
Any `--tia --parallel` complete run writes `files=4` where sequential writes `files=25`. Only
|
||||
`app/**` and blade views survive; every `config/`, `routes/`, `bootstrap/` edge and every test
|
||||
self-edge is lost. A `routes/web.php` edit that sequential TIA catches, parallel TIA reports as
|
||||
`No affected tests found`.
|
||||
|
||||
Root cause — `src/Restarters/PcovRestarter.php:33`:
|
||||
|
||||
```php
|
||||
if (! Tia::isEnabledForRun($arguments)) {
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
`Tia::isEnabledForRun()` (`src/Plugins/Tia.php:275-288`) returns true only for `--tia` in argv or
|
||||
the `PEST_TIA` env flag. A paratest worker's argv carries neither, so workers never re-exec with
|
||||
`-d pcov.directory=<projectRoot>` and record under pcov's empty default scope.
|
||||
|
||||
Proven both directions:
|
||||
- `PEST_PCOV_RESTARTER_RESTARTED=1 pest --tia` (sequential, restart suppressed) reproduces
|
||||
`files=4` with edge sets identical to the parallel graph.
|
||||
- `PEST_TIA=1 pest --parallel` restores the full `files=25` graph. **Working workaround today.**
|
||||
|
||||
Coverage config is irrelevant: widening `phpunit.xml` `<include>` to `app|config|routes|bootstrap|tests`
|
||||
still yields `files=4`.
|
||||
|
||||
Sticky, not just per-run: `pest --tia --filtered --parallel` re-records only the affected test
|
||||
through a worker and strips that entry's edges from an otherwise-healthy sequential graph, so one
|
||||
parallel run degrades a good graph incrementally.
|
||||
|
||||
**Fix direction:** propagate the TIA-enabled decision into workers (e.g. set `PEST_TIA` in the
|
||||
worker env, or have the parent stamp a recording global the restarter also consults) so
|
||||
`PcovRestarter` restarts them. Then assert `edges` equivalence between sequential and parallel.
|
||||
|
||||
### 2. Deleted test file with a cached failure wedges `--filtered` forever (I9) — **high**
|
||||
|
||||
Expected a WARN and a full-suite fallback. Actual: `--filtered` selects the phantom file, runs
|
||||
nothing, exits **0 green**, on every subsequent invocation.
|
||||
|
||||
Root cause — `src/Plugins/Tia/Graph.php:1523-1525`:
|
||||
|
||||
```php
|
||||
if ($real === false) {
|
||||
$real = $path;
|
||||
}
|
||||
```
|
||||
|
||||
`relative()` falls back to the raw path when `realpath()` fails, so a deleted-but-in-project test
|
||||
file is never "unlocated", `Graph::hasUnlocatedTestsToRerun()` never fires, and the WARN branch at
|
||||
`src/Plugins/Tia.php:982-986` is unreachable for the deletion case. `pruneMissingTests()` only runs
|
||||
under `--fresh`, so nothing clears it.
|
||||
|
||||
**Fix direction:** distinguish "path could not be resolved" from "path resolved outside the root" in
|
||||
`relative()`, or have `hasUnlocatedTestsToRerun()` stat the file directly.
|
||||
|
||||
### 3. `--coverage-text` does not disable filtered mode (I8) — **low**
|
||||
|
||||
`coverageReportActive()` (`src/Plugins/Tia.php:1705-1710`) reads only Pest's own `--coverage`, so
|
||||
raw PHPUnit coverage flags (`--coverage-text`, `--coverage-html`, `--coverage-clover`, …) leave
|
||||
filtered mode on. `Tia::COVERAGE_FLAGS` at `:113-115` already enumerates them — that list is not
|
||||
consulted here.
|
||||
|
||||
### 4. `--coverage` narrows edges like parallel does — **medium, needs triage**
|
||||
|
||||
`pest --tia --coverage` prints `fresh graph (recording coverage baseline)` even when a graph
|
||||
exists, and writes `Feature/ExampleTest` with 2 files instead of 16, dropping self-edges. Same
|
||||
observable shape as G12, likely a different mechanism (piggyback collector scoped by
|
||||
`phpunit.xml <include>` rather than the pcov-restarted recorder). Theme: **edges silently narrow to
|
||||
`app/**` whenever recording does not go through the pcov-restarted TIA recorder.** Confirm whether
|
||||
these are one fix or two.
|
||||
|
||||
### 5. Smaller items
|
||||
|
||||
| Item | Where | Note |
|
||||
|---|---|---|
|
||||
| Warning/deprecation recorded as `status=0` | write path | Codes `6` and `4` appear unreachable; risky/skipped/incomplete record faithfully |
|
||||
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
|
||||
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
|
||||
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
|
||||
| Replay clobbers cached `time` | write path | `0.058` → `0.001`; timing data degrades toward zero across runs |
|
||||
| `pest --parallel` without `--tia` writes nothing | G3, G6 | Sequential `pest --filter=…` does record. Parallel CI contributes nothing to the cached-failure replay path |
|
||||
| `--tia --parallel --filter`/`--shard` record nothing | G2, G8 | More conservative than the results-only contract requires |
|
||||
| `--min=50` without `--coverage` is a silent no-op | — | — |
|
||||
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
|
||||
|
||||
### 6. Test-harness gaps to close before re-running
|
||||
|
||||
- The playground has no `UsesClass`, `->note()`, `->flaky()`, `->issue()`, `->pr()`, `->ticket()` or
|
||||
`->assignee()` annotations, so **C19, C23–C28, C37–C39 matched zero tests** and their
|
||||
graph-invariant assertions proved nothing. Add annotated fixtures to make those rows load-bearing.
|
||||
- Use **sentinel patching** (rewrite every cached entry to `time=9.999 assertions=42`, then see which
|
||||
entries get overwritten) as the discriminator. It is the only way to tell "wrote identical values"
|
||||
from "wrote nothing". A canary test absent from `edges` is unreliable under `--filter` because it
|
||||
never matches the filter and so never runs.
|
||||
- `--tia` on a clean green tree can never cache a failure (unchanged tests replay rather than
|
||||
execute); seeding one requires `--fresh` or an env-driven flaky fixture.
|
||||
- Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix (target outcomes)
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Status column: `PASS` = conforming today · `FIX` = must pass once the item above lands ·
|
||||
`SKIP` = case not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | PASS |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | PASS |
|
||||
| A4 | `git status` after reset ritual | clean | PASS |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | PASS |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | PASS |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | PASS |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | PASS |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS |
|
||||
| B10 | `pest --tia` twice | `time` differs; statuses stable | PASS |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key + new result appear | PASS |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written | PASS |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS |
|
||||
| D11 | `pest --tia --bail` | RO | PASS |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran | PASS |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | PASS |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | PASS |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written | PASS |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | PASS (but see G12 — it strips edges) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | PASS |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | PASS |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** (`files=25` both) | **FIX (item 1)** |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | PASS |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | PASS |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **FIX (item 1)** |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report | **FIX (item 3)** |
|
||||
| I9 | cached failure whose test file was deleted | WARN `could not be located on disk`; falls back to the full suite with replay | **FIX (item 2)** |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild (and the drift reason should be printed — item 5) | PASS |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Status |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE | PASS |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (signal must go to the re-exec'd child — item 5) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS |
|
||||
|
||||
---
|
||||
|
||||
## Re-run procedure
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json"
|
||||
|
||||
# per case
|
||||
git checkout -q . && git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
Every pest invocation must be prefixed with `PAO_DISABLE=1` (the app has `laravel/pao`, which emits
|
||||
JSON when it detects an agent). The shell is zsh — build commands with arrays or `eval`; unquoted
|
||||
`$args` does not word-split.
|
||||
@@ -0,0 +1,455 @@
|
||||
# TIA default-branch fallback — phase three
|
||||
|
||||
## Your task
|
||||
|
||||
Fix [pestphp/pest#1823](https://github.com/pestphp/pest/issues/1823) — TIA's cached results never
|
||||
hit for repos whose default branch is not literally `main` — then re-run the **whole** conformance
|
||||
matrix against the playground app, plus the new section **L** that covers the fix.
|
||||
|
||||
Work in this order, and **stop where the plan says stop**:
|
||||
|
||||
1. **Part 1** — read the diagnosis. It is already measured; do not re-derive it, but do re-confirm
|
||||
the two reproductions in Part 1.3 take ~2 minutes and prove your environment is sane.
|
||||
2. **Part 2** — implement the fix in the `pestphp/pest` repo. **Do not commit. Do not touch the
|
||||
playground's `vendor/`.**
|
||||
3. **HARD STOP → Part 3.** Report the diff to Nuno and wait. He validates, commits, and syncs it
|
||||
into the playground himself. You must not proceed until he confirms.
|
||||
4. **Part 4** — verify the sync landed, then run section **L** (new) and the full phase-two matrix
|
||||
(**A–K**, all 156 rows) against the playground.
|
||||
5. **Part 5** — report in the given format.
|
||||
|
||||
Per `CLAUDE.md`: **do not write new `pestphp/pest` unit tests and do not run `composer test`.** Make
|
||||
the change, report it, and ask whether repo tests should be added. The section-L rows are *playground
|
||||
invocations*, not repo tests — those are the deliverable and are always in scope.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — The diagnosis (already measured; commit `bfd5b756`)
|
||||
|
||||
### 1.1 Root cause — two independent hardcoded `'main'` literals
|
||||
|
||||
**(a) The read fallback.** `src/Plugins/Tia/Graph.php` — seven methods default the fallback to the
|
||||
literal `'main'`:
|
||||
|
||||
| line | method |
|
||||
|---|---|
|
||||
| 579 | `recordedAtSha(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 614 | `getAssertions(…, string $fallbackBranch = 'main')` |
|
||||
| 625 | `getTime(…, string $fallbackBranch = 'main')` |
|
||||
| 636 | `getResult(…, string $fallbackBranch = 'main')` |
|
||||
| 663 | `testFilesToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 700 | `hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main')` |
|
||||
| 811 | `lastRunTree(string $branch, string $fallbackBranch = 'main')` |
|
||||
|
||||
They all funnel into `Graph::baselineFor()` (line 819), which *does* implement a real cross-branch
|
||||
fallback:
|
||||
|
||||
```php
|
||||
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
|
||||
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
|
||||
return ['sha' => null, 'tree' => [], 'results' => []];
|
||||
```
|
||||
|
||||
The mechanism is deliberate. The problem is that **no caller ever passes `$fallbackBranch`** — all
|
||||
nine call sites in `src/Plugins/Tia.php` (lines 419, 429, 432, 785, 857, 1026, 1031, 1048, 1056) pass
|
||||
only `$this->branch`. So on a `master`-named repo the second branch of `baselineFor()` can never
|
||||
fire, and the whole mechanism is dead code.
|
||||
|
||||
`grep -rniE 'defaultBranch|symbolic-ref|init\.defaultBranch|origin/HEAD' src/` returns **nothing** —
|
||||
no default-branch resolution exists anywhere.
|
||||
|
||||
**(b) The detached-HEAD default.** `src/Plugins/Tia.php:206` declares `private string $branch = 'main';`
|
||||
and `ChangedFiles::currentBranch()` (`ChangedFiles.php:208`) returns `null` for detached HEAD. So on
|
||||
detached HEAD `$this->branch` stays the literal `'main'` and is used for **both reads and writes** —
|
||||
minting a baseline key for a branch that does not exist. This is a *write*-side bug and a separate fix
|
||||
from (a).
|
||||
|
||||
### 1.2 Not a regression
|
||||
|
||||
`git log -S"fallbackBranch = 'main'"` bottoms out at `c7e32f5d feat(tia): continues to work on poc`.
|
||||
This is original PoC code, untouched by phase one. Phase one's change 1 modified
|
||||
`hasUnlocatedTestsToRerun()`'s file-existence check — one of the seven methods — without going near
|
||||
the fallback. Do not report it as a phase-one regression.
|
||||
|
||||
No test anywhere exercises the mechanism: `tests/Unit/Plugins/Tia/Graph.php` uses `'main'` as the
|
||||
*actual* branch name, so those assertions pass whether or not the fallback exists. A branch-name
|
||||
mismatch is never tested.
|
||||
|
||||
### 1.3 The two reproductions (re-confirm these before you start)
|
||||
|
||||
Both on the playground, `master`-named default branch, zero local changes:
|
||||
|
||||
```
|
||||
default = master default = main
|
||||
1. record on default → full run (cold) → full run (cold)
|
||||
2. 1st run on feature-x → 25 UNCACHED → 25 replayed ← (a)
|
||||
3. 2nd run on feature-x → 25 replayed → 25 replayed
|
||||
4. back on default → 25 replayed → 25 replayed
|
||||
5. 1st run on feature-y → 25 UNCACHED → 25 replayed
|
||||
```
|
||||
|
||||
```
|
||||
master-only graph, then `git checkout --detach`, then `pest --tia`:
|
||||
→ 25 uncached, and keys become [master,main] ← (b) spurious key
|
||||
```
|
||||
|
||||
The cost is **one full run per new branch, forever**, with no output explaining why — the only clue
|
||||
is the `N uncached` count; the headline is just `─ Experimental TIA mode enabled.`
|
||||
|
||||
### 1.4 Correction to phase two
|
||||
|
||||
Phase two reported **H6 and H7 as passing. Both were false passes.** They ran after H5, which had
|
||||
renamed `master`→`main` and left a `main` key in the graph, so the hardcoded fallback resolved by
|
||||
accident. Re-measured against a clean `master`-only graph, both fail. Section L replaces them as the
|
||||
load-bearing rows; H6/H7 must be re-run **from a cold graph** this time (see Part 4.2).
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — The fix to implement
|
||||
|
||||
### 2.1 Recommended design
|
||||
|
||||
**Step 1 — add a non-throwing resolver** to `src/Plugins/Tia/ChangedFiles.php`, next to
|
||||
`currentBranch()`:
|
||||
|
||||
```php
|
||||
public function defaultBranch(): ?string
|
||||
```
|
||||
|
||||
Resolution order, each step failing soft to the next:
|
||||
|
||||
1. `git symbolic-ref --short refs/remotes/origin/HEAD` → strip a leading `origin/`
|
||||
2. `git config --get init.defaultBranch`
|
||||
3. `null`
|
||||
|
||||
Unlike `currentBranch()`, this must **never throw** `MissingDependency` — it is advisory. Return
|
||||
`null` on any non-zero exit or empty output.
|
||||
|
||||
**Step 2 — add a config surface.** `src/Plugins/Tia/Configuration.php` already exposes `always()`,
|
||||
`locally()`, `filtered()`, `baselined()`, `watch()`. Add:
|
||||
|
||||
```php
|
||||
public function defaultBranch(string $branch): self
|
||||
```
|
||||
|
||||
so `pest()->tia()->defaultBranch('master')` works in `tests/Pest.php`. Explicit config **always
|
||||
wins** over autodetection — that is the escape hatch when `origin/HEAD` is unset.
|
||||
|
||||
**Step 3 — resolve once, in `Tia.php`.** The read path is hot (`getResult()` is called per test at
|
||||
line 419), so resolution must not shell out per call. Resolve alongside `$this->branch` at
|
||||
`Tia.php:1910`, under the existing `$branchResolved` guard:
|
||||
|
||||
```php
|
||||
$this->fallbackBranch = $configuredDefaultBranch
|
||||
?? $changedFiles->defaultBranch()
|
||||
?? 'main';
|
||||
```
|
||||
|
||||
**Step 4 — thread it into `Graph`.** Prefer a `Graph`-level property over editing nine call sites:
|
||||
add `Graph::setFallbackBranch(string $branch)`, change the seven signatures to
|
||||
`?string $fallbackBranch = null`, and resolve inside each with
|
||||
`$fallbackBranch ??= $this->fallbackBranch;`. `baselineFor()` itself needs no change. This keeps the
|
||||
public signatures backward-compatible and minimises blast radius.
|
||||
|
||||
**Step 5 — fix the detached-HEAD write.** `Tia.php:206`'s `= 'main'` default must become the
|
||||
resolved default branch, so detached HEAD stops minting a phantom key.
|
||||
|
||||
### 2.2 Invariants the fix must not break
|
||||
|
||||
These are all covered by existing matrix rows — the fix is wrong if any of them moves:
|
||||
|
||||
- **Read-only.** The fallback must affect *reads* only. Writes go through `ensureBaseline($branch)`
|
||||
and must keep using the real current branch. Otherwise H1–H4/H8 ("no baseline key other than the
|
||||
real branch") break.
|
||||
- **H9** — a non-git dir with `--tia` must still raise
|
||||
`MissingDependency: The feature "Tia mode" requires "git".` Adding a soft resolver must not
|
||||
swallow that.
|
||||
- **H10** — plain `pest` in a non-git dir must still run and create no baseline dir.
|
||||
- **A2/A3** — cold-graph recording unchanged.
|
||||
- **I1/E3** — clean+green `--tia --filtered` must still be a true zero-delta run.
|
||||
- Filtered mode reads `testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so the fallback must
|
||||
reach those two as well, not just `getResult()`.
|
||||
|
||||
### 2.3 Decisions for Nuno (raise these at the Part 3 stop)
|
||||
|
||||
- **D1** — Consult `origin/HEAD` at all? It requires `git remote set-head` and is absent in many CI
|
||||
checkouts and all remote-less repos. Config + `init.defaultBranch` only is simpler but helps fewer
|
||||
people out of the box. *Recommendation: keep it, first in the chain, since it fails soft.*
|
||||
- **D2** — Single-key heuristic: if the graph holds exactly one baseline key, use it as the fallback?
|
||||
Fixes the issue with zero git calls, but is implicit and surprising when several keys exist.
|
||||
*Recommendation: no.*
|
||||
- **D3** — Should detached HEAD write a baseline at all, or be read-only? Current behaviour mints a
|
||||
key. *Recommendation: read-only.*
|
||||
- **D4** — Should `pest()->tia()->defaultBranch()` validate that the branch exists, or accept any
|
||||
string? *Recommendation: accept any string; a nonexistent name degrades to a full run, which is
|
||||
safe.*
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — HARD STOP
|
||||
|
||||
When the code is written:
|
||||
|
||||
1. Show Nuno the diff (`git -C /Users/nunomaduro/Work/projects/pestphp/pest diff`) and a one-paragraph
|
||||
summary of each file's change.
|
||||
2. Answer/raise the D1–D4 decisions.
|
||||
3. State explicitly that you have **not** committed and have **not** synced `vendor/`.
|
||||
4. Ask whether repo unit tests should be added (per `CLAUDE.md`), describing the tests you have in
|
||||
mind — do not write them yet.
|
||||
5. **Wait.** Nuno commits and applies the change to the playground.
|
||||
|
||||
**Never sync the playground's `vendor/` yourself.** `vendor/pestphp/pest` there is a dist copy, not a
|
||||
symlink (composer installed `dev-fix/tia-filtered as 5.2.0`), so pest-repo edits do not reach it. Say
|
||||
what is stale and wait. This applies to before/after contrasts too.
|
||||
|
||||
Once he confirms, verify the sync actually landed before measuring anything:
|
||||
|
||||
```bash
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
grep -c 'defaultBranch' "$V/src/Plugins/Tia/ChangedFiles.php" # must be ≥ 1
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php src/Plugins/Tia/ChangedFiles.php \
|
||||
src/Plugins/Tia/Configuration.php; do
|
||||
diff -q "/Users/nunomaduro/Work/projects/pestphp/pest/$f" "$V/$f" >/dev/null \
|
||||
&& echo "SAME $f" || echo "STALE $f"
|
||||
done
|
||||
```
|
||||
|
||||
If anything reports `STALE`, stop and tell him. State in your final report which pest commit produced
|
||||
the playground numbers.
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Measurement
|
||||
|
||||
### 4.1 Environment and traps
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel` (branch `master`).
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON when
|
||||
it detects an agent.
|
||||
2. **Pin the interpreter to `php85`.** The playground requires PHP `>= 8.4.1` *and* pcov. Locally only
|
||||
`php85` (8.5.8) has both — `php84` (8.4.23) has no pcov, and the bare Herd `php` shim has been
|
||||
observed drifting to 8.3.32 mid-session, which kills every run in
|
||||
`vendor/composer/platform_check.php`. Also put a `php` → `php85` symlink first on `PATH`: the
|
||||
`--shard` list-tests probe spawns a subprocess via bare `php`, not `PHP_BINARY`.
|
||||
3. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`. Scope
|
||||
resets to `git checkout -- tests app` / `git clean -fd tests app`. For `phpunit.xml` and
|
||||
`composer.lock`, copy aside and copy back, verifying with `shasum`.
|
||||
4. **zsh does not word-split unquoted parameters.** A `$PEST` string containing a space becomes one
|
||||
command name. Route every invocation through `eval` (the `pest()` helper below does this).
|
||||
5. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
6. **The sentinel technique must not falsify `assertions` on zero-assertion tests.** A
|
||||
risky/skipped/incomplete status is *derived* from "performed no assertions", so patching those to
|
||||
`42` rewrites the status on replay and destroys the discriminator — it shows up as a phantom
|
||||
`status 5→0` defect. The `sentinel.php` below only patches `assertions` where it is already
|
||||
non-zero. With that, a full replay gives a clean `rewritten=0`.
|
||||
7. Restore branch state after every L row. Several rows rename or detach; leaving a stray `main` key
|
||||
in the graph is exactly what produced phase two's false H6/H7 passes.
|
||||
|
||||
### 4.2 Reference numbers
|
||||
|
||||
The playground already carries the phase-two fixtures at commit `fb2e77e` — **do not rebuild them.**
|
||||
A healthy sequential `--tia` graph is:
|
||||
|
||||
> **`files=27`, 10 edge keys (one per test file), self-edge on all 10, `n=25` results.**
|
||||
|
||||
`sha` will differ once Nuno commits the vendor sync — re-derive it once and use it throughout. The
|
||||
`files`/`edges`/`n` numbers hold as long as no test fixture changes. Suite shape: 10 test files, 25
|
||||
tests, including six deliberate status fixtures (skipped, todo, incomplete, risky, warning,
|
||||
deprecation), a 3-row dataset, a `smoke` group, an env-driven flaky test (green unless
|
||||
`FLAKY_FAIL=1`), and the annotation set (`covers`/`note`/`flaky`/`issue`/`pr`/`ticket`/`assignee`).
|
||||
|
||||
**Re-run H6 and H7 from a cold graph** (`rm -rf` the graph dir, record on `master` only, *then*
|
||||
branch/detach). Their phase-two results are void.
|
||||
|
||||
### 4.3 Harness
|
||||
|
||||
Write these to your scratchpad. `$SP` is your own scratchpad dir.
|
||||
|
||||
<details>
|
||||
<summary><code>lib.sh</code></summary>
|
||||
|
||||
```bash
|
||||
#!/bin/zsh
|
||||
export PAO_DISABLE=1
|
||||
PG=/Users/nunomaduro/Work/projects/playground/laravel
|
||||
SP="<your scratchpad>"
|
||||
PHPBIN="php85"
|
||||
PEST="$PHPBIN $PG/vendor/bin/pest"
|
||||
cd "$PG" || exit 1
|
||||
pest() { eval "$PEST $*"; } # zsh: no word-splitting, must eval
|
||||
GRAPHDIR="$(pest --baseline)"
|
||||
GRAPH="$GRAPHDIR/graph.json"
|
||||
mkdir -p "$SP/bin" && ln -sf "$(command -v php85)" "$SP/bin/php"
|
||||
export PATH="$SP/bin:$PATH" # --shard spawns bare `php`
|
||||
reset_tree() { git checkout -- tests app 2>/dev/null; git clean -qfd tests app 2>/dev/null; }
|
||||
seed() { rm -rf "$GRAPHDIR"; pest --tia >/dev/null 2>&1; $PHPBIN "$SP/sentinel.php" "$GRAPH" >/dev/null; cp "$GRAPH" "$SP/before.json"; }
|
||||
snap() { cp "$GRAPH" "$SP/before.json"; }
|
||||
delta() { $PHPBIN "$SP/cmp.php" "$SP/before.json" "$GRAPH"; }
|
||||
keys() { $PHPBIN -r '$g=json_decode(file_get_contents($argv[1]),true);echo "[".implode(",",array_keys($g["baselines"]??[]))."]";' "$GRAPH"; }
|
||||
tally() { sed -E $'s/\x1b\\[[0-9;]*[a-zA-Z]//g' <"$SP/out.txt" | grep -E 'Tests:' | sed -E 's/^ +//;s/Tests: +//'; }
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>sentinel.php</code> — the write discriminator</summary>
|
||||
|
||||
```php
|
||||
<?php // sentinel.php <graph.json>
|
||||
$p = $argv[1];
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR);
|
||||
$n = 0;
|
||||
foreach ($g['baselines'] ?? [] as $br => $b) {
|
||||
foreach (array_keys($b['results'] ?? []) as $id) {
|
||||
$g['baselines'][$br]['results'][$id]['time'] = 9.999;
|
||||
// Only falsify a non-zero assertion count: risky/skipped/incomplete are
|
||||
// DERIVED from "performed no assertions", so patching those to 42 would
|
||||
// rewrite the status on replay and destroy the discriminator.
|
||||
if ((int) ($b['results'][$id]['assertions'] ?? 0) > 0) {
|
||||
$g['baselines'][$br]['results'][$id]['assertions'] = 42;
|
||||
}
|
||||
$n++;
|
||||
}
|
||||
}
|
||||
file_put_contents($p, json_encode($g, JSON_THROW_ON_ERROR));
|
||||
echo "sentinelled $n results\n";
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><code>oneline.php</code> — one compact tier verdict per row</summary>
|
||||
|
||||
```php
|
||||
<?php // oneline.php <before.json> <after.json>
|
||||
function load(string $p): ?array {
|
||||
return is_file($p) ? json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR) : null;
|
||||
}
|
||||
$a = load($argv[1]); $b = load($argv[2]);
|
||||
if ($a === null || $b === null) { echo 'GRAPH '.($b === null ? 'DELETED' : 'CREATED'); exit; }
|
||||
function edgeSets(array $g): array {
|
||||
$out = [];
|
||||
foreach ($g['edges'] ?? [] as $t => $ids) { $s = array_map(fn($i) => $g['files'][$i] ?? "?$i", (array) $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$moved = [];
|
||||
if (edgeSets($a) !== edgeSets($b)) $moved[] = 'edges';
|
||||
if (($a['files'] ?? []) !== ($b['files'] ?? [])) $moved[] = 'files';
|
||||
if (($a['fingerprint'] ?? null) !== ($b['fingerprint'] ?? null)) $moved[] = 'fingerprint';
|
||||
$brA = array_keys($a['baselines'] ?? []); $brB = array_keys($b['baselines'] ?? []);
|
||||
if ($brA !== $brB) $moved[] = 'branchkeys('.implode('|', $brA).'->'.implode('|', $brB).')';
|
||||
$add = $rem = $wr = 0; $shaMoved = $treeMoved = false;
|
||||
foreach ($brB as $br) {
|
||||
$ra = $a['baselines'][$br]['results'] ?? []; $rb = $b['baselines'][$br]['results'] ?? [];
|
||||
if (($a['baselines'][$br]['sha'] ?? null) !== ($b['baselines'][$br]['sha'] ?? null)) $shaMoved = true;
|
||||
if (($a['baselines'][$br]['tree'] ?? null) !== ($b['baselines'][$br]['tree'] ?? null)) $treeMoved = true;
|
||||
$add += count(array_diff(array_keys($rb), array_keys($ra)));
|
||||
$rem += count(array_diff(array_keys($ra), array_keys($rb)));
|
||||
foreach ($ra as $id => $x) {
|
||||
if (! isset($rb[$id])) continue;
|
||||
foreach (['status','time','assertions','message'] as $f) {
|
||||
if (($x[$f] ?? null) !== ($rb[$id][$f] ?? null)) { $wr++; break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
$n = 0; foreach ($brB as $br) $n = max($n, count($b['baselines'][$br]['results'] ?? []));
|
||||
printf('n=%d w=%-2d +%d -%d %s%s%s', $n, $wr, $add, $rem,
|
||||
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
|
||||
$shaMoved ? ' sha:CHANGED' : '', $treeMoved ? ' tree:chg' : '');
|
||||
```
|
||||
</details>
|
||||
|
||||
`cmp.php` (verbose per-entry version of the same) and `summarise.php` / `edgediff.php` are in
|
||||
`PLAN_PHASE_TWO.md` §"Graph summariser" — reuse them for drill-downs. Reading the verdict:
|
||||
|
||||
- `w=` — entries actually **written**. Under sentinel patching this is the only reliable way to tell
|
||||
"wrote identical values" from "wrote nothing". A full replay must give `w=0`.
|
||||
- `struct:ok` + `+0 -0` — no prune, no edges/files/fingerprint movement. Required by RESULTS-ONLY.
|
||||
- `STRUCT:branchkeys(...)` — a new baseline key appeared. For section L this is the headline signal.
|
||||
|
||||
### 4.4 Section L — new rows for this fix
|
||||
|
||||
Tiers, unchanged from phase two: **COMPLETE** may change everything · **RESULTS-ONLY (RO)** may
|
||||
change only `baselines[<branch>].results` for tests that ran, and must never remove an entry, add a
|
||||
result for a test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Every L row starts from a **cold graph recorded on the named default branch only** — verify
|
||||
`keys=[<default>]` before branching. Restore branch state afterwards.
|
||||
|
||||
| # | Case | Target outcome |
|
||||
|---|---|---|
|
||||
| L1 | default `master`, record, `git switch -c feature-x`, `pest --tia`, zero changes | **all 25 replayed** (`w=0`), not `25 uncached`. The headline fix. |
|
||||
| L2 | as L1 but default `main` | still all replayed — regression guard, this already worked |
|
||||
| L3 | default `trunk`, then `develop` | replayed for both; the fix must not special-case two names |
|
||||
| L4 | L1, then a *second* new branch `feature-y` | replayed too — the toll must not return per branch |
|
||||
| L5 | L1 then edit `app/Services/Calculator.php` on `feature-x` | narrows to the 2 affected files (`CalculatorTest` + `AnnotationsTest`, which `covers` it); the other 17 replay |
|
||||
| L6 | L1 with `--tia --filtered` | filtered mode reads the fallback too (`testFilesToRerun`, `hasUnlocatedTestsToRerun`) → `No affected tests found`, zero delta |
|
||||
| L7 | L1 with `--tia --parallel` | fallback works in workers as well as the parent |
|
||||
| L8 | L1, then `pest --tia` twice on `feature-x` | idempotent; second run also `w=0` |
|
||||
| L9 | detached HEAD on a `master`-only graph | replays, and **no `main` key minted** — `keys` stays `[master]`. Bug (b). |
|
||||
| L10 | `pest()->tia()->defaultBranch('master')` in `tests/Pest.php`, repo default renamed away | config wins over autodetect |
|
||||
| L11 | config set to a nonexistent branch (`defaultBranch('nope')`) | degrades to a full run; no crash, no phantom key |
|
||||
| L12 | no remote at all (`git remote remove origin` if present) | still resolves (via `init.defaultBranch`) or degrades safely — must not throw |
|
||||
| L13 | branch name with a slash (`feature/x/y`) | replayed; no key-splitting bugs |
|
||||
| L14 | L1, then confirm writes | `feature-x` gets its **own** key; the `master` key is **not** written to (fallback is read-only) |
|
||||
| L15 | git worktree on a new branch (the issue's scenario) | replays from the default-branch baseline |
|
||||
| L16 | non-git dir, `pest --tia` | still `MissingDependency: The feature "Tia mode" requires "git".` — the soft resolver must not swallow it |
|
||||
| L17 | non-git dir, plain `pest` | runs normally; no baseline dir created |
|
||||
| L18 | count `git` subprocesses during one `--tia` run | default-branch resolution is cached, not one call per test. Probe by shimming `git` on `PATH` to a logging wrapper. |
|
||||
|
||||
L10–L11 need a `tests/Pest.php` edit — that file is tracked and **outside** the `tests app` reset
|
||||
scope in practice (it lives in `tests/`, so `git checkout -- tests` does restore it; verify with
|
||||
`git status` after).
|
||||
|
||||
For L16/L17, build a throwaway non-git project — and note the trap that burned phase two: a
|
||||
**symlinked** `vendor` makes Pest resolve the project root back to the playground (identical baseline
|
||||
hash), silently invalidating the test. Use a hardlinked copy:
|
||||
|
||||
```bash
|
||||
NG="$SP/nogit"; rm -rf "$NG"; mkdir -p "$NG"
|
||||
cp -R composer.json composer.lock phpunit.xml artisan tests app bootstrap config routes resources storage "$NG/"
|
||||
[ -f .env ] && cp .env "$NG/"
|
||||
cp -Rl vendor "$NG/vendor" || cp -R vendor "$NG/vendor"
|
||||
```
|
||||
|
||||
Confirm the baseline path differs (`nogit-<hash>`, not `laravel-4a455a95622ac0ec`), and delete both
|
||||
the temp project and its `~/.pest/tia/nogit-*` dir afterwards.
|
||||
|
||||
### 4.5 Re-run the phase-two matrix (A–K, 156 rows)
|
||||
|
||||
Re-run every row of `PLAN_PHASE_TWO.md` Part 2 against the fixed build. No row's status is trusted
|
||||
until re-measured — the fix touches `Graph`'s read path, which nearly every row exercises. Sections
|
||||
**A, B, H, I** are the load-bearing ones here (H is branch-key resolution; I is filtered mode; both
|
||||
consume the changed methods directly). **H6 and H7 must be re-derived from a cold graph** (Part 1.4).
|
||||
|
||||
Most rows batch cheaply — phase two ran C1–C20 in one call at roughly one line of output each. Use
|
||||
`oneline.php` for the sweep and `cmp.php` only to drill into anomalies.
|
||||
|
||||
### 4.6 Known pre-existing failures — do not report as regressions
|
||||
|
||||
| item | status |
|
||||
|---|---|
|
||||
| **G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests. `mergeWorkerReplayPartials()` takes `$result['time']` verbatim at `Tia.php:1451`, never routing through `resultTime()` as the sequential sites (1681, 1762) do. Assertions survive because workers replay those themselves. | Pre-existing (pre-fix had no preservation at all), **out of scope for phase three**. Report it as still-present; do not fix it unless Nuno asks. |
|
||||
| **C19** — `--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
|
||||
| **J11** — `--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
|
||||
| **J10** — `--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Reporting
|
||||
|
||||
Per row: **tier respected (yes/no)**, the **graph delta** under sentinel patching, and for any failure
|
||||
the **pre-fix contrast** so a regression is told apart from a pre-existing defect. For a pre-fix
|
||||
contrast you need `db70017c` (or `bfd5b756` for "before phase three") files swapped into the
|
||||
playground's `vendor/` — that is a sync, so **ask Nuno first** and always restore afterwards.
|
||||
|
||||
State which pest commit produced the numbers. Close with:
|
||||
|
||||
1. Whether L1–L18 all pass (the fix works).
|
||||
2. Whether A–K regressed anywhere relative to phase two's 154/156.
|
||||
3. The D1–D4 decisions as implemented.
|
||||
4. Anything still open — including G4, which will still be failing.
|
||||
|
||||
Leave the playground on `master` with only the four user-modified files dirty, no stray branches, and
|
||||
no leftover `~/.pest/tia/*` dirs beyond `laravel-4a455a95622ac0ec`.
|
||||
@@ -0,0 +1,466 @@
|
||||
# TIA write-tier conformance — phase two
|
||||
|
||||
## Your task
|
||||
|
||||
**Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app** at
|
||||
`/Users/nunomaduro/Work/projects/playground/laravel`. Every row, including the ones already marked
|
||||
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
|
||||
re-measured against the current code. The `Phase 1` column is prior evidence and a hint at what to
|
||||
watch, never a reason to skip a row.
|
||||
|
||||
These are **not** the `pestphp/pest` repo's own tests (`composer test`) — do not run those. Each row is
|
||||
a `pest` invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
|
||||
|
||||
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any
|
||||
failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do **not**
|
||||
stop at reading this file or summarising it; the deliverable is executed results.
|
||||
|
||||
Work in this order: **Part 1 (environment traps) → Part 1b (build the fixtures — the playground has
|
||||
none of them) → Part 2 (the matrix) → Part 3 (priorities)**. The matrix is too large for one context;
|
||||
take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are
|
||||
the load-bearing ones — do those first if you run short.
|
||||
|
||||
Phase one implemented `PLAN.md` Part 1 items 1–4 plus three §5 items. This file turns the matrix into
|
||||
a conformance check rather than a bug list.
|
||||
|
||||
Code under test: `pestphp/pest` at `/Users/nunomaduro/Work/projects/pestphp/pest`, branch
|
||||
`fix/tia-filtered`, commit **`bfd5b756`** or later. Verify with
|
||||
`grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php` → at least 3 hits. Pre-fix
|
||||
baseline for every contrast is **`db70017c`**.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — What phase one changed
|
||||
|
||||
| # | Change | Files |
|
||||
|---|---|---|
|
||||
| 1 | `hasUnlocatedTestsToRerun()` stats the file, so a deleted test file is "unlocated" | `src/Plugins/Tia/Graph.php` |
|
||||
| 2 | `enterReplayMode()` uses `activateLinkTracking()` under piggyback coverage | `src/Plugins/Tia.php` |
|
||||
| 3 | `enterReplayMode()` stamps `TIA_PIGGYBACK_COVERAGE` for workers | `src/Plugins/Tia.php` |
|
||||
| 4 | `replaceEdges(…, keepExisting:)` — piggyback edges seed empty sets, never overwrite populated ones | `Graph.php`, `Tia.php` |
|
||||
| 5 | `renderFreshGraph()` stops claiming "fresh graph" when the graph is kept; reason reworded to `recording a coverage baseline` | `Tia.php` |
|
||||
| 6 | `COVERAGE_REPORT_FLAGS` + `coverageReportActive()` union over `originalArguments`; new `pestCoverageActive()` keeps the coverage-cache marker/hijack on Pest's own `--coverage` | `Tia.php` |
|
||||
| 7 | `Tia::recordsEdgesInWorkers()` + `WrapperRunner::handleTia()` inject `-d pcov.directory=<root>` into worker argv | `Tia.php`, `src/Plugins/Parallel/Paratest/WrapperRunner.php` |
|
||||
| 8 | Sequential record runs announce structural drift via `renderFreshGraph()` | `Tia.php` |
|
||||
| 9 | `Graph::getTime()` + `cachedTimeByTestId` + `resultTime()` preserve replayed durations; edge-less write guard is now `$recordsEdges = $complete && ($markKnownTestFiles \|\| $this->recordingActive)` | `Graph.php`, `Tia.php` |
|
||||
|
||||
Deliberately **not** done: `PLAN.md` §5 SIGINT propagation, §5 warning/deprecation `status=0`
|
||||
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
|
||||
|
||||
### Target-outcome changes this forces
|
||||
|
||||
Two rows in the original matrix asserted the **old**, buggy behaviour. Their targets are updated
|
||||
below — do not report them as regressions:
|
||||
|
||||
- **B10** was "`time` differs; statuses stable". Change 9 means replayed entries now **keep** their
|
||||
recorded `time`. New target: `time` differs only for tests that actually executed.
|
||||
- **B12** gains an edge-preservation assertion it never had (see K1).
|
||||
|
||||
### One known-failing repo test
|
||||
|
||||
`tests/Unit/Plugins/Tia/Graph.php:69-76` asserts `hasUnlocatedTestsToRerun('main')` is `false` for
|
||||
`tests/Feature/FooTest.php` under `new Graph(sys_get_temp_dir())` — a path that does not exist. That
|
||||
assertion encodes the I9 bug and **will fail** under change 1. It needs re-pointing at a root/file
|
||||
that exists (e.g. `dirname(__DIR__, 4)` + `'tests/Unit/Plugins/Tia/Graph.php'`). Left untouched by
|
||||
request; it is a repo-test matter, not a playground one.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Environment, and the traps in it
|
||||
|
||||
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel`
|
||||
|
||||
```bash
|
||||
cd /Users/nunomaduro/Work/projects/playground/laravel
|
||||
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
|
||||
```
|
||||
|
||||
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON
|
||||
when it detects an agent.
|
||||
2. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
|
||||
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock` —
|
||||
that a blanket reset would destroy. `PLAN.md`'s "reset ritual" is unsafe as written. Scope resets
|
||||
to what you touched: `git checkout -- tests app` / `git clean -fd tests app`, and for
|
||||
`composer.lock` (needed for the drift rows) **copy it aside and copy it back**, verifying with
|
||||
`shasum`.
|
||||
3. **`vendor/pestphp/pest` is a dist copy, not a symlink.** Composer installed
|
||||
`dev-fix/tia-filtered as 5.2.0`, so edits in the pest repo do **not** reach the playground.
|
||||
|
||||
**Tell Nuno whenever a sync is needed to move forward — do not sync silently.** Say what is stale
|
||||
and what the sync would be, then wait. This includes temporarily swapping in `db70017c` files for a
|
||||
before/after contrast. When you report any playground result, state which commit produced it.
|
||||
The sync itself, once he agrees:
|
||||
```bash
|
||||
PEST=/Users/nunomaduro/Work/projects/pestphp/pest
|
||||
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
|
||||
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \
|
||||
src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; done
|
||||
```
|
||||
Verify with `grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"` → `1`. **Always restore the
|
||||
current version before continuing** after a pre-fix contrast.
|
||||
4. **Coverage driver:** pcov only, no xdebug. `ini_get('pcov.directory')` is `''` by default — that
|
||||
emptiness is the entire mechanism behind G12.
|
||||
5. **Suite shape as found:** 7 tests in 5 files — `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,
|
||||
TrioTest}.php`, `tests/Feature/ExampleTest.php`. A healthy sequential graph is **`files=22`,
|
||||
5 edge keys, self-edge on every test** (`PLAN.md`'s `files=25` is stale). Pre-fix parallel gives
|
||||
`files=4` with zero self-edges. **These numbers shift the moment you add the Part 1b fixtures** —
|
||||
re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants
|
||||
that do *not* shift: sequential and parallel must agree, and every test must have a self-edge.
|
||||
6. **Sentinel patching is the only reliable discriminator** between "wrote identical values" and
|
||||
"wrote nothing": rewrite every cached result to `time=9.999 assertions=42`, snapshot, run the
|
||||
case, diff. A canary test absent from `edges` is unreliable under `--filter` because it never
|
||||
matches the filter and so never runs.
|
||||
7. Seeding a cached failure needs `--fresh` (or an env-driven flaky fixture): `--tia` on a clean
|
||||
green tree replays rather than executes, so it can never cache a failure. Working recipe — break
|
||||
an assertion, `pest --tia --fresh`, then restore the source.
|
||||
8. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
|
||||
9. The shell is zsh — build commands with arrays or `eval`; unquoted `$args` does not word-split.
|
||||
|
||||
### Graph summariser
|
||||
|
||||
Write this to a scratch path and use it for every diff.
|
||||
|
||||
```php
|
||||
<?php // summarise.php <graph.json> [label]
|
||||
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
|
||||
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
|
||||
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
|
||||
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
|
||||
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
|
||||
ksort($edges);
|
||||
foreach ($edges as $test => $ids) {
|
||||
$self = 'no';
|
||||
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
|
||||
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
|
||||
}
|
||||
foreach ($g['baselines'] ?? [] as $branch => $b) {
|
||||
$r = $b['results'] ?? [];
|
||||
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
|
||||
foreach ($r as $id => $x) {
|
||||
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
|
||||
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Normalised edge-set equality (for G12 / I5 / K1):
|
||||
|
||||
```php
|
||||
<?php // edgediff.php <a.json> <b.json>
|
||||
function edges(string $p): array {
|
||||
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
|
||||
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
|
||||
ksort($out); return $out;
|
||||
}
|
||||
$a = edges($argv[1]); $b = edges($argv[2]);
|
||||
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
|
||||
foreach ($a as $t => $s) {
|
||||
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
|
||||
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
|
||||
}
|
||||
```
|
||||
|
||||
### Per-case loop
|
||||
|
||||
```bash
|
||||
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
|
||||
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
|
||||
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
|
||||
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 1b — Fixtures you must build first
|
||||
|
||||
**The playground has none of the fixtures the matrix depends on.** Verified inventory: the only test
|
||||
files are `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php` and
|
||||
`tests/Feature/ExampleTest.php` — 7 plain tests, zero occurrences of `->group()`, `->only()`,
|
||||
`->skip()`, `->todo()`, `->note()`, `->flaky()`, `->covers()`, `->uses()`, `->issue()`, `->pr()`,
|
||||
`->ticket()`, `->assignee()`, datasets, or any `FLAKY_OK`-style env hook. `phpunit.xml` defines only
|
||||
the `Unit` and `Feature` testsuites.
|
||||
|
||||
So roughly 30 rows below cannot run as written until you create the fixtures. Build them **all up
|
||||
front, in one commit**, then re-derive the baseline graph shape once and use those numbers for the
|
||||
whole sweep — every fixture you add changes `files`, the `edges` key count and `n`, so adding them
|
||||
piecemeal invalidates earlier rows.
|
||||
|
||||
| Fixture to create | Rows that need it |
|
||||
|---|---|
|
||||
| a test with `->group('smoke')` | C7, C8, C9, C10 |
|
||||
| an env-driven flaky test (passes iff `FLAKY_OK=1`) | E1, E2, E3, E4 |
|
||||
| a `->skip()`ed test | E9, D8 |
|
||||
| a `->todo()` test | E10, C23, C39 |
|
||||
| a risky test (no assertions; pair with `--disallow-test-output`) | E11, D7 |
|
||||
| a test that triggers a PHPUnit warning | D6 |
|
||||
| a test calling `markTestIncomplete()` | D9 |
|
||||
| a test that triggers a deprecation | D10 |
|
||||
| a dataset test with ≥3 rows | E13, E14 |
|
||||
| `->only()` — added and removed per case, not left in | C31, C32, G7 |
|
||||
| `->covers(App\Services\Calculator::class)` | C18 |
|
||||
| `->uses(...)` / `UsesClass` annotation | C19 |
|
||||
| `->note(...)` | C24 |
|
||||
| `->flaky()` **annotation** (distinct from the env-driven flaky test above) | C25 |
|
||||
| `->issue(123)`, `->pr(1)`, `->ticket('X')`, `->assignee('X')` | C26, C27, C28, C37, C38 |
|
||||
|
||||
This is `PLAN.md` §6's "test-harness gaps to close before re-running", now itemised: without these,
|
||||
the listed rows' graph-invariant assertions match zero tests and **prove nothing** — they pass
|
||||
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous *only because* its fixture is
|
||||
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
|
||||
|
||||
Rows needing an *action* rather than a fixture — breaking `trio one` for the D rows, an uncommitted
|
||||
test edit for `--dirty`, branch renames and a non-git dir for H5–H10, `--mutate` against
|
||||
`app/Services` for the F rows — are fine as written; `pestphp/pest-plugin-mutate` is installed.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Full case matrix
|
||||
|
||||
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
|
||||
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
|
||||
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
|
||||
**HARD-SUPPRESSED** may change nothing.
|
||||
|
||||
Phase-1 column: **VERIFIED** = re-run against the fixed code in the phase-one session, pre-fix
|
||||
contrast captured · **PASS (sweep)** = passed in the original `db70017` sweep and *not* re-checked
|
||||
since the changes — these are the bulk of phase two's work · **SKIP** = not runnable as written.
|
||||
|
||||
### A — Setup & sanity
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS (sweep) |
|
||||
| A2 | `pest --baseline` | prints an existing dir; exit 0 | VERIFIED |
|
||||
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | VERIFIED |
|
||||
| A4 | `git status` after reset ritual | clean *except the four user-modified files* (see trap 2) | VERIFIED |
|
||||
| A5 | `extension_loaded("pcov")` | `true` | VERIFIED |
|
||||
| A6 | delete graph, plain `pest` | no graph created | PASS (sweep) |
|
||||
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS (sweep) |
|
||||
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | VERIFIED |
|
||||
|
||||
### B — COMPLETE runs still write
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | VERIFIED |
|
||||
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS (sweep) — recheck under change 9 |
|
||||
| B3 | `pest` (plain) | same as B2 | PASS (sweep) — recheck under change 9 |
|
||||
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | VERIFIED |
|
||||
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS (sweep) |
|
||||
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS (sweep) |
|
||||
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS (sweep) |
|
||||
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS (sweep) |
|
||||
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS (sweep) |
|
||||
| B10 | `pest --tia` twice | **UPDATED TARGET:** `time` differs only for tests that executed; replayed entries keep their recorded `time`; statuses stable | VERIFIED |
|
||||
| B11 | add a test file, `pest --tia` | new `edges` key **and** new result appear in the **same** run | VERIFIED (regression guard for change 9) |
|
||||
| B12 | `pest --tia --coverage` | completes; graph written; coverage report prints | VERIFIED |
|
||||
|
||||
### C — Selection narrowing → RESULTS-ONLY
|
||||
|
||||
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS (sweep) |
|
||||
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS (sweep) |
|
||||
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS (sweep) |
|
||||
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS (sweep) |
|
||||
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS (sweep) |
|
||||
| C7 | `pest --group=smoke` | RO; no notice | PASS (sweep) |
|
||||
| C8 | `pest --tia --group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS (sweep) |
|
||||
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS (sweep) |
|
||||
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS (sweep) |
|
||||
| C12 | `pest --tia tests/Unit` | RO; notice | PASS (sweep) |
|
||||
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS (sweep) |
|
||||
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS (sweep) |
|
||||
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS (sweep) |
|
||||
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS (sweep) |
|
||||
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS (sweep) |
|
||||
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS (sweep) |
|
||||
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
|
||||
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS (sweep) |
|
||||
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS (sweep) |
|
||||
| C22 | `pest --tia --dirty` | RO; notice | PASS (sweep) |
|
||||
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
|
||||
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
|
||||
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
|
||||
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
|
||||
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
|
||||
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
|
||||
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS (sweep) |
|
||||
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
|
||||
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS (sweep) |
|
||||
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS (sweep) |
|
||||
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS (sweep) |
|
||||
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS (sweep) |
|
||||
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS (sweep) |
|
||||
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
|
||||
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
|
||||
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
|
||||
|
||||
### D — Truncation → RESULTS-ONLY
|
||||
|
||||
D1–D5, D11–D13, D16 precondition: `trio one` broken. D6–D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS (sweep) |
|
||||
| D2 | `pest --retry` | RO; siblings survive | PASS (sweep) |
|
||||
| D3 | `pest --stop-on-failure` | RO | PASS (sweep) |
|
||||
| D4 | `pest --stop-on-defect` | RO | PASS (sweep) |
|
||||
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS (sweep) |
|
||||
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS (sweep) |
|
||||
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS (sweep) |
|
||||
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS (sweep) |
|
||||
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS (sweep) |
|
||||
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS (sweep) |
|
||||
| D11 | `pest --tia --bail` | RO | PASS (sweep) |
|
||||
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS (sweep) |
|
||||
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS (sweep) |
|
||||
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
|
||||
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
|
||||
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS (sweep) |
|
||||
|
||||
### E — Result merge semantics
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS (sweep) |
|
||||
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS (sweep) |
|
||||
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| E5 | any partial run | `assertions` and `time` update for the test that ran (it executed, so change 9 does not apply) | PASS (sweep) |
|
||||
| E6 | any partial run | `message` of untouched tests unchanged | PASS (sweep) |
|
||||
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
|
||||
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS (sweep) |
|
||||
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS (sweep) |
|
||||
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS (sweep) |
|
||||
| E11 | partial run of a risky test | `status=5` | PASS (sweep) |
|
||||
| E12 | any partial run | `fingerprint` byte-identical | PASS (sweep) |
|
||||
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS (sweep) |
|
||||
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS (sweep) |
|
||||
|
||||
### F — Guard rails
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | VERIFIED |
|
||||
| F2 | complete `pest --tia` first, then the same filter | result is recorded | VERIFIED |
|
||||
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS (sweep) |
|
||||
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
|
||||
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS (sweep) |
|
||||
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS (sweep) |
|
||||
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS (sweep) |
|
||||
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS (sweep) |
|
||||
|
||||
### G — Parallel
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written; all 7 results present | VERIFIED |
|
||||
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS (sweep) |
|
||||
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS (sweep) |
|
||||
| G4 | `pest --tia --parallel --bail` | RO | PASS (sweep) |
|
||||
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | VERIFIED (see I5) |
|
||||
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | VERIFIED |
|
||||
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS (sweep) |
|
||||
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS (sweep) |
|
||||
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS (sweep) |
|
||||
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS (sweep) |
|
||||
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | VERIFIED |
|
||||
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** — `files=22`, self-edge on all 5 tests, both | **VERIFIED (fixed)** — pre-fix: `files=4`, 0 self-edges |
|
||||
|
||||
### H — Baseline key / branch resolution
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| H1 | `pest --tia` on `master` | only a `master` key | VERIFIED |
|
||||
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS (sweep) |
|
||||
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS (sweep) |
|
||||
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS (sweep) |
|
||||
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS (sweep) |
|
||||
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS (sweep) |
|
||||
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
|
||||
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS (sweep) |
|
||||
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS (sweep) |
|
||||
|
||||
### I — Filtered mode
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS (sweep) |
|
||||
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | VERIFIED |
|
||||
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS (sweep) |
|
||||
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS (sweep) |
|
||||
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **VERIFIED (fixed)** — 1 affected test, edges identical |
|
||||
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS (sweep) |
|
||||
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS (sweep) |
|
||||
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report → full suite runs | **VERIFIED (fixed)** — pre-fix: `No affected tests found` |
|
||||
| I9 | cached failure whose test file was deleted | WARN `Some cached tests due a re-run could not be located on disk` + `Running the full suite with replay instead of a filtered run` | **VERIFIED (fixed)** — pre-fix: `No tests found`, exit 0 green |
|
||||
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS (sweep) |
|
||||
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
|
||||
| I12 | edit `composer.lock` | fingerprint drift → full rebuild, **and the reason is printed sequentially**: `fresh graph (composer.lock changed)` | **VERIFIED (fixed)** — pre-fix: bare `Running in TIA mode.` |
|
||||
|
||||
### J — Interactions & regressions
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS (sweep) |
|
||||
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS (sweep) |
|
||||
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS (sweep) |
|
||||
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS (sweep) |
|
||||
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
|
||||
| J6 | `pest --tia --compact` | COMPLETE | PASS (sweep) |
|
||||
| J7 | `pest --tia -v` | COMPLETE | PASS (sweep) |
|
||||
| J8 | `pest --tia --profile` | COMPLETE | PASS (sweep) |
|
||||
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS (sweep) |
|
||||
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS (sweep) |
|
||||
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
|
||||
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS (sweep) |
|
||||
| J13 | `pest --tia --min=50` | COMPLETE (silent no-op without `--coverage`) | PASS (sweep) |
|
||||
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
|
||||
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS (sweep) |
|
||||
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS (sweep) |
|
||||
|
||||
### K — New rows for the phase-one fixes
|
||||
|
||||
These assert behaviour no original row covered. All were verified in phase one; re-run them as
|
||||
regression guards.
|
||||
|
||||
| # | Case | Target outcome | Phase 1 |
|
||||
|---|---|---|---|
|
||||
| K1 | healthy `pest --tia` graph, then `pest --tia --coverage` | `edges` **byte-identical** — piggyback data may seed empty sets, never narrow populated ones; `files=22` stays `22` | **VERIFIED (fixed)** — pre-fix: `ExampleTest` 16→2 edges, self-edges lost |
|
||||
| K2 | same run's headline | `Experimental TIA mode enabled / recording a coverage baseline.` — no false `fresh graph` | **VERIFIED (fixed)** |
|
||||
| K3 | `pest --tia` ×3 on a clean tree | replayed entries keep their recorded `time` across all three | **VERIFIED (fixed)** — pre-fix: `0.053 → 0.001 → 0.001 → 0.001` |
|
||||
| K4 | add a test file, one `pest --tia` (graph exists, fingerprint matches → replay+refresh) | result **and** edges appear in that same run; `n` grows by 1 | **VERIFIED** — regression guard for change 9 |
|
||||
| K5 | add a test file, plain `pest --no-tia` | no result written for it; `n` unchanged; no edges | **VERIFIED (fixed)** — pre-fix: edge-less result inflated `n` |
|
||||
| K6 | `pest --tia --parallel` worker argv | carries `-d pcov.directory=<projectRoot>`; `PEST_TIA` unset; not re-exec'd | **VERIFIED** — probe `bin/worker.php` |
|
||||
| K7 | second `pest --tia --coverage` (cache primed → replay) | recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — **new work** |
|
||||
| K8 | `pest --tia --parallel --coverage` | workers read `TIA_PIGGYBACK_COVERAGE`; no widened pcov scope; report intact | Not yet measured — **new work** |
|
||||
| K9 | `pest --tia --filtered --coverage-html=<dir>` / `--coverage-clover=<file>` | filtered mode off, same as I8, for every flag in `COVERAGE_REPORT_FLAGS` | Not yet measured — **new work** |
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Priorities for phase two
|
||||
|
||||
0. **Part 1b fixtures** — nothing in C, D6–D10, or E9–E14 means anything until they exist.
|
||||
1. **K7, K8, K9** — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
|
||||
reasoned about but not observed; K9 covers the seven `COVERAGE_REPORT_FLAGS` beyond
|
||||
`--coverage-text`.
|
||||
2. **B2, B3, F4** and all of **D** and **E** — change 9 touched the shared result-write path, and
|
||||
these are the rows that exercise it hardest. `$recordsEdges` is the thing to falsify: it must be
|
||||
false for every partial and every non-recording run.
|
||||
3. **F3–F8** — change 7 injects a `-d` into worker argv; `--mutate --parallel` is the one place that
|
||||
both spawns workers and must write nothing.
|
||||
4. **H1–H10** — untouched by these changes; cheapest bulk confirmation.
|
||||
5. Raw PHPUnit coverage flags print **no report at all** in Pest, with or without TIA
|
||||
(`--no-tia --coverage-text` is equally silent). Pre-existing, unrelated to change 6 — do not
|
||||
chase it as a regression, but it means I8/K9 can only assert the selection half.
|
||||
|
||||
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
|
||||
failure the pre-fix contrast (write `git show db70017c:<file>` into vendor, re-run, restore) so a
|
||||
regression is told apart from a pre-existing defect.
|
||||
@@ -35,6 +35,7 @@ We cannot thank our sponsors enough for their incredible support in funding Pest
|
||||
- **[SerpApi](https://serpapi.com/?ref=nunomaduro)**
|
||||
- **[Typesense](https://typesense.org/?ref=nunomaduro)**
|
||||
- **[Bento](https://bentonow.com/?ref=nunomaduro)**
|
||||
- **[Redberry](https://redberry.international/laravel-development/)**
|
||||
- **[Pixel](https://wearepixel.com.au/?ref=nunomaduro)**
|
||||
- **[Redberry](https://redberry.international/laravel-development/?ref=nunomaduro)**
|
||||
|
||||
Pest is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.
|
||||
|
||||
+4
-4
@@ -23,7 +23,7 @@
|
||||
"nunomaduro/termwind": "^2.4.0",
|
||||
"pestphp/pest-plugin": "^5.0.0",
|
||||
"pestphp/pest-plugin-arch": "^5.0.0",
|
||||
"pestphp/pest-plugin-mutate": "^5.0.0",
|
||||
"pestphp/pest-plugin-mutate": "^5.0.1",
|
||||
"pestphp/pest-plugin-profanity": "^5.0.0",
|
||||
"phpunit/phpunit": "^13.2.6",
|
||||
"symfony/process": "^8.1.0"
|
||||
@@ -48,6 +48,7 @@
|
||||
"Tests\\Fixtures\\Covers\\": "tests/Fixtures/Covers",
|
||||
"Tests\\Fixtures\\Inheritance\\": "tests/Fixtures/Inheritance",
|
||||
"Tests\\Fixtures\\Arch\\": "tests/Fixtures/Arch",
|
||||
"Tests\\Fixtures\\Tia\\": "tests/Fixtures/Tia",
|
||||
"Tests\\": "tests/PHPUnit/"
|
||||
},
|
||||
"classmap": [
|
||||
@@ -58,12 +59,11 @@
|
||||
]
|
||||
},
|
||||
"require-dev": {
|
||||
"laravel/pao": "^1.1.3",
|
||||
"pestphp/pest-dev-tools": "^5.0.0",
|
||||
"pestphp/pest-plugin-browser": "^5.0.0",
|
||||
"pestphp/pest-plugin-phpstan": "^5.0.0",
|
||||
"pestphp/pest-plugin-rector": "^5.0.0",
|
||||
"pestphp/pest-plugin-type-coverage": "^5.0.0",
|
||||
"pestphp/pest-plugin-rector": "^5.0.2",
|
||||
"pestphp/pest-plugin-type-coverage": "^5.0.2",
|
||||
"psy/psysh": "^0.12.24"
|
||||
},
|
||||
"minimum-stability": "dev",
|
||||
|
||||
@@ -90,11 +90,11 @@ final class TestSuiteLoader
|
||||
(static function () use ($suiteClassFile) {
|
||||
try {
|
||||
include_once $suiteClassFile;
|
||||
|
||||
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
|
||||
} catch (Throwable $e) {
|
||||
Panic::with($e);
|
||||
}
|
||||
|
||||
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
|
||||
})();
|
||||
|
||||
$loadedClasses = array_values(
|
||||
|
||||
+2
-1
@@ -18,8 +18,9 @@
|
||||
<directory suffix=".php">./tests</directory>
|
||||
<directory suffix=".php">./tests-external</directory>
|
||||
<exclude>./tests/.snapshots</exclude>
|
||||
<exclude>./tests/.tests</exclude>
|
||||
<exclude>./tests/Fixtures/Inheritance</exclude>
|
||||
<exclude>./tests/Fixtures/Suites</exclude>
|
||||
<exclude>./tests/Fixtures/Tia</exclude>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
<source>
|
||||
|
||||
@@ -28,6 +28,9 @@ return RectorConfig::configure()
|
||||
->withSkip([
|
||||
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
|
||||
__DIR__.'/tests/Fixtures/Arch',
|
||||
// Fixture suites are pinned by the TeamCity / JUnit snapshots, down to
|
||||
// the line numbers — rewriting their source would break them.
|
||||
__DIR__.'/tests/Fixtures/Suites',
|
||||
ReturnNeverTypeRector::class,
|
||||
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
|
||||
NarrowObjectReturnTypeRector::class,
|
||||
|
||||
@@ -280,12 +280,17 @@ trait Testable
|
||||
|
||||
/** @var Tia $tia */
|
||||
$tia = Container::getInstance()->get(Tia::class);
|
||||
$status = $tia->getStatus(self::$__filename, $this::class.'::'.$this->name());
|
||||
$status = $tia->getStatus(self::$__filename, $this->valueObjectForEvents()->id());
|
||||
$replay = ReplayType::fromStatus($status);
|
||||
|
||||
if ($replay !== ReplayType::None) {
|
||||
assert($status !== null);
|
||||
|
||||
// Marks the replay before the branches below throw, so `tearDown`
|
||||
// short-circuits for every replayed result — the throwing branches
|
||||
// never reach `parent::setUp`, so no user hook may run after them.
|
||||
$this->__replay = $replay;
|
||||
|
||||
match ($replay) {
|
||||
ReplayType::Pass, ReplayType::Risky => $this->__beginReplay($replay, $tia),
|
||||
ReplayType::Skipped => $this->markTestSkipped($status->message()),
|
||||
@@ -319,7 +324,7 @@ trait Testable
|
||||
private function __beginReplay(ReplayType $replay, Tia $tia): void
|
||||
{
|
||||
$this->__replay = $replay;
|
||||
$this->__replayAssertions = $tia->getAssertionCount($this::class.'::'.$this->name());
|
||||
$this->__replayAssertions = $tia->getAssertionCount($this->valueObjectForEvents()->id());
|
||||
$this->__ran = true;
|
||||
}
|
||||
|
||||
|
||||
@@ -19,6 +19,6 @@ final class AfterAllAlreadyExist extends InvalidArgumentException implements Exc
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct(sprintf('The afterAll already exists in the filename `%s`.', $filename));
|
||||
parent::__construct(sprintf('The afterAll already exists in the filename [%s].', $filename));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class AfterAllWithinDescribe extends InvalidArgumentException implements E
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename `%s`.', $filename));
|
||||
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename [%s].', $filename));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class AfterBeforeTestFunction extends InvalidArgumentException implements
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct('After method cannot be used with before the [test|it] functions in the filename `['.$filename.']`.');
|
||||
parent::__construct('After method cannot be used with before the [test|it] functions in the filename ['.$filename.'].');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class BeforeAllAlreadyExist extends InvalidArgumentException implements Ex
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct(sprintf('The beforeAll already exists in the filename `%s`.', $filename));
|
||||
parent::__construct(sprintf('The beforeAll already exists in the filename [%s].', $filename));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class BeforeAllWithinDescribe extends InvalidArgumentException implements
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename `%s`.', $filename));
|
||||
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename [%s].', $filename));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class DatasetAlreadyExists extends InvalidArgumentException implements Exc
|
||||
*/
|
||||
public function __construct(string $name, string $scope)
|
||||
{
|
||||
parent::__construct(sprintf('A dataset with the name `%s` already exists in scope [%s].', $name, $scope));
|
||||
parent::__construct(sprintf('A dataset with the name [%s] already exists in scope [%s].', $name, $scope));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class DatasetDoesNotExist extends InvalidArgumentException implements Exce
|
||||
*/
|
||||
public function __construct(string $name)
|
||||
{
|
||||
parent::__construct(sprintf("A dataset with the name `%s` does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
|
||||
parent::__construct(sprintf("A dataset with the name [%s] does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class FileOrFolderNotFound extends InvalidArgumentException implements Exc
|
||||
*/
|
||||
public function __construct(string $filename)
|
||||
{
|
||||
parent::__construct(sprintf('The file or folder with the name `%s` could not be found.', $filename));
|
||||
parent::__construct(sprintf('The file or folder with the name [%s] could not be found.', $filename));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Exceptions;
|
||||
|
||||
use InvalidArgumentException;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessEditor;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessTrace;
|
||||
use Symfony\Component\Console\Exception\ExceptionInterface;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class InvalidTestClassName extends InvalidArgumentException implements ExceptionInterface, RenderlessEditor, RenderlessTrace
|
||||
{
|
||||
/**
|
||||
* Creates a new Exception instance for the given class name.
|
||||
*/
|
||||
public static function fromClassName(string $filename, string $className): self
|
||||
{
|
||||
return new self(sprintf(
|
||||
'The test file [%s] would create the class [%s], which is not a valid PHP class name. Please rename the test file.',
|
||||
$filename,
|
||||
$className,
|
||||
));
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new Exception instance for the given namespace.
|
||||
*/
|
||||
public static function fromNamespace(string $filename, string $namespace, string $part): self
|
||||
{
|
||||
return new self(sprintf(
|
||||
'The test file [%s] would create the namespace [%s], which is not a valid PHP namespace, as [%s] may not be used as a namespace name. Please rename the folder in question.',
|
||||
$filename,
|
||||
$namespace,
|
||||
$part,
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -19,6 +19,6 @@ final class TestAlreadyExist extends InvalidArgumentException implements Excepti
|
||||
*/
|
||||
public function __construct(string $fileName, string $description)
|
||||
{
|
||||
parent::__construct(sprintf('A test with the description `%s` already exists in the filename `%s`.', $description, $fileName));
|
||||
parent::__construct(sprintf('A test with the description [%s] already exists in the filename [%s].', $description, $fileName));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class TestCaseClassOrTraitNotFound extends InvalidArgumentException implem
|
||||
*/
|
||||
public function __construct(string $testCaseClass)
|
||||
{
|
||||
parent::__construct(sprintf('The class `%s` was not found.', $testCaseClass));
|
||||
parent::__construct(sprintf('The class [%s] was not found.', $testCaseClass));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ final class TestDescriptionMissing extends InvalidArgumentException implements E
|
||||
*/
|
||||
public function __construct(string $fileName)
|
||||
{
|
||||
parent::__construct(sprintf('Test description is missing in the filename `%s`.', $fileName));
|
||||
parent::__construct(sprintf('Test description is missing in the filename [%s].', $fileName));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Exceptions;
|
||||
|
||||
use NunoMaduro\Collision\Contracts\RenderlessEditor;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessTrace;
|
||||
use Pest\Contracts\Panicable;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Console\Exception\ExceptionInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class TiaRequiresDefaultBranch extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
|
||||
{
|
||||
public function __construct()
|
||||
{
|
||||
parent::__construct(
|
||||
'Tia mode could not determine the default branch every other branch falls back to reading.',
|
||||
);
|
||||
}
|
||||
|
||||
public function render(OutputInterface $output): void
|
||||
{
|
||||
$output->writeln([
|
||||
'',
|
||||
' <fg=white;options=bold;bg=red> ERROR </> Tia mode could not determine the default branch.',
|
||||
'',
|
||||
' It is the branch whose baseline every other branch falls back to reading, and',
|
||||
' nothing in this checkout names it: the repository has a remote, but no',
|
||||
' <fg=yellow>origin/HEAD</>, and no CI provider stated it either. Guessing would re-run the',
|
||||
' whole suite on every new branch while reporting it as a cache hit.',
|
||||
'',
|
||||
' Name the branch in <fg=yellow>tests/Pest.php</>:',
|
||||
'',
|
||||
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
|
||||
'',
|
||||
' Or let git answer, once per clone:',
|
||||
'',
|
||||
' <fg=yellow>git remote set-head origin --auto</>',
|
||||
'',
|
||||
]);
|
||||
}
|
||||
|
||||
public function exitCode(): int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Exceptions;
|
||||
|
||||
use NunoMaduro\Collision\Contracts\RenderlessEditor;
|
||||
use NunoMaduro\Collision\Contracts\RenderlessTrace;
|
||||
use Pest\Contracts\Panicable;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Console\Exception\ExceptionInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class TiaRequiresRemote extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
|
||||
{
|
||||
public function __construct()
|
||||
{
|
||||
parent::__construct(
|
||||
'Tia mode requires a repository with a remote, so the default branch every other branch falls back to reading can be resolved.',
|
||||
);
|
||||
}
|
||||
|
||||
public function render(OutputInterface $output): void
|
||||
{
|
||||
$output->writeln([
|
||||
'',
|
||||
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires a repository with a remote.',
|
||||
'',
|
||||
' Without one there is no way to tell which branch is the default — the branch',
|
||||
' whose baseline every other branch falls back to reading — so the first run on',
|
||||
' each new branch would silently re-run the whole suite.',
|
||||
'',
|
||||
' Add a remote, or name the branch yourself in <fg=yellow>tests/Pest.php</>:',
|
||||
'',
|
||||
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
|
||||
'',
|
||||
]);
|
||||
}
|
||||
|
||||
public function exitCode(): int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,7 @@ use Pest\Concerns;
|
||||
use Pest\Contracts\HasPrintableTestCaseName;
|
||||
use Pest\Evaluators\Attributes;
|
||||
use Pest\Exceptions\DatasetMissing;
|
||||
use Pest\Exceptions\InvalidTestClassName;
|
||||
use Pest\Exceptions\ShouldNotHappen;
|
||||
use Pest\Exceptions\TestAlreadyExist;
|
||||
use Pest\Exceptions\TestClosureMustNotBeStatic;
|
||||
@@ -138,6 +139,16 @@ final class TestCaseFactory
|
||||
|
||||
if (trim($className) === '') {
|
||||
$className = 'InvalidTestName'.Str::random();
|
||||
} elseif (! Str::isValidClassName($className)) {
|
||||
throw InvalidTestClassName::fromClassName($this->filename, $className);
|
||||
}
|
||||
|
||||
if ($this->namespace === null) {
|
||||
foreach ($partsFQN as $partFQN) {
|
||||
if (! Str::isValidIdentifier($partFQN)) {
|
||||
throw InvalidTestClassName::fromNamespace($this->filename, $namespace, $partFQN);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$this->attributes = [
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ namespace Pest;
|
||||
|
||||
function version(): string
|
||||
{
|
||||
return '5.0.2';
|
||||
return '5.0.3';
|
||||
}
|
||||
|
||||
function testDirectory(string $file = ''): string
|
||||
|
||||
@@ -14,6 +14,7 @@ use ParaTest\RunnerInterface;
|
||||
use ParaTest\WrapperRunner\MissingResultsException;
|
||||
use ParaTest\WrapperRunner\SuiteLoader;
|
||||
use ParaTest\WrapperRunner\WrapperWorker;
|
||||
use Pest\Plugins\Tia;
|
||||
use Pest\Result;
|
||||
use Pest\TestSuite;
|
||||
use PHPUnit\Event\Facade as EventFacade;
|
||||
@@ -155,6 +156,7 @@ final class WrapperRunner implements RunnerInterface
|
||||
|
||||
/** @var array<int, non-empty-string> $parameters */
|
||||
$parameters = $this->handleLaravelHerd($parameters);
|
||||
$parameters = $this->handleTia($parameters);
|
||||
|
||||
$parameters[] = $wrapper;
|
||||
$parameters[] = '--test-directory='.TestSuite::getInstance()->testPath;
|
||||
@@ -202,6 +204,28 @@ final class WrapperRunner implements RunnerInterface
|
||||
return $parameters;
|
||||
}
|
||||
|
||||
/**
|
||||
* Widens pcov's instrumentation scope to the whole project for workers that
|
||||
* record TIA edges.
|
||||
*
|
||||
* pcov's default scope is a single source directory it auto-detects, so
|
||||
* `config/`, `routes/`, `bootstrap/` and every test's own file never reach
|
||||
* the recorder — a worker-recorded graph selects a fraction of what a
|
||||
* sequential one does. `pcov.directory` is only settable at startup, hence
|
||||
* the command line rather than an `ini_set()` inside the worker.
|
||||
*
|
||||
* @param array<int, non-empty-string> $parameters
|
||||
* @return array<int, non-empty-string>
|
||||
*/
|
||||
private function handleTia(array $parameters): array
|
||||
{
|
||||
if (! Tia::recordsEdgesInWorkers()) {
|
||||
return $parameters;
|
||||
}
|
||||
|
||||
return array_merge($parameters, ['-d', 'pcov.directory='.TestSuite::getInstance()->rootPath]);
|
||||
}
|
||||
|
||||
private function startWorkers(): void
|
||||
{
|
||||
for ($token = 1; $token <= $this->options->processes; $token++) {
|
||||
|
||||
+574
-25
@@ -7,13 +7,19 @@ namespace Pest\Plugins;
|
||||
use NunoMaduro\Collision\Adapters\Phpunit\Printers\DefaultPrinter;
|
||||
use Pest\Contracts\Plugins\AddsOutput;
|
||||
use Pest\Contracts\Plugins\HandlesArguments;
|
||||
use Pest\Contracts\Plugins\HandlesOriginalArguments;
|
||||
use Pest\Contracts\Plugins\Terminable;
|
||||
use Pest\Exceptions\InvalidOption;
|
||||
use Pest\Exceptions\MissingDependency;
|
||||
use Pest\Exceptions\NoAffectedTestsFound;
|
||||
use Pest\Exceptions\TiaRequiresDefaultBranch;
|
||||
use Pest\Exceptions\TiaRequiresRemote;
|
||||
use Pest\Exceptions\TiaRequiresRepositoryRoot;
|
||||
use Pest\Panic;
|
||||
use Pest\Plugins\Concerns\HandleArguments;
|
||||
use Pest\Plugins\Tia\BaselineSync;
|
||||
use Pest\Plugins\Tia\ChangedFiles;
|
||||
use Pest\Plugins\Tia\CiDefaultBranch;
|
||||
use Pest\Plugins\Tia\Contracts\State;
|
||||
use Pest\Plugins\Tia\CoverageCollector;
|
||||
use Pest\Plugins\Tia\Fingerprint;
|
||||
@@ -30,13 +36,14 @@ use Pest\Support\View;
|
||||
use Pest\TestCaseFilters\TiaTestCaseFilter;
|
||||
use Pest\TestSuite;
|
||||
use PHPUnit\Framework\TestStatus\TestStatus;
|
||||
use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* @internal
|
||||
*/
|
||||
final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArguments, Terminable
|
||||
{
|
||||
use HandleArguments;
|
||||
|
||||
@@ -56,6 +63,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
private const string BASELINE_PATH_OPTION = '--baseline';
|
||||
|
||||
/**
|
||||
* Set by the mutation plugin on the subprocess running a single mutant,
|
||||
* and nowhere else. Its own `--mutate` flag is popped before the argv is
|
||||
* handed to that subprocess, so the flag cannot be matched instead.
|
||||
*/
|
||||
private const string ENV_MUTATION_TESTING = 'PEST_MUTATION_TESTING';
|
||||
|
||||
private const string ENV_TIA = 'PEST_TIA';
|
||||
|
||||
private const string ENV_FILTERED = 'PEST_TIA_FILTERED';
|
||||
@@ -88,6 +102,21 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
private const string PIGGYBACK_COVERAGE_GLOBAL = 'TIA_PIGGYBACK_COVERAGE';
|
||||
|
||||
/**
|
||||
* The parent's resolved fallback branch, handed to the workers.
|
||||
*
|
||||
* A worker cannot resolve it for itself: the restarters run before
|
||||
* `tests/Pest.php` is loaded, so a `defaultBranch()` declared there is
|
||||
* invisible to it — and autodetecting again would spend a git call per
|
||||
* worker to reach the answer the parent already has.
|
||||
*/
|
||||
private const string FALLBACK_BRANCH_GLOBAL = 'TIA_FALLBACK_BRANCH';
|
||||
|
||||
/**
|
||||
* The branch assumed when a repository cannot name its own default.
|
||||
*/
|
||||
private const string DEFAULT_BRANCH = 'main';
|
||||
|
||||
/**
|
||||
* PHPUnit/Pest CLI flags whose subsequent argument is a value, not a path.
|
||||
*
|
||||
@@ -101,12 +130,76 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
'--include-path', '--whitelist',
|
||||
'--log-junit', '--log-teamcity', '--testdox-html', '--testdox-text',
|
||||
'--coverage-clover', '--coverage-cobertura', '--coverage-crap4j',
|
||||
'--coverage-html', '--coverage-php', '--coverage-text', '--coverage-xml',
|
||||
'--coverage-html', '--coverage-openclover', '--coverage-php',
|
||||
'--coverage-text', '--coverage-xml',
|
||||
'--coverage-filter', '--path-coverage',
|
||||
'--repeat', '--retry-times', '--memory-limit', '--seed',
|
||||
'--compact', '--ci-build-id', '--min',
|
||||
];
|
||||
|
||||
/**
|
||||
* PHPUnit flags that make this run produce a coverage report.
|
||||
*
|
||||
* Pest's own `--coverage` is tracked by the Coverage plugin, but a raw
|
||||
* PHPUnit report flag never reaches it. A run that reports coverage must
|
||||
* not be narrowed to the affected tests — the report would then describe a
|
||||
* subset of the suite — and must let PHPUnit own the coverage driver rather
|
||||
* than have the TIA recorder clear it mid-collection.
|
||||
*
|
||||
* Flags that only shape collection or an existing report — `--coverage-filter`,
|
||||
* `--path-coverage`, `--warm-coverage-cache`, `--only-summary-for-coverage-text`,
|
||||
* `--show-uncovered-for-coverage-text`, `--disable-coverage-ignore` — produce no
|
||||
* report on their own, so they are deliberately absent.
|
||||
*
|
||||
* @var list<string>
|
||||
*/
|
||||
private const array COVERAGE_REPORT_FLAGS = [
|
||||
'--coverage-clover', '--coverage-cobertura', '--coverage-crap4j',
|
||||
'--coverage-html', '--coverage-openclover', '--coverage-php',
|
||||
'--coverage-text', '--coverage-xml',
|
||||
];
|
||||
|
||||
/**
|
||||
* Flags that narrow this run to a subset of the suite.
|
||||
*
|
||||
* Only user-supplied, per-run narrowing belongs here. A filter that is
|
||||
* always in force — `<groups>` in phpunit.xml, or a plugin registering a
|
||||
* test case filter from `boot()` — applies equally to the runs that build
|
||||
* the baseline, so it does not make this run narrower than the baseline
|
||||
* and must not disable baseline writes.
|
||||
*
|
||||
* `--shard` is rewritten to `--filter` before this plugin sees the
|
||||
* arguments, so it is covered here too. The `bin/pest`-only flags are
|
||||
* stripped from the handled arguments, so they are matched against the
|
||||
* original argv instead.
|
||||
*
|
||||
* Flags that cut a run short instead of narrowing it — `--bail`, `--retry`,
|
||||
* `--stop-on-*` — do not belong here either. They only narrow the run when
|
||||
* something actually fails, and that is not known until it is over, so they
|
||||
* are handled by stoppedEarly() from addOutput().
|
||||
*
|
||||
* @var list<string>
|
||||
*/
|
||||
private const array PARTIAL_SELECTION_FLAGS = [
|
||||
'--filter', '--exclude-filter', '--group', '--exclude-group',
|
||||
'--covers', '--uses', '--testsuite', '--exclude-testsuite', '--test-suffix',
|
||||
'--dirty', '--todo', '--todos', '--flaky', '--notes',
|
||||
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
|
||||
];
|
||||
|
||||
/**
|
||||
* Options that cannot be combined with Tia mode.
|
||||
*
|
||||
* `--covers` and `--uses` select on coverage metadata Tia does not model,
|
||||
* so they resolve to no tests at all rather than to the ones the user meant.
|
||||
* `--random-order-seed` exits non-zero on its own, with or without Tia.
|
||||
* Either way the run cannot honour both things it was asked for, so it says
|
||||
* so instead of silently dropping Tia and running something else.
|
||||
*/
|
||||
private const array UNSUPPORTED_OPTIONS = [
|
||||
'--covers', '--uses', '--random-order-seed',
|
||||
];
|
||||
|
||||
private bool $graphWritten = false;
|
||||
|
||||
private bool $replayRan = false;
|
||||
@@ -120,9 +213,50 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
/** @var array<string, int> */
|
||||
private array $cachedAssertionsByTestId = [];
|
||||
|
||||
/**
|
||||
* Recorded durations of the tests this run replayed rather than executed.
|
||||
*
|
||||
* A replayed test never runs, so the duration PHPUnit reports for it is the
|
||||
* cost of replaying it — near zero. Writing that back would decay every
|
||||
* cached timing toward zero one run at a time.
|
||||
*
|
||||
* @var array<string, float>
|
||||
*/
|
||||
private array $cachedTimeByTestId = [];
|
||||
|
||||
private ?Graph $replayGraph = null;
|
||||
|
||||
private string $branch = 'main';
|
||||
/**
|
||||
* The baseline this run reads from and writes to.
|
||||
*
|
||||
* The repository's default branch is only the fallback for a checkout whose
|
||||
* branch cannot be read — a detached HEAD. It is also the branch every
|
||||
* other baseline falls back to reading, so writing there by accident
|
||||
* corrupts the shared baseline. Resolved through resolveBranch() rather
|
||||
* than at every use site, because the git call it needs is not free.
|
||||
*/
|
||||
private string $branch = self::DEFAULT_BRANCH;
|
||||
|
||||
/**
|
||||
* The baseline branches with none of their own read from.
|
||||
*
|
||||
* Read-only, and the whole point of the exercise: without it the first run
|
||||
* on every new branch re-runs a suite whose results the default branch
|
||||
* already holds.
|
||||
*/
|
||||
private string $fallbackBranch = self::DEFAULT_BRANCH;
|
||||
|
||||
/**
|
||||
* Whether anything actually named the branch above.
|
||||
*
|
||||
* When nothing did, the value is a guess, and a guess is what the TIA path
|
||||
* refuses to run on: an unresolved fallback reads no baseline at all, which
|
||||
* looks exactly like a hit in the output. Runs that never asked for TIA
|
||||
* still have to write somewhere, so the guess stands for them.
|
||||
*/
|
||||
private bool $fallbackBranchResolved = false;
|
||||
|
||||
private bool $branchResolved = false;
|
||||
|
||||
/** @var array<string, true> */
|
||||
private array $affectedFiles = [];
|
||||
@@ -142,6 +276,27 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
private bool $filteredMode = false;
|
||||
|
||||
/**
|
||||
* Bars this run from touching the graph at all, results included.
|
||||
*
|
||||
* Reserved for runs whose results describe something other than the code
|
||||
* in the working tree, which is nothing the baseline can ever use.
|
||||
*/
|
||||
private bool $writesSuppressed = false;
|
||||
|
||||
/**
|
||||
* Narrows this run's writes to the results of the tests it actually ran.
|
||||
*
|
||||
* A run that covered only part of the suite still learns something true
|
||||
* about the tests it did reach. What it cannot do is speak for the rest:
|
||||
* pruning results, advancing the recorded sha and replacing the edge map
|
||||
* all claim the whole suite reported, so they stay behind a complete run.
|
||||
*/
|
||||
private bool $resultsOnlyWrites = false;
|
||||
|
||||
/** @var array<int, string> */
|
||||
private array $originalArguments = [];
|
||||
|
||||
private ?string $driftLabel = null;
|
||||
|
||||
private ?string $driftDetails = null;
|
||||
@@ -185,7 +340,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return null;
|
||||
}
|
||||
|
||||
return Graph::decode($json, $projectRoot);
|
||||
$graph = Graph::decode($json, $projectRoot);
|
||||
|
||||
// Every read of a baseline goes through a graph loaded here, so this is
|
||||
// the one place the resolved fallback has to reach.
|
||||
$graph?->setFallbackBranch($this->fallbackBranch);
|
||||
|
||||
return $graph;
|
||||
}
|
||||
|
||||
private function saveGraph(Graph $graph): bool
|
||||
@@ -227,6 +388,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return ! self::argumentPresent('--ci', $arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the workers of this run record their own coverage edges.
|
||||
*
|
||||
* Stamped by the parent before paratest spawns anything, because a worker
|
||||
* cannot tell on its own: its argv carries no `--tia`, and the restarters
|
||||
* run before `tests/Pest.php` is loaded, so {@see self::isEnabledForRun()}
|
||||
* sees an empty {@see WatchPatterns} too. Left unanswered, pcov keeps its
|
||||
* default scope — a single auto-detected source directory — and every edge
|
||||
* outside it, test self-edges included, is silently dropped.
|
||||
*
|
||||
* Piggyback runs are excluded: their edges come from PHPUnit's own coverage
|
||||
* session, so widening pcov there costs time and buys nothing.
|
||||
*/
|
||||
public static function recordsEdgesInWorkers(): bool
|
||||
{
|
||||
return (string) Parallel::getGlobal(self::RECORDING_GLOBAL) === '1'
|
||||
&& (string) Parallel::getGlobal(self::PIGGYBACK_COVERAGE_GLOBAL) !== '1';
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
@@ -305,6 +485,12 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$this->replayedCount++;
|
||||
$assertions = $this->replayGraph->getAssertions($this->branch, $testId);
|
||||
$this->cachedAssertionsByTestId[$testId] = $assertions ?? 0;
|
||||
|
||||
$time = $this->replayGraph->getTime($this->branch, $testId);
|
||||
|
||||
if ($time !== null) {
|
||||
$this->cachedTimeByTestId[$testId] = $time;
|
||||
}
|
||||
} else {
|
||||
$this->executedCount++;
|
||||
}
|
||||
@@ -317,6 +503,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return $this->cachedAssertionsByTestId[$testId] ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* {@inheritDoc}
|
||||
*/
|
||||
public function handleOriginalArguments(array $arguments): void
|
||||
{
|
||||
$this->originalArguments = $arguments;
|
||||
}
|
||||
|
||||
/**
|
||||
* {@inheritDoc}
|
||||
*/
|
||||
@@ -339,9 +533,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$cliEnabled = $this->hasArgument(self::OPTION, $arguments) || self::envFlagEnabled(self::ENV_TIA);
|
||||
$alwaysEnabled = $watchPatterns->isEnabled()
|
||||
&& (! $watchPatterns->isLocally() || Environment::name() === Environment::LOCAL);
|
||||
if (! $isWorker && ! $disabled && ($cliEnabled || $alwaysEnabled)) {
|
||||
$this->guardUnsupportedOptions($arguments);
|
||||
}
|
||||
|
||||
$hasExplicitPath = $this->hasExplicitPathArgument($arguments);
|
||||
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
|
||||
$disabled = $disabled || $partial;
|
||||
|
||||
// A mutation subprocess runs the suite against source the mutation
|
||||
// plugin has deliberately broken. Its failures describe the mutant, not
|
||||
// the working tree, so unlike every other narrowed run there is nothing
|
||||
// in its results worth keeping. The parent `--mutate` run is untouched
|
||||
// by this: it runs the whole suite against real source.
|
||||
if (getenv(self::ENV_MUTATION_TESTING) !== false) {
|
||||
$this->writesSuppressed = true;
|
||||
}
|
||||
$enabled = ! $disabled && ($cliEnabled || $alwaysEnabled);
|
||||
$this->filteredMode = ($this->hasArgument(self::FILTERED_OPTION, $arguments) || self::envFlagEnabled(self::ENV_FILTERED) || $watchPatterns->isFiltered())
|
||||
&& ! $this->hasExplicitPathArgument($arguments)
|
||||
&& ! $hasExplicitPath
|
||||
&& ! $this->coverageReportActive();
|
||||
$freshRequested = $this->hasArgument(self::FRESH_OPTION, $arguments);
|
||||
$this->forceRefetch = $this->hasArgument(self::REFETCH_OPTION, $arguments);
|
||||
@@ -355,6 +565,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$arguments = $this->popArgument(self::BASELINED_OPTION, $arguments);
|
||||
|
||||
if ($disabled) {
|
||||
if ($partial) {
|
||||
// TIA cannot choose what runs here — the user already did — but
|
||||
// the tests they picked still report honestly, so their results
|
||||
// are kept and everything that would speak for the excluded ones
|
||||
// is not. `--no-tia` needs none of this: it still runs the whole
|
||||
// suite, so it remains a complete run.
|
||||
$this->resultsOnlyWrites = true;
|
||||
|
||||
// `$this->filteredMode` counts as asking for it: reaching here
|
||||
// means the narrowing came from the command line while filtered
|
||||
// mode came from the environment or the config, and a run that
|
||||
// silently declines what the config asked for is the one most
|
||||
// in need of the explanation.
|
||||
if ($cliEnabled || $freshRequested || $this->forceRefetch || $this->filteredMode) {
|
||||
$this->output->writeln('');
|
||||
$this->renderChild('TIA does not apply to partial runs — running the selected tests directly.');
|
||||
}
|
||||
}
|
||||
|
||||
$this->forceRefetch = false;
|
||||
$this->filteredMode = false;
|
||||
$this->freshRebuild = false;
|
||||
@@ -392,6 +621,18 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$this->flushWorkerReplay();
|
||||
}
|
||||
|
||||
// Both only ever set for the parent — addOutput() returns early in
|
||||
// workers, whose partials are ephemeral and only reach the baseline if
|
||||
// the parent consumes them. Everything this method goes on to write is
|
||||
// whole-suite by nature — the edge map above all — so a narrowed run
|
||||
// stops here too, its results already persisted by addOutput().
|
||||
if ($this->writesSuppressed || $this->resultsOnlyWrites) {
|
||||
$this->recorder->reset();
|
||||
$this->coverageCollector->reset();
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
$recorder = $this->recorder;
|
||||
|
||||
if (! $this->recordingActive && ! $recorder->isActive()) {
|
||||
@@ -453,7 +694,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$this->branch,
|
||||
$changedFiles->snapshotTree($changedFiles->since($currentSha) ?? []),
|
||||
);
|
||||
$graph->replaceEdges($perTest);
|
||||
$graph->replaceEdges($perTest, keepExisting: $this->piggybackCoverage);
|
||||
$graph->replaceTestTables($perTestTables);
|
||||
$graph->replaceTestInertiaComponents($perTestInertia);
|
||||
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
|
||||
@@ -481,12 +722,34 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
// `->only()` narrows the executed set exactly like `--filter` does, but
|
||||
// is only knowable once the suite has been collected — too late to turn
|
||||
// TIA off up front. Sampled in addOutput() because Only's lock file is
|
||||
// already gone by the time terminate() runs (its plugin terminates
|
||||
// first). Whether the run was cut short is likewise only knowable now.
|
||||
if (Only::isEnabled() || $this->stoppedEarly()) {
|
||||
$this->resultsOnlyWrites = true;
|
||||
}
|
||||
|
||||
$this->reportMissingWorkerDrivers();
|
||||
|
||||
// Runs before the checks below: it is what fills the parent's result
|
||||
// collector in parallel, and a worker that stopped early narrows the
|
||||
// whole run.
|
||||
if (Parallel::isEnabled()) {
|
||||
$this->mergeWorkerReplayPartials();
|
||||
}
|
||||
|
||||
if ($this->writesSuppressed) {
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
if ($this->resultsOnlyWrites) {
|
||||
$this->snapshotTestResults(complete: false);
|
||||
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
if ($this->replayRan) {
|
||||
$this->bumpRecordedSha();
|
||||
}
|
||||
@@ -547,7 +810,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return $exitCode;
|
||||
}
|
||||
|
||||
$graph->replaceEdges($finalised);
|
||||
$graph->replaceEdges($finalised, keepExisting: $this->piggybackCoverage);
|
||||
$graph->replaceTestTables($finalisedTables);
|
||||
$graph->replaceTestInertiaComponents($finalisedInertia);
|
||||
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
|
||||
@@ -635,7 +898,19 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
Panic::with(new TiaRequiresRepositoryRoot($subdirectoryPrefix));
|
||||
}
|
||||
|
||||
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
|
||||
$this->resolveBranch($projectRoot);
|
||||
|
||||
// After resolveBranch(), so a directory that is no repository at all
|
||||
// still reports the missing git dependency rather than an unresolved
|
||||
// default branch. Nothing named the branch every other baseline reads
|
||||
// through, so every new branch would re-run the whole suite while the
|
||||
// output called it a hit. A repository with no remote is the likeliest
|
||||
// reason and gets said out loud.
|
||||
if (! $this->fallbackBranchResolved) {
|
||||
Panic::with(new ChangedFiles($projectRoot)->hasRemote()
|
||||
? new TiaRequiresDefaultBranch
|
||||
: new TiaRequiresRemote);
|
||||
}
|
||||
|
||||
$fingerprint = Fingerprint::compute($projectRoot);
|
||||
$this->startFingerprint = $fingerprint;
|
||||
@@ -673,13 +948,20 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
}
|
||||
}
|
||||
|
||||
if ($this->piggybackCoverage) {
|
||||
// Both of these belong to the coverage cache, which only Pest's own
|
||||
// `--coverage` ever writes or merges. A raw PHPUnit report flag takes
|
||||
// the piggyback path — it must not drive the driver itself — but must
|
||||
// not leave a marker behind, nor force a recording run to prime a cache
|
||||
// that nothing on its path will fill.
|
||||
$coverageCacheOwned = $this->piggybackCoverage && $this->pestCoverageActive();
|
||||
|
||||
if ($coverageCacheOwned) {
|
||||
$this->state->write(self::KEY_COVERAGE_MARKER, '');
|
||||
}
|
||||
|
||||
if ($this->piggybackCoverage && ! $this->state->exists(self::KEY_COVERAGE_CACHE)) {
|
||||
if ($coverageCacheOwned && ! $this->state->exists(self::KEY_COVERAGE_CACHE)) {
|
||||
if ($graph instanceof Graph && $this->driftLabel === null) {
|
||||
$this->freshGraphReason = 'recording coverage baseline';
|
||||
$this->freshGraphReason = 'recording a coverage baseline';
|
||||
}
|
||||
|
||||
return $this->enterRecordMode($arguments);
|
||||
@@ -698,7 +980,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
*/
|
||||
private function handleWorker(array $arguments, string $projectRoot, bool $recordingGlobal, bool $replayingGlobal): array
|
||||
{
|
||||
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
|
||||
$this->resolveBranch($projectRoot);
|
||||
|
||||
if ($replayingGlobal) {
|
||||
$this->installWorkerReplay($projectRoot);
|
||||
@@ -875,7 +1157,15 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
if (! Parallel::isEnabled()) {
|
||||
if ($canRefreshReplayEdges) {
|
||||
$this->recorder->activate();
|
||||
// Piggyback runs read PHPUnit's own coverage session. Driving
|
||||
// the driver alongside it would clear the data PHPUnit is about
|
||||
// to read, so only link tracking may run here.
|
||||
if ($this->piggybackCoverage) {
|
||||
$this->recorder->activateLinkTracking();
|
||||
} else {
|
||||
$this->recorder->activate();
|
||||
}
|
||||
|
||||
$this->recordingActive = true;
|
||||
}
|
||||
|
||||
@@ -894,6 +1184,10 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
if ($canRefreshReplayEdges) {
|
||||
Parallel::setGlobal(self::RECORDING_GLOBAL, '1');
|
||||
|
||||
if ($this->piggybackCoverage) {
|
||||
Parallel::setGlobal(self::PIGGYBACK_COVERAGE_GLOBAL, '1');
|
||||
}
|
||||
}
|
||||
|
||||
if ($this->filteredMode) {
|
||||
@@ -1035,6 +1329,16 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$recorder->activate();
|
||||
$this->recordingActive = true;
|
||||
|
||||
// Why this run is rebuilding is worth saying whenever there is a reason
|
||||
// for it — the parallel and piggyback branches above already do. Runs
|
||||
// that are simply recording for the first time have nothing to explain.
|
||||
if ($this->driftLabel !== null || $this->freshGraphReason !== null) {
|
||||
$this->output->writeln('');
|
||||
$this->renderFreshGraph();
|
||||
|
||||
return $arguments;
|
||||
}
|
||||
|
||||
$this->renderChild('Running in TIA mode.');
|
||||
|
||||
return $arguments;
|
||||
@@ -1042,14 +1346,18 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
|
||||
private function renderFreshGraph(): void
|
||||
{
|
||||
$headline = 'Experimental TIA mode enabled / fresh graph';
|
||||
|
||||
if ($this->driftLabel !== null) {
|
||||
$headline .= sprintf(' (%s changed)', $this->driftLabel);
|
||||
} elseif ($this->freshGraphReason !== null) {
|
||||
$headline .= sprintf(' (%s)', $this->freshGraphReason);
|
||||
if ($this->driftLabel === null && $this->freshGraphReason !== null) {
|
||||
// The reason is only ever set for a run that keeps its graph and
|
||||
// records alongside it, so "fresh graph" would be a lie here.
|
||||
$headline = sprintf('Experimental TIA mode enabled / %s.', $this->freshGraphReason);
|
||||
} else {
|
||||
$headline .= '.';
|
||||
$headline = 'Experimental TIA mode enabled / fresh graph';
|
||||
|
||||
if ($this->driftLabel !== null) {
|
||||
$headline .= sprintf(' (%s changed)', $this->driftLabel);
|
||||
} else {
|
||||
$headline .= '.';
|
||||
}
|
||||
}
|
||||
|
||||
$this->renderChild($headline);
|
||||
@@ -1065,7 +1373,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
{
|
||||
$this->output->writeln('');
|
||||
|
||||
$this->renderChild('Running in TIA mode, however TIA as skipped as it needs Needs ext-pcov or Xdebug.');
|
||||
$this->renderChild('Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug.');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1136,11 +1444,24 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return;
|
||||
}
|
||||
|
||||
// A replayed result carries the duration PHPUnit measured for a test
|
||||
// that never ran — near zero. Unlike the assertion count, which the
|
||||
// replay injects into the result itself, the cached duration lives only
|
||||
// in this process: the parent replayed nothing of its own, so once the
|
||||
// partial is written the real value is unrecoverable. Launder it here
|
||||
// and the parent's verbatim read is correct by construction.
|
||||
foreach ($results as $testId => $result) {
|
||||
$results[$testId]['time'] = $this->resultTime($testId, $result['time']);
|
||||
}
|
||||
|
||||
$json = json_encode([
|
||||
'results' => $results,
|
||||
'replayed' => $this->replayedCount,
|
||||
'affected' => $this->affectedCount,
|
||||
'executed' => $this->executedCount,
|
||||
// Only the worker knows it stopped early — the parent runs no tests
|
||||
// of its own, so its own check would always come back clean.
|
||||
'truncated' => $this->stoppedEarly(),
|
||||
], JSON_UNESCAPED_SLASHES);
|
||||
|
||||
if ($json === false) {
|
||||
@@ -1177,6 +1498,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
continue;
|
||||
}
|
||||
|
||||
// One worker stopping early leaves the whole suite incomplete: the
|
||||
// tests it never reached are missing from the merged result set just
|
||||
// as if they had been filtered out.
|
||||
if (($decoded['truncated'] ?? false) === true) {
|
||||
$this->resultsOnlyWrites = true;
|
||||
}
|
||||
|
||||
if (isset($decoded['replayed']) && is_int($decoded['replayed'])) {
|
||||
$this->replayedCount += $decoded['replayed'];
|
||||
}
|
||||
@@ -1400,6 +1728,15 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return $coverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* The duration to record for a test: its own, unless it was replayed rather
|
||||
* than executed, in which case the duration it was recorded with stands.
|
||||
*/
|
||||
private function resultTime(string $testId, float $time): float
|
||||
{
|
||||
return $this->cachedTimeByTestId[$testId] ?? $time;
|
||||
}
|
||||
|
||||
private function seedResultsInto(Graph $graph): void
|
||||
{
|
||||
/** @var ResultCollector $collector */
|
||||
@@ -1424,7 +1761,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$testId,
|
||||
$result['status'],
|
||||
$result['message'],
|
||||
$result['time'],
|
||||
$this->resultTime($testId, $result['time']),
|
||||
$result['assertions'],
|
||||
$file,
|
||||
);
|
||||
@@ -1436,7 +1773,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$collector->reset();
|
||||
}
|
||||
|
||||
private function snapshotTestResults(bool $markKnownTestFiles = false): void
|
||||
/**
|
||||
* Folds the run's results into the existing graph.
|
||||
*
|
||||
* An incomplete run passes `$complete: false`, which keeps the additive
|
||||
* half — the results of the tests it did run — and drops the half that
|
||||
* speaks for the suite as a whole.
|
||||
*/
|
||||
private function snapshotTestResults(bool $markKnownTestFiles = false, bool $complete = true): void
|
||||
{
|
||||
/** @var ResultCollector $collector */
|
||||
$collector = Container::getInstance()->get(ResultCollector::class);
|
||||
@@ -1455,8 +1799,28 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
$this->resolveBranch($projectRoot);
|
||||
} catch (MissingDependency) {
|
||||
// This run never asked for TIA, so a missing git must not turn it
|
||||
// into a failure the way it does on the TIA path. Writing to the
|
||||
// fallback baseline is the lesser of the two evils.
|
||||
}
|
||||
|
||||
// The graph above was loaded before the branch was known — this path
|
||||
// only writes, but a graph carrying an unresolved fallback is the exact
|
||||
// bug this whole change is about.
|
||||
$graph->setFallbackBranch($this->fallbackBranch);
|
||||
|
||||
$touchedFiles = [];
|
||||
|
||||
// Whether this run is the one that records the edges its results will be
|
||||
// invalidated through. A recording run's edges are written after this
|
||||
// (terminate() runs last), and a parallel one's arrive with the worker
|
||||
// partials that ask for $markKnownTestFiles — either way the graph on
|
||||
// disk cannot be asked yet, so the run is taken at its word.
|
||||
$recordsEdges = $complete && ($markKnownTestFiles || $this->recordingActive);
|
||||
|
||||
foreach ($results as $testId => $result) {
|
||||
$file = $result['file'] ?? null;
|
||||
|
||||
@@ -1468,12 +1832,22 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$touchedFiles[$file] = true;
|
||||
}
|
||||
|
||||
// A result is only ever invalidated through the edges of the test
|
||||
// that produced it, so one recorded for a test the graph has no
|
||||
// edges for could never be invalidated again — it would be replayed
|
||||
// as settled however far the code around it moved. A run that
|
||||
// records no edges leaves such a test exactly as unknown as it
|
||||
// found it, whether or not it ran the whole suite.
|
||||
if (! $recordsEdges && (! is_string($file) || ! $graph->knowsTest($file))) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$graph->setResult(
|
||||
$this->branch,
|
||||
$testId,
|
||||
$result['status'],
|
||||
$result['message'],
|
||||
$result['time'],
|
||||
$this->resultTime($testId, $result['time']),
|
||||
$result['assertions'],
|
||||
$file,
|
||||
);
|
||||
@@ -1483,7 +1857,11 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
$graph->markKnownTestFiles(array_keys($touchedFiles));
|
||||
}
|
||||
|
||||
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
|
||||
// Pruning reads the absence of a test from this run as the test being
|
||||
// gone. That only holds if every test was invited to report.
|
||||
if ($complete) {
|
||||
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
|
||||
}
|
||||
|
||||
$this->saveGraph($graph);
|
||||
$collector->reset();
|
||||
@@ -1520,7 +1898,29 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this run produces a coverage report, however it was asked for.
|
||||
*
|
||||
* The original argv, not the handled arguments: Pest's own Coverage plugin
|
||||
* appends `--coverage-php <path>` to those and runs before this one, and a
|
||||
* paratest worker's arguments always carry it too. `bin/worker.php` never
|
||||
* hands over the original argv, so a worker sees `[]` here and keeps taking
|
||||
* this from {@see self::PIGGYBACK_COVERAGE_GLOBAL} instead.
|
||||
*/
|
||||
private function coverageReportActive(): bool
|
||||
{
|
||||
if ($this->pestCoverageActive()) {
|
||||
return true;
|
||||
}
|
||||
|
||||
return array_any(self::COVERAGE_REPORT_FLAGS, fn (string $flag): bool => $this->hasArgument($flag, $this->originalArguments));
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether Pest's own `--coverage` was given — the only entry point that
|
||||
* writes the coverage cache these two flags read and clean up.
|
||||
*/
|
||||
private function pestCoverageActive(): bool
|
||||
{
|
||||
$coverage = Container::getInstance()->get(Coverage::class);
|
||||
assert($coverage instanceof Coverage);
|
||||
@@ -1528,6 +1928,155 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
|
||||
return $coverage->coverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* Panics when the run asks for Tia alongside an option Tia cannot honour.
|
||||
*
|
||||
* Checked against the original argv as well, because `bin/pest` consumes
|
||||
* some of these itself before PHPUnit ever sees them.
|
||||
*
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function guardUnsupportedOptions(array $arguments): void
|
||||
{
|
||||
foreach (self::UNSUPPORTED_OPTIONS as $option) {
|
||||
if (! $this->hasArgument($option, $arguments) && ! $this->hasArgument($option, $this->originalArguments)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
Panic::with(new InvalidOption(sprintf(
|
||||
'The [%s] option cannot be combined with [%s].',
|
||||
$option,
|
||||
self::OPTION,
|
||||
)));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a selection-narrowing flag was given, either among the arguments
|
||||
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
|
||||
* the original argv. Explicit path arguments and `->only()` are detected
|
||||
* separately.
|
||||
*
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function hasPartialSelection(array $arguments): bool
|
||||
{
|
||||
foreach (self::PARTIAL_SELECTION_FLAGS as $flag) {
|
||||
if ($this->hasArgument($flag, $arguments)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if ($this->hasArgument($flag, $this->originalArguments)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the run stopped before reaching every test it had queued.
|
||||
*
|
||||
* Covers `--bail`, `--retry` and every `--stop-on-*` flag, the equivalent
|
||||
* `phpunit.xml` attributes, and an interrupted run — none of which narrow
|
||||
* the selection up front, so hasPartialSelection() cannot see them. The
|
||||
* tests queued behind the defect that halted the run never reported, and
|
||||
* folding what did report into the baseline prunes the cached results of
|
||||
* their siblings in every file the run had already entered.
|
||||
*
|
||||
* Deliberately unguarded. Both callers run only once PHPUnit's
|
||||
* configuration is registered — the kernel reads it unguarded itself just
|
||||
* before dispatching addOutput(), and flushWorkerReplay() bails out unless
|
||||
* the worker actually executed something. Swallowing a failure here would
|
||||
* report every truncated run as complete, which is the corruption this
|
||||
* guards against in the first place.
|
||||
*/
|
||||
private function stoppedEarly(): bool
|
||||
{
|
||||
return TestResultFacade::shouldStop();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the baselines this run reads from and writes to, once.
|
||||
*
|
||||
* Results are written on runs where TIA itself took no part, and those
|
||||
* never reach handleParent(). Without this the default would stand and
|
||||
* every such run would write its results to `main`, whatever branch it
|
||||
* actually ran on.
|
||||
*/
|
||||
private function resolveBranch(string $projectRoot): void
|
||||
{
|
||||
if ($this->branchResolved) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->branchResolved = true;
|
||||
|
||||
$changedFiles = new ChangedFiles($projectRoot);
|
||||
|
||||
// Resolved before the current branch, which throws where git is
|
||||
// missing: the fallback is advisory, so a run that cannot name its
|
||||
// branch at all should still carry the best answer available.
|
||||
$resolved = $this->resolveFallbackBranch($changedFiles);
|
||||
|
||||
$this->fallbackBranchResolved = $resolved !== null;
|
||||
$this->fallbackBranch = $resolved ?? self::DEFAULT_BRANCH;
|
||||
|
||||
Parallel::setGlobal(self::FALLBACK_BRANCH_GLOBAL, $this->fallbackBranch);
|
||||
|
||||
// A detached HEAD has no branch of its own to write to. The default
|
||||
// branch is the honest key there — it is the commit the checkout most
|
||||
// likely sits on, and it keeps a phantom baseline from being minted
|
||||
// under a branch name the repository never had.
|
||||
$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch;
|
||||
}
|
||||
|
||||
/**
|
||||
* The branch every other baseline falls back to reading, or null when
|
||||
* nothing in the checkout can name it.
|
||||
*
|
||||
* Ordered by how much the source actually knows. Configuration first: it is
|
||||
* the escape hatch for a repository whose git-side answers disagree with its
|
||||
* branches. Then the CI provider, which states the answer outright where git
|
||||
* is at its least informed. Then git itself. Then the recorded graph, whose
|
||||
* single baseline can only have come from the branch this repository
|
||||
* integrates on.
|
||||
*/
|
||||
private function resolveFallbackBranch(ChangedFiles $changedFiles): ?string
|
||||
{
|
||||
$inherited = Parallel::getGlobal(self::FALLBACK_BRANCH_GLOBAL);
|
||||
|
||||
if (is_string($inherited) && $inherited !== '') {
|
||||
return $inherited;
|
||||
}
|
||||
|
||||
return $this->watchPatterns->defaultBranch()
|
||||
?? CiDefaultBranch::detect()
|
||||
?? $changedFiles->defaultBranch()
|
||||
?? $this->soleRecordedBranch();
|
||||
}
|
||||
|
||||
/**
|
||||
* The one branch a recorded graph holds a baseline for.
|
||||
*
|
||||
* Last in the chain and deliberately narrow: with a single baseline on disk
|
||||
* there is only one branch whose results can be read at all, so naming it is
|
||||
* strictly better than resolving to a branch that holds nothing. Two or more
|
||||
* baselines carry no such implication and are left alone.
|
||||
*/
|
||||
private function soleRecordedBranch(): ?string
|
||||
{
|
||||
$json = $this->state->read(self::KEY_GRAPH);
|
||||
|
||||
if ($json === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$branches = Graph::branchesIn($json);
|
||||
|
||||
return count($branches) === 1 ? $branches[0] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
|
||||
@@ -219,6 +219,73 @@ final readonly class ChangedFiles
|
||||
return $branch === '' || $branch === 'HEAD' ? null : $branch;
|
||||
}
|
||||
|
||||
/**
|
||||
* The repository's default branch — the one every other branch's baseline
|
||||
* falls back to reading.
|
||||
*
|
||||
* Advisory, unlike {@see self::currentBranch()}: a repository that cannot
|
||||
* answer the question is not a broken repository. A remote-less checkout
|
||||
* has no `origin/HEAD`, and plenty of CI checkouts never run
|
||||
* `git remote set-head`, so every step here fails soft and the caller is
|
||||
* left to pick its own default.
|
||||
*/
|
||||
public function defaultBranch(): ?string
|
||||
{
|
||||
$head = $this->gitOutput(['git', 'symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
|
||||
|
||||
if ($head !== null) {
|
||||
$branch = preg_replace('#^origin/#', '', $head);
|
||||
|
||||
if (is_string($branch) && $branch !== '') {
|
||||
return $branch;
|
||||
}
|
||||
}
|
||||
|
||||
// `init.defaultBranch` is a setting of the machine, not of the
|
||||
// repository — it names what `git init` would have called the first
|
||||
// branch here, which is worth nothing once the repository disagrees.
|
||||
// Taken only when a branch by that name actually exists.
|
||||
$configured = $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
|
||||
|
||||
if ($configured === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$exists = $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/heads/'.$configured]) !== null
|
||||
|| $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/remotes/origin/'.$configured]) !== null;
|
||||
|
||||
return $exists ? $configured : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the repository has any remote configured.
|
||||
*
|
||||
* Advisory like {@see self::defaultBranch()} — a `git` that cannot answer
|
||||
* is reported as "no remote", and the caller decides what that means.
|
||||
*/
|
||||
public function hasRemote(): bool
|
||||
{
|
||||
return $this->gitOutput(['git', 'remote']) !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $command
|
||||
*/
|
||||
private function gitOutput(array $command): ?string
|
||||
{
|
||||
$process = new Process($command, $this->projectRoot);
|
||||
$process->setTimeout(5.0);
|
||||
$process->run();
|
||||
|
||||
if (! $process->isSuccessful()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$output = trim($process->getOutput());
|
||||
|
||||
return $output === '' ? null : $output;
|
||||
}
|
||||
|
||||
private function shaIsReachable(string $sha): bool
|
||||
{
|
||||
$process = new Process(
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Pest\Plugins\Tia;
|
||||
|
||||
/**
|
||||
* The default branch as the CI provider itself reports it.
|
||||
*
|
||||
* Worth asking before git: a CI checkout is the one place where git knows the
|
||||
* least. `actions/checkout` builds the working copy with `git init` plus a
|
||||
* single-ref `fetch` rather than a `clone`, so `origin/HEAD` is never set and
|
||||
* `init.defaultBranch` — a setting of the runner image, not of the repository —
|
||||
* is all git has left to offer. The provider, meanwhile, states the answer
|
||||
* outright in the environment it handed us.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class CiDefaultBranch
|
||||
{
|
||||
/**
|
||||
* Advisory, like every other source in the chain: anything unreadable,
|
||||
* unparsable, or simply absent means "no answer", never a failure.
|
||||
*/
|
||||
public static function detect(): ?string
|
||||
{
|
||||
return self::fromGitLab() ?? self::fromGitHubEvent();
|
||||
}
|
||||
|
||||
private static function fromGitLab(): ?string
|
||||
{
|
||||
return self::environment('CI_DEFAULT_BRANCH');
|
||||
}
|
||||
|
||||
/**
|
||||
* GitHub publishes no default-branch variable, but every repository-scoped
|
||||
* event payload carries `repository.default_branch`, and the path to that
|
||||
* payload is in the environment.
|
||||
*/
|
||||
private static function fromGitHubEvent(): ?string
|
||||
{
|
||||
$path = self::environment('GITHUB_EVENT_PATH');
|
||||
|
||||
if ($path === null || ! is_file($path) || ! is_readable($path)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$contents = @file_get_contents($path);
|
||||
|
||||
if ($contents === false) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$payload = json_decode($contents, true);
|
||||
|
||||
if (! is_array($payload) || ! is_array($payload['repository'] ?? null)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$branch = $payload['repository']['default_branch'] ?? null;
|
||||
|
||||
return is_string($branch) && $branch !== '' ? $branch : null;
|
||||
}
|
||||
|
||||
private static function environment(string $name): ?string
|
||||
{
|
||||
$value = getenv($name);
|
||||
|
||||
if (! is_string($value)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$value = trim($value);
|
||||
|
||||
return $value === '' ? null : $value;
|
||||
}
|
||||
}
|
||||
@@ -60,6 +60,24 @@ final class Configuration
|
||||
return $this;
|
||||
}
|
||||
|
||||
/**
|
||||
* The branch whose baseline every other branch falls back to reading.
|
||||
*
|
||||
* Autodetected from the repository when left unset; declare it here when
|
||||
* the repository cannot answer for itself — no `origin/HEAD`, or an
|
||||
* `init.defaultBranch` that disagrees with reality.
|
||||
*
|
||||
* @return $this
|
||||
*/
|
||||
public function defaultBranch(string $branch): self
|
||||
{
|
||||
/** @var WatchPatterns $watchPatterns */
|
||||
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
|
||||
$watchPatterns->setDefaultBranch($branch);
|
||||
|
||||
return $this;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, string> $patterns glob → project-relative test dir
|
||||
* @return $this
|
||||
|
||||
@@ -48,6 +48,17 @@ final class Graph
|
||||
*/
|
||||
private array $baselines = [];
|
||||
|
||||
/**
|
||||
* The baseline a branch with none of its own reads from.
|
||||
*
|
||||
* Only ever read from: a branch writes to its own key, so a fallback that
|
||||
* leaked into the write path would corrupt the baseline every other branch
|
||||
* depends on. Resolved once per run by the plugin — see
|
||||
* {@see self::setFallbackBranch()} — because the git calls it takes are not
|
||||
* free and the read path runs per test.
|
||||
*/
|
||||
private string $fallbackBranch = 'main';
|
||||
|
||||
private readonly string $projectRoot;
|
||||
|
||||
/** @var array<string, true>|null */
|
||||
@@ -576,7 +587,12 @@ final class Graph
|
||||
return $this->fingerprint;
|
||||
}
|
||||
|
||||
public function recordedAtSha(string $branch, string $fallbackBranch = 'main'): ?string
|
||||
public function setFallbackBranch(string $branch): void
|
||||
{
|
||||
$this->fallbackBranch = $branch;
|
||||
}
|
||||
|
||||
public function recordedAtSha(string $branch, ?string $fallbackBranch = null): ?string
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
|
||||
@@ -611,7 +627,7 @@ final class Graph
|
||||
$this->baselines[$branch]['results'][$testId] = $entry;
|
||||
}
|
||||
|
||||
public function getAssertions(string $branch, string $testId, string $fallbackBranch = 'main'): ?int
|
||||
public function getAssertions(string $branch, string $testId, ?string $fallbackBranch = null): ?int
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
|
||||
@@ -622,7 +638,18 @@ final class Graph
|
||||
return $baseline['results'][$testId]['assertions'];
|
||||
}
|
||||
|
||||
public function getResult(string $branch, string $testId, string $fallbackBranch = 'main'): ?TestStatus
|
||||
public function getTime(string $branch, string $testId, ?string $fallbackBranch = null): ?float
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
|
||||
if (! isset($baseline['results'][$testId]['time'])) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return $baseline['results'][$testId]['time'];
|
||||
}
|
||||
|
||||
public function getResult(string $branch, string $testId, ?string $fallbackBranch = null): ?TestStatus
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
|
||||
@@ -649,7 +676,7 @@ final class Graph
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function testFilesToRerun(string $branch, string $fallbackBranch = 'main'): array
|
||||
public function testFilesToRerun(string $branch, ?string $fallbackBranch = null): array
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
$files = [];
|
||||
@@ -677,7 +704,16 @@ final class Graph
|
||||
return array_keys($files);
|
||||
}
|
||||
|
||||
public function hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main'): bool
|
||||
/**
|
||||
* Whether any cached result due a re-run points at a test file that is not
|
||||
* on disk — deleted, or never locatable in the first place (`eval()`'d code,
|
||||
* a path outside the project).
|
||||
*
|
||||
* A filtered run cannot honour such an entry: it would select a file that
|
||||
* collects no tests, so the run reports green without ever re-running the
|
||||
* failure — and does so again on every subsequent invocation.
|
||||
*/
|
||||
public function hasUnlocatedTestsToRerun(string $branch, ?string $fallbackBranch = null): bool
|
||||
{
|
||||
$baseline = $this->baselineFor($branch, $fallbackBranch);
|
||||
|
||||
@@ -688,7 +724,16 @@ final class Graph
|
||||
|
||||
$file = $result['file'] ?? null;
|
||||
|
||||
if ($file === null || $file === '' || $this->relative($file) === null) {
|
||||
if ($file === null || $file === '') {
|
||||
return true;
|
||||
}
|
||||
|
||||
$rel = $this->relative($file);
|
||||
|
||||
// Results are stored relative, so `relative()` answers "is this
|
||||
// inside the project" without ever touching the filesystem. The
|
||||
// stat is what tells a deleted test file apart from a live one.
|
||||
if ($rel === null || ! is_file($this->projectRoot.'/'.$rel)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -779,7 +824,7 @@ final class Graph
|
||||
/**
|
||||
* @return array<string, string>
|
||||
*/
|
||||
public function lastRunTree(string $branch, string $fallbackBranch = 'main'): array
|
||||
public function lastRunTree(string $branch, ?string $fallbackBranch = null): array
|
||||
{
|
||||
return $this->baselineFor($branch, $fallbackBranch)['tree'];
|
||||
}
|
||||
@@ -787,8 +832,10 @@ final class Graph
|
||||
/**
|
||||
* @return array{sha: ?string, tree: array<string, string>, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
|
||||
*/
|
||||
private function baselineFor(string $branch, string $fallbackBranch): array
|
||||
private function baselineFor(string $branch, ?string $fallbackBranch): array
|
||||
{
|
||||
$fallbackBranch ??= $this->fallbackBranch;
|
||||
|
||||
if (isset($this->baselines[$branch])) {
|
||||
return $this->baselines[$branch];
|
||||
}
|
||||
@@ -809,8 +856,15 @@ final class Graph
|
||||
|
||||
/**
|
||||
* @param array<string, array<int, string>> $testToFiles
|
||||
* @param bool $keepExisting Leave already-recorded edge sets alone. For runs
|
||||
* whose edges are piggybacked off a PHPUnit coverage
|
||||
* session: that data is scoped by `<source>`, so it
|
||||
* can only ever be narrower than what the TIA
|
||||
* recorder sees — it never contains the test's own
|
||||
* file, for one — and a narrower edge set silently
|
||||
* stops selecting the tests it used to select.
|
||||
*/
|
||||
public function replaceEdges(array $testToFiles): void
|
||||
public function replaceEdges(array $testToFiles, bool $keepExisting = false): void
|
||||
{
|
||||
foreach ($testToFiles as $testFile => $sources) {
|
||||
$testRel = $this->relative($testFile);
|
||||
@@ -819,6 +873,12 @@ final class Graph
|
||||
continue;
|
||||
}
|
||||
|
||||
// An empty set means "known, covers nothing", so piggyback data is
|
||||
// still an improvement there — only a populated set is protected.
|
||||
if ($keepExisting && ($this->edges[$testRel] ?? []) !== []) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$this->edges[$testRel] = [];
|
||||
|
||||
foreach ($sources as $source) {
|
||||
@@ -1422,6 +1482,35 @@ final class Graph
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The branches a recorded graph holds baselines for, read straight from the
|
||||
* encoded form.
|
||||
*
|
||||
* Answerable before the graph is hydrated because the default branch has to
|
||||
* be resolved first: the fallback is what every hydrated graph reads its
|
||||
* baselines through.
|
||||
*
|
||||
* @return list<string>
|
||||
*/
|
||||
public static function branchesIn(string $json): array
|
||||
{
|
||||
$data = json_decode($json, true);
|
||||
|
||||
if (! is_array($data) || ! is_array($data['baselines'] ?? null)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$branches = [];
|
||||
|
||||
foreach (array_keys($data['baselines']) as $branch) {
|
||||
if (is_string($branch) && $branch !== '') {
|
||||
$branches[] = $branch;
|
||||
}
|
||||
}
|
||||
|
||||
return $branches;
|
||||
}
|
||||
|
||||
public static function decode(string $json, string $projectRoot): ?self
|
||||
{
|
||||
$data = json_decode($json, true);
|
||||
|
||||
@@ -56,7 +56,7 @@ final class TableExtractor
|
||||
$tables[strtolower($name)] = true;
|
||||
}
|
||||
|
||||
$out = array_keys($tables);
|
||||
$out = array_map(strval(...), array_keys($tables));
|
||||
sort($out);
|
||||
|
||||
return $out;
|
||||
@@ -112,7 +112,7 @@ final class TableExtractor
|
||||
}
|
||||
}
|
||||
|
||||
$out = array_keys($tables);
|
||||
$out = array_map(strval(...), array_keys($tables));
|
||||
sort($out);
|
||||
|
||||
return $out;
|
||||
|
||||
@@ -44,6 +44,8 @@ final class WatchPatterns
|
||||
|
||||
private bool $baselined = false;
|
||||
|
||||
private ?string $defaultBranch = null;
|
||||
|
||||
public function useDefaults(string $projectRoot): void
|
||||
{
|
||||
$testPath = TestSuite::getInstance()->testPath;
|
||||
@@ -177,6 +179,16 @@ final class WatchPatterns
|
||||
return $this->baselined;
|
||||
}
|
||||
|
||||
public function setDefaultBranch(string $branch): void
|
||||
{
|
||||
$this->defaultBranch = $branch;
|
||||
}
|
||||
|
||||
public function defaultBranch(): ?string
|
||||
{
|
||||
return $this->defaultBranch;
|
||||
}
|
||||
|
||||
public function reset(): void
|
||||
{
|
||||
$this->patterns = [];
|
||||
@@ -185,6 +197,7 @@ final class WatchPatterns
|
||||
$this->locally = false;
|
||||
$this->filtered = false;
|
||||
$this->baselined = false;
|
||||
$this->defaultBranch = null;
|
||||
}
|
||||
|
||||
private function keyMatches(string $key, string $file): bool
|
||||
|
||||
@@ -22,7 +22,7 @@ final readonly class EnsureTiaAssertionsAreRecordedOnFinished implements Finishe
|
||||
|
||||
if ($test instanceof TestMethod) {
|
||||
$this->collector->recordAssertions(
|
||||
$test->className().'::'.$test->methodName(),
|
||||
$test->id(),
|
||||
$event->numberOfAssertionsPerformed(),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -21,7 +21,7 @@ final readonly class EnsureTiaResultsAreCollected implements PreparationStartedS
|
||||
$test = $event->test();
|
||||
|
||||
if ($test instanceof TestMethod) {
|
||||
$this->collector->testPrepared($test->className().'::'.$test->methodName(), $test->file());
|
||||
$this->collector->testPrepared($test->id(), $test->file());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,33 @@ final class Str
|
||||
|
||||
private const string PREFIX = '__pest_evaluable_';
|
||||
|
||||
/**
|
||||
* The list of names PHP reserves, and therefore refuses, as class names.
|
||||
*
|
||||
* @see https://github.com/php/php-src/blob/master/Zend/zend_compile.c
|
||||
*
|
||||
* @var array<int, string>
|
||||
*/
|
||||
private const array RESERVED_CLASS_NAMES = [
|
||||
'array',
|
||||
'bool',
|
||||
'callable',
|
||||
'false',
|
||||
'float',
|
||||
'int',
|
||||
'iterable',
|
||||
'mixed',
|
||||
'never',
|
||||
'null',
|
||||
'object',
|
||||
'parent',
|
||||
'self',
|
||||
'static',
|
||||
'string',
|
||||
'true',
|
||||
'void',
|
||||
];
|
||||
|
||||
/**
|
||||
* Create a (unsecure & non-cryptographically safe) random alpha-numeric
|
||||
* string value.
|
||||
@@ -64,6 +91,35 @@ final class Str
|
||||
return (string) preg_replace('/[^a-zA-Z0-9_\x80-\xff]/', '_', $code);
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine if the given name is a valid PHP identifier, and therefore may
|
||||
* be used as a single namespace name.
|
||||
*/
|
||||
public static function isValidIdentifier(string $name): bool
|
||||
{
|
||||
return preg_match('/^[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*$/', $name) === 1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine if the given name may be declared as a class name by an `eval`.
|
||||
*/
|
||||
public static function isValidClassName(string $name): bool
|
||||
{
|
||||
if (! self::isValidIdentifier($name)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (in_array(strtolower($name), self::RESERVED_CLASS_NAMES, true)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$tokens = token_get_all(sprintf('<?php %s;', $name));
|
||||
|
||||
// Anything the lexer sees as a keyword, like `list` or `match`, may not
|
||||
// be used as a class name.
|
||||
return is_array($tokens[1] ?? null) && $tokens[1][0] === T_STRING;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the portion of a string before the last occurrence of a given value.
|
||||
*/
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
|
||||
Pest Testing Framework 5.0.2.
|
||||
Pest Testing Framework 5.0.3.
|
||||
|
||||
USAGE: pest <file> [options]
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
|
||||
Pest Testing Framework 5.0.2.
|
||||
Pest Testing Framework 5.0.3.
|
||||
|
||||
|
||||
@@ -1,27 +1,27 @@
|
||||
##teamcity[testSuiteStarted name='Tests/tests/Failure' locationHint='pest_qn://tests/.tests/Failure.php' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/Failure' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php' flowId='1234']
|
||||
##teamcity[testCount count='8' flowId='1234']
|
||||
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/.tests/Failure.php::it can fail with comparison' flowId='1234']
|
||||
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/.tests/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
|
||||
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail with comparison' flowId='1234']
|
||||
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/Fixtures/Suites/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
|
||||
##teamcity[testFinished name='it can fail with comparison' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because of no assertions' flowId='1234']
|
||||
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because of no assertions' flowId='1234']
|
||||
##teamcity[testIgnored name='it can be ignored because of no assertions' message='This test did not perform any assertions' details='' flowId='1234']
|
||||
##teamcity[testFinished name='it can be ignored because of no assertions' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because it is skipped' flowId='1234']
|
||||
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because it is skipped' flowId='1234']
|
||||
##teamcity[testIgnored name='it can be ignored because it is skipped' message='This test was ignored.' details='' flowId='1234']
|
||||
##teamcity[testFinished name='it can be ignored because it is skipped' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/.tests/Failure.php::it can fail' flowId='1234']
|
||||
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/.tests/Failure.php:18' flowId='1234']
|
||||
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail' flowId='1234']
|
||||
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/Fixtures/Suites/Failure.php:18' flowId='1234']
|
||||
##teamcity[testFinished name='it can fail' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/.tests/Failure.php::it throws exception' flowId='1234']
|
||||
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/.tests/Failure.php:22' flowId='1234']
|
||||
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it throws exception' flowId='1234']
|
||||
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/Fixtures/Suites/Failure.php:22' flowId='1234']
|
||||
##teamcity[testFinished name='it throws exception' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/.tests/Failure.php::it is not done yet' flowId='1234']
|
||||
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is not done yet' flowId='1234']
|
||||
##teamcity[testFinished name='it is not done yet' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/.tests/Failure.php::build this one.' flowId='1234']
|
||||
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::build this one.' flowId='1234']
|
||||
##teamcity[testFinished name='build this one.' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/.tests/Failure.php::it is passing' flowId='1234']
|
||||
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is passing' flowId='1234']
|
||||
##teamcity[testFinished name='it is passing' duration='100000' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='Tests/tests/Failure' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/Failure' flowId='1234']
|
||||
|
||||
[90mTests:[39m [31;1m3 failed[39;22m[90m,[39m[39m [39m[33;1m1 risky[39;22m[90m,[39m[39m [39m[36;1m2 todos[39;22m[90m,[39m[39m [39m[33;1m1 skipped[39;22m[90m,[39m[39m [39m[32;1m1 passed[39;22m[90m (3 assertions)[39m
|
||||
[90mDuration:[39m [39m1.00s[39m
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
##teamcity[testSuiteStarted name='Tests/tests/SuccessOnly' locationHint='pest_qn://tests/.tests/SuccessOnly.php' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/SuccessOnly' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php' flowId='1234']
|
||||
##teamcity[testCount count='4' flowId='1234']
|
||||
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/.tests/SuccessOnly.php::it can pass with comparison' flowId='1234']
|
||||
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison' flowId='1234']
|
||||
##teamcity[testFinished name='it can pass with comparison' duration='100000' flowId='1234']
|
||||
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can also pass' flowId='1234']
|
||||
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can also pass' flowId='1234']
|
||||
##teamcity[testFinished name='can also pass' duration='100000' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset' flowId='1234']
|
||||
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset' flowId='1234']
|
||||
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
|
||||
##teamcity[testFinished name='can pass with dataset with data set "(true)"' duration='100000' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='can pass with dataset' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
|
||||
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
|
||||
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
|
||||
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
|
||||
##teamcity[testFinished name='`block` → can pass with dataset in describe block with data set "(1)"' duration='100000' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='`block` → can pass with dataset in describe block' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='Tests/tests/SuccessOnly' flowId='1234']
|
||||
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/SuccessOnly' flowId='1234']
|
||||
|
||||
[90mTests:[39m [32;1m4 passed[39;22m[90m (4 assertions)[39m
|
||||
[90mDuration:[39m [39m1.00s[39m
|
||||
|
||||
@@ -1,17 +1,4 @@
|
||||
WARN Tests\Fixtures\CollisionTest
|
||||
- error
|
||||
- success
|
||||
|
||||
PASS Tests\Fixtures\DirectoryWithTests\ExampleTest
|
||||
✓ it example 1
|
||||
|
||||
PASS Tests\Fixtures\ExampleTest
|
||||
✓ it example 2
|
||||
|
||||
WARN Tests\Fixtures\Inheritance\Base\ExampleTest
|
||||
- example
|
||||
|
||||
PASS Tests\Fixtures\Inheritance\ExampleTest
|
||||
✓ example
|
||||
|
||||
Tests: 3 skipped, 3 passed (3 assertions)
|
||||
Tests: 1 passed (1 assertions)
|
||||
|
||||
@@ -1565,6 +1565,37 @@
|
||||
✓ it allows performing no expectations without being risky
|
||||
✓ a "describe" group of tests → it allows performing no expectations without being risky
|
||||
|
||||
PASS Tests\Features\Tia
|
||||
✓ it does not run user hooks when replaying cached skipped and incomplete results
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchReplay
|
||||
✓ replays the default branch baseline on a new branch
|
||||
✓ replays whatever the default branch is called with ('main')
|
||||
✓ replays whatever the default branch is called with ('master')
|
||||
✓ replays whatever the default branch is called with ('trunk')
|
||||
✓ replays whatever the default branch is called with ('develop')
|
||||
✓ replays on a second new branch too
|
||||
✓ writes nothing on a second run on the same branch
|
||||
✓ replays on a branch whose name contains slashes
|
||||
✓ replays inside a worktree on a new branch
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchResolution
|
||||
✓ a declared default branch beats autodetection
|
||||
✓ a declared default branch that does not exist degrades to a full run
|
||||
✓ a repository with no remote is refused rather than silently re-run
|
||||
✓ a declared default branch stands in for a missing remote
|
||||
✓ tia still requires git
|
||||
✓ a plain run outside a repository creates no baseline
|
||||
✓ the default branch is resolved once per run, not once per test
|
||||
|
||||
PASS Tests\Features\Tia\DefaultBranchWriteTier
|
||||
✓ narrows to the affected tests on a new branch
|
||||
✓ filtered mode reads the fallback too
|
||||
✓ filtered mode finds nothing to do on a clean green feature branch
|
||||
✓ a detached HEAD replays without minting a branch key
|
||||
✓ the branch that ran gets its own key and the default branch keeps its baseline
|
||||
✓ the fallback reaches parallel workers
|
||||
|
||||
PASS Tests\Features\Ticket
|
||||
✓ it may be associated with an ticket #1, #2
|
||||
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
|
||||
@@ -1908,6 +1939,11 @@
|
||||
✓ activateLinkTracking() → it tracks linked sources across consecutive tests
|
||||
✓ activateLinkTracking() → it records nothing while inactive
|
||||
|
||||
PASS Tests\Unit\Plugins\Tia\ResultKey
|
||||
✓ it keys a result per dataset row rather than per method
|
||||
✓ it records assertions against the same per-dataset key
|
||||
✓ it leaves a test without a dataset keyed by class and method
|
||||
|
||||
PASS Tests\Unit\Plugins\Tia\TableExtractor
|
||||
✓ fromSql() → it extracts tables from plain DML
|
||||
✓ fromSql() → it extracts tables from joins
|
||||
@@ -1916,10 +1952,12 @@
|
||||
✓ fromSql() → it records the table, not the schema, for qualified identifiers
|
||||
✓ fromSql() → it handles quoted identifiers
|
||||
✓ fromSql() → it ignores schema metadata tables
|
||||
✓ fromSql() → it does not leak int keys for numeric identifiers
|
||||
✓ fromSql() → it returns nothing for non-DML statements
|
||||
✓ fromMigrationSource() → it extracts tables from Schema builder calls
|
||||
✓ fromMigrationSource() → it extracts tables from raw DDL statements
|
||||
✓ fromMigrationSource() → it records the table, not the schema, in qualified DDL and DML
|
||||
✓ fromMigrationSource() → it does not leak int keys for numeric table names
|
||||
✓ fromMigrationSource() → it extracts tables from DB::table calls
|
||||
|
||||
PASS Tests\Unit\Plugins\Tia\TestPaths
|
||||
@@ -2153,6 +2191,7 @@
|
||||
✓ a parallel test can extend another test with same name
|
||||
✓ parallel reports invalid datasets as failures
|
||||
✓ parallel can have multiple exclude-groups
|
||||
✓ parallel can have multiple groups
|
||||
|
||||
PASS Tests\Visual\ParallelNestedDatasets
|
||||
✓ parallel loads nested datasets from nested directories
|
||||
@@ -2186,4 +2225,4 @@
|
||||
✓ pass with dataset with ('my-datas-set-value')
|
||||
✓ within describe → pass with dataset with ('my-datas-set-value')
|
||||
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1542 passed (3375 assertions)
|
||||
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1571 passed (3446 assertions)
|
||||
@@ -18,27 +18,27 @@ $run = function (string $target): array {
|
||||
};
|
||||
|
||||
test('reports missing datasets as errors for a single file run', function () use ($run): void {
|
||||
$result = $run('tests/.tests/IssueOnly.php');
|
||||
$result = $run('tests/Fixtures/Suites/IssueOnly.php');
|
||||
|
||||
expect($result['output'])
|
||||
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
|
||||
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
|
||||
->toContain('FAILED')
|
||||
->toContain('Tests: 1 failed')
|
||||
->and($result['code'])->not->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('reports missing datasets as errors alongside passing tests', function () use ($run): void {
|
||||
$result = $run('tests/.tests/IssueWithPassing.php');
|
||||
$result = $run('tests/Fixtures/Suites/IssueWithPassing.php');
|
||||
|
||||
expect($result['output'])
|
||||
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
|
||||
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
|
||||
->toContain('1 passed')
|
||||
->toContain('1 failed')
|
||||
->and($result['code'])->not->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('reports dataset closure exceptions as errors', function () use ($run): void {
|
||||
$result = $run('tests/.tests/DatasetClosureThrows.php');
|
||||
$result = $run('tests/Fixtures/Suites/DatasetClosureThrows.php');
|
||||
|
||||
expect($result['output'])
|
||||
->toContain('boom from dataset')
|
||||
|
||||
@@ -10,12 +10,12 @@ beforeEach(function (): void {
|
||||
});
|
||||
|
||||
it('throws exception if dataset does not exist', function (): void {
|
||||
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name `first` does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
|
||||
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name [first] does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
|
||||
});
|
||||
|
||||
it('throws exception if dataset already exist', function (): void {
|
||||
DatasetsRepository::set('second', [[]], __DIR__);
|
||||
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name `second` already exists in scope ['.__DIR__.'].');
|
||||
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name [second] already exists in scope ['.__DIR__.'].');
|
||||
});
|
||||
|
||||
it('sets closures', function (): void {
|
||||
|
||||
@@ -142,7 +142,7 @@ it('works as higher order test')
|
||||
|
||||
it('fails after exhausting all retries', function (): void {
|
||||
$process = new Process(
|
||||
['php', 'bin/pest', 'tests/.tests/FlakyFailure.php'],
|
||||
['php', 'bin/pest', 'tests/Fixtures/Suites/FlakyFailure.php'],
|
||||
dirname(__DIR__, 2),
|
||||
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
|
||||
);
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
<?php
|
||||
|
||||
use Pest\Plugins\Tia;
|
||||
use Pest\Plugins\Tia\ChangedFiles;
|
||||
use Pest\Plugins\Tia\FileState;
|
||||
use Pest\Plugins\Tia\Fingerprint;
|
||||
use Pest\Plugins\Tia\Graph;
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use Pest\Support\Str;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
it('does not run user hooks when replaying cached skipped and incomplete results', function (): void {
|
||||
$projectRoot = dirname(__DIR__, 2);
|
||||
$home = sys_get_temp_dir().'/pest-tia-'.bin2hex(random_bytes(8));
|
||||
$fixture = 'tests/Fixtures/Suites/TiaReplayHooks.php';
|
||||
|
||||
mkdir($home, 0755, true);
|
||||
|
||||
try {
|
||||
$changedFiles = new ChangedFiles($projectRoot);
|
||||
$branch = $changedFiles->currentBranch() ?? 'main';
|
||||
$sha = $changedFiles->currentSha();
|
||||
|
||||
$id = fn (string $description): string => 'P\Tests\Fixtures\Suites\TiaReplayHooks::'.Str::evaluable($description);
|
||||
|
||||
$graph = new Graph($projectRoot);
|
||||
$graph->setFingerprint(Fingerprint::compute($projectRoot));
|
||||
$graph->setRecordedAtSha($branch, $sha);
|
||||
// Hashes the working tree as it stands, so the replay sees nothing as changed.
|
||||
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
|
||||
$graph->markKnownTestFiles([$fixture]);
|
||||
$graph->setResult($branch, $id('replayed pass'), 0, '', 0.01, 1, $fixture);
|
||||
$graph->setResult($branch, $id('replayed skip'), 1, 'cached skip', 0.01, 0, $fixture);
|
||||
$graph->setResult($branch, $id('replayed incomplete'), 2, 'cached incomplete', 0.01, 0, $fixture);
|
||||
|
||||
$json = $graph->encode();
|
||||
|
||||
expect($json)->not->toBeNull();
|
||||
|
||||
$originalHome = getenv('HOME');
|
||||
putenv('HOME='.$home);
|
||||
|
||||
try {
|
||||
$storage = new FileState(Storage::tempDir($projectRoot));
|
||||
} finally {
|
||||
putenv($originalHome === false ? 'HOME' : 'HOME='.$originalHome);
|
||||
}
|
||||
|
||||
expect($storage->write(Tia::KEY_GRAPH, (string) $json))->toBeTrue();
|
||||
|
||||
$process = new Process(
|
||||
['php', 'bin/pest', $fixture, '--tia'],
|
||||
$projectRoot,
|
||||
[
|
||||
'COLLISION_PRINTER' => 'DefaultPrinter',
|
||||
'COLLISION_IGNORE_DURATION' => 'true',
|
||||
'PARATEST' => 0,
|
||||
'PAO_DISABLE' => '1',
|
||||
'HOME' => $home,
|
||||
],
|
||||
);
|
||||
|
||||
$process->run();
|
||||
|
||||
$output = removeAnsiEscapeSequences($process->getOutput().$process->getErrorOutput());
|
||||
|
||||
// Both hooks throw, so the run stays green only if neither one ran.
|
||||
expect($output)->toContain('3 replayed')
|
||||
->and($output)->not->toContain('must not run for replayed tests')
|
||||
->and($output)->toContain('1 incomplete, 1 skipped, 1 passed')
|
||||
->and($process->getExitCode())->toBe(0);
|
||||
} finally {
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($home, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::CHILD_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $path) {
|
||||
$path->isDir() ? @rmdir($path->getPathname()) : @unlink($path->getPathname());
|
||||
}
|
||||
|
||||
@rmdir($home);
|
||||
}
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,108 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* Reading a baseline recorded on another branch.
|
||||
*
|
||||
* Without the default-branch fallback, the first `--tia` run on every new branch
|
||||
* re-runs a whole suite whose results the default branch already holds — once
|
||||
* per branch, forever, on any repository not named `main`.
|
||||
*
|
||||
* @see https://github.com/pestphp/pest/issues/1823
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('replays the default branch baseline on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($result->exitCode)->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays whatever the default branch is called', function (string $defaultBranch): void {
|
||||
$project = Project::make($defaultBranch);
|
||||
$project->seed($defaultBranch);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->with(['main', 'master', 'trunk', 'develop'])->skipOnWindows();
|
||||
|
||||
test('replays on a second new branch too', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->git()->switchTo('master');
|
||||
$project->git()->switchTo('feature-y', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The toll is one full run per new branch. It must not come back for the
|
||||
// second branch either.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('writes nothing on a second run on the same branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$project->snapshot();
|
||||
$result = $project->pest('--tia');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary())
|
||||
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays on a branch whose name contains slashes', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature/x/y', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toContain('feature/x/y');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('replays inside a worktree on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$worktree = $project->worktree('feature-worktree');
|
||||
|
||||
// Seeded against the worktree rather than the main checkout: a worktree's
|
||||
// `.git` is a file, so `Storage::originIdentity()` cannot read the remote
|
||||
// from it and the worktree resolves a storage key of its own. That gap is
|
||||
// separate from the branch fallback, and it is the fallback this row is
|
||||
// about — the worktree is checked out on a branch the baseline does not
|
||||
// name, which is the scenario from the issue.
|
||||
$project->seedFor($worktree, 'master');
|
||||
|
||||
$result = $project->pestIn($worktree, '--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,237 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Symfony\Component\Process\ExecutableFinder;
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* How the default branch gets named: declared in `tests/Pest.php`, autodetected
|
||||
* from the repository, or not answerable at all.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('a declared default branch beats autodetection', function (): void {
|
||||
// The repository autodetects `develop`, which holds no baseline. Only the
|
||||
// declaration in `tests/Pest.php` can reach the `master` one.
|
||||
$project = Project::make('develop', overlay: 'configured-default-branch');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a declared default branch that does not exist degrades to a full run', function (): void {
|
||||
$project = Project::make('master', overlay: 'unknown-default-branch');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing to fall back to, so everything runs — and no baseline is minted
|
||||
// under the name that resolved to nothing.
|
||||
expect($result->uncached())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master', 'feature-x']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the CI provider names the default branch where the checkout cannot', function (): void {
|
||||
// A CI checkout: `actions/checkout` fetches a single ref instead of cloning,
|
||||
// so there is no `origin/HEAD` for git to read the default branch from. The
|
||||
// event payload GitHub hands the job says it outright.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$project->write('.home/event.json', (string) json_encode([
|
||||
'repository' => ['default_branch' => 'master'],
|
||||
]));
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), [
|
||||
'GITHUB_EVENT_PATH' => $project->path('.home/event.json'),
|
||||
], '--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('GitLab names the default branch through its own variable', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), [
|
||||
'CI_DEFAULT_BRANCH' => 'master',
|
||||
], '--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a lone recorded baseline names the default branch', function (): void {
|
||||
// Nothing left to ask: no `origin/HEAD`, no CI provider, and an
|
||||
// `init.defaultBranch` that names a branch this repository does not have.
|
||||
// The graph holds exactly one baseline, and it is the only one any branch
|
||||
// could read — so it is the answer.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'main');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a default branch nothing can name is refused rather than guessed', function (): void {
|
||||
// Same checkout as above, without the graph that answered it. Guessing here
|
||||
// is what made this bug expensive: the guess reads no baseline at all, and
|
||||
// the output calls that a hit.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'main');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->output)->toContain('Tia mode could not determine the default branch.')
|
||||
->and($result->output)->toContain('git remote set-head origin --auto')
|
||||
->and($result->exitCode)->toBe(1, $result->describe())
|
||||
->and($project->graphExists())->toBeFalse();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('an init.defaultBranch naming a branch that exists is still trusted', function (): void {
|
||||
// The setting is the machine's, not the repository's — worth taking only
|
||||
// where the repository has a branch by that name. It does here, and with no
|
||||
// graph on disk it is the only source left, so the run must not be refused.
|
||||
$project = Project::make('master');
|
||||
$project->git()->unsetOriginHead();
|
||||
$project->git()->config('init.defaultBranch', 'master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->output)->not->toContain('could not determine the default branch')
|
||||
->and($project->branchKeys())->toBe(['feature-x']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a repository with no remote is refused rather than silently re-run', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Nothing can name the default branch, so every new branch would re-run the
|
||||
// whole suite with no explanation. Saying so beats doing that quietly. The
|
||||
// missing remote is the likeliest reason and gets named as such.
|
||||
expect($result->output)->toContain('Tia mode requires a repository with a remote.')
|
||||
->and($result->exitCode)->toBe(1, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a remote-less repository holding one baseline is not refused', function (): void {
|
||||
// The refusal above exists to stop a guess, not to demand a remote for its
|
||||
// own sake. With a baseline on disk there is nothing left to guess at.
|
||||
$project = Project::make('master');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a declared default branch stands in for a missing remote', function (): void {
|
||||
// The escape hatch the refusal above points at: with the branch named by
|
||||
// hand there is nothing left for a remote to answer.
|
||||
$project = Project::make('master', overlay: 'configured-default-branch');
|
||||
|
||||
$project->git()->removeOrigin();
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('tia still requires git', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// The soft default-branch resolver runs before the branch is named, and it
|
||||
// must not swallow this.
|
||||
expect($result->output)->toContain('The feature "Tia mode" requires "git".')
|
||||
->and($result->exitCode)->not->toBe(0);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a plain run outside a repository creates no baseline', function (): void {
|
||||
$project = Project::withoutGit();
|
||||
|
||||
$result = $project->pest();
|
||||
|
||||
expect($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
|
||||
->and($project->graphExists())->toBeFalse()
|
||||
->and($project->graphDir())->not->toBeDirectory();
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the default branch is resolved once per run, not once per test', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$git = (new ExecutableFinder)->find('git');
|
||||
|
||||
expect($git)->not->toBeNull();
|
||||
|
||||
// A `git` first on `PATH` that records what it was asked before handing over
|
||||
// to the real one.
|
||||
$log = $project->path('git-calls.log');
|
||||
$project->write('shim/git', implode("\n", [
|
||||
'#!/bin/sh',
|
||||
'echo "$@" >> '.escapeshellarg($log),
|
||||
'exec '.escapeshellarg((string) $git).' "$@"',
|
||||
'',
|
||||
]));
|
||||
chmod($project->path('shim/git'), 0755);
|
||||
|
||||
$result = $project->pestWithEnvironment($project->path(), [
|
||||
'PATH' => $project->path('shim').':'.getenv('PATH'),
|
||||
], '--tia');
|
||||
|
||||
$calls = file_exists($log) ? explode("\n", trim((string) file_get_contents($log))) : [];
|
||||
$resolutions = array_filter($calls, fn (string $call): bool => str_contains($call, 'symbolic-ref')
|
||||
|| str_contains($call, 'init.defaultBranch'));
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($calls)->not->toBeEmpty('the git shim was never reached')
|
||||
->and($resolutions)->toHaveCount(1, implode("\n", $calls));
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,133 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Tests\Fixtures\Tia\Project;
|
||||
|
||||
/**
|
||||
* What the fallback is allowed to touch.
|
||||
*
|
||||
* Reading another branch's baseline must stay a read: writes belong to the
|
||||
* branch that ran, and a branch that has no name of its own must not mint one.
|
||||
*/
|
||||
afterEach(function (): void {
|
||||
Project::destroyAll();
|
||||
});
|
||||
|
||||
test('narrows to the affected tests on a new branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
// Semantic, not cosmetic: PHP is hashed at the AST level, so a comment
|
||||
// would not register as a change at all.
|
||||
$project->write('app/Calculator.php', <<<'PHP'
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Calculator
|
||||
{
|
||||
public function add(int $a, int $b): int
|
||||
{
|
||||
return $a + $b;
|
||||
}
|
||||
|
||||
public function subtract(int $a, int $b): int
|
||||
{
|
||||
return $a - $b;
|
||||
}
|
||||
|
||||
public function multiply(int $a, int $b): int
|
||||
{
|
||||
return $a * $b;
|
||||
}
|
||||
}
|
||||
PHP);
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// Two of the three test files cover `Calculator`; the third replays from the
|
||||
// default branch's baseline.
|
||||
expect($result->affected())->toBe(4, $result->describe())
|
||||
->and($result->replayed())->toBe(2, $result->describe())
|
||||
->and($result->exitCode)->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode reads the fallback too', function (): void {
|
||||
$project = Project::make('master');
|
||||
|
||||
// A failure cached on the default branch. Filtered mode asks the graph which
|
||||
// test files are due a re-run, and that read has to reach the fallback as
|
||||
// well: without it a new branch sees nothing to do, reports green, and never
|
||||
// re-runs the failure — on every subsequent invocation.
|
||||
$project->seed('master', failing: ['adds two numbers']);
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
|
||||
// Selection is by file, so the failure's two siblings in `CalculatorTest`
|
||||
// come along; the other two test files stay out of the run entirely.
|
||||
expect($result->output)->toContain('from 1 previously unsuccessful test')
|
||||
->and($result->output)->not->toContain('No affected tests found')
|
||||
->and($result->tally())->toContain('2 passed');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('filtered mode finds nothing to do on a clean green feature branch', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--filtered');
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($result->output)->toContain('No affected tests found')
|
||||
->and($result->exitCode)->toBe(0, $result->describe())
|
||||
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('a detached HEAD replays without minting a branch key', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->detach();
|
||||
|
||||
$result = $project->pest('--tia');
|
||||
|
||||
// A detached HEAD has no branch of its own. The default branch is the
|
||||
// honest key, and no phantom one appears beside it.
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe())
|
||||
->and($project->branchKeys())->toBe(['master']);
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the branch that ran gets its own key and the default branch keeps its baseline', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
$project->pest('--tia');
|
||||
|
||||
$delta = $project->delta();
|
||||
|
||||
expect($project->branchKeys())->toBe(['master', 'feature-x'])
|
||||
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
|
||||
->and($delta->writtenCount())->toBe(0, $delta->summary());
|
||||
})->skipOnWindows();
|
||||
|
||||
test('the fallback reaches parallel workers', function (): void {
|
||||
$project = Project::make('master');
|
||||
$project->seed('master');
|
||||
|
||||
$project->git()->switchTo('feature-x', new: true);
|
||||
|
||||
$result = $project->pest('--tia', '--parallel', '--processes=2');
|
||||
|
||||
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
|
||||
->and($result->uncached())->toBe(0, $result->describe());
|
||||
})->skipOnWindows();
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
// The folder name creates the namespace segment `2fa`, which starts with a number.
|
||||
|
||||
it('never runs')->assertTrue(true);
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
// The file name creates the class `list`, which is a PHP keyword.
|
||||
|
||||
it('never runs')->assertTrue(true);
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
// The file name creates the class `int`, which is a name PHP reserves.
|
||||
|
||||
it('never runs')->assertTrue(true);
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
// The file name creates the class `2fa`, which starts with a number.
|
||||
|
||||
it('never runs')->assertTrue(true);
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
it('belongs to group one', function () {
|
||||
expect(true)->toBeTrue();
|
||||
})->group('one');
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
it('belongs to group three', function () {
|
||||
expect(true)->toBeTrue();
|
||||
})->group('three');
|
||||
@@ -0,0 +1,5 @@
|
||||
<?php
|
||||
|
||||
it('belongs to group two', function () {
|
||||
expect(true)->toBeTrue();
|
||||
})->group('two');
|
||||
@@ -0,0 +1,24 @@
|
||||
<?php
|
||||
|
||||
// Only ever run through `tests/Features/Tia.php`, against a seeded TIA graph.
|
||||
// Both hooks throw, so a replayed test that wrongly runs one fails the run.
|
||||
|
||||
beforeEach(function (): void {
|
||||
throw new RuntimeException('The beforeEach hook must not run for replayed tests.');
|
||||
});
|
||||
|
||||
afterEach(function (): void {
|
||||
throw new RuntimeException('The afterEach hook must not run for replayed tests.');
|
||||
});
|
||||
|
||||
test('replayed pass', function (): void {
|
||||
expect(true)->toBeTrue();
|
||||
});
|
||||
|
||||
test('replayed skip', function (): void {
|
||||
expect(true)->toBeTrue();
|
||||
});
|
||||
|
||||
test('replayed incomplete', function (): void {
|
||||
expect(true)->toBeTrue();
|
||||
});
|
||||
@@ -0,0 +1,181 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A git repository the scenario tests drive.
|
||||
*
|
||||
* Hermetic by construction: every invocation neutralises the machine's global
|
||||
* and system config and carries its own committer identity. Without that, an
|
||||
* ambient `init.defaultBranch = main` would answer questions the scenario means
|
||||
* to leave unanswered, and rows like "resolves to nothing, degrades safely"
|
||||
* would pass by accident.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GitRepo
|
||||
{
|
||||
/**
|
||||
* @var array<string, string>
|
||||
*/
|
||||
public const array ENV = [
|
||||
'GIT_CONFIG_GLOBAL' => '/dev/null',
|
||||
'GIT_CONFIG_SYSTEM' => '/dev/null',
|
||||
// `GIT_CONFIG_SYSTEM` does not cover every system-level file git reads:
|
||||
// Apple's git also loads one from inside Xcode, and it sets
|
||||
// `init.defaultBranch`. Only this suppresses all of them.
|
||||
'GIT_CONFIG_NOSYSTEM' => '1',
|
||||
'GIT_AUTHOR_NAME' => 'Pest Fixture',
|
||||
'GIT_AUTHOR_EMAIL' => 'fixture@pestphp.io',
|
||||
'GIT_COMMITTER_NAME' => 'Pest Fixture',
|
||||
'GIT_COMMITTER_EMAIL' => 'fixture@pestphp.io',
|
||||
];
|
||||
|
||||
public function __construct(public string $path) {}
|
||||
|
||||
/**
|
||||
* Initialises the repository on `$branch` and commits everything in it.
|
||||
*/
|
||||
public function init(string $branch = 'master'): void
|
||||
{
|
||||
$this->run(['init', '--quiet']);
|
||||
$this->run(['checkout', '--quiet', '-b', $branch]);
|
||||
$this->commit('Initial commit');
|
||||
}
|
||||
|
||||
public function commit(string $message): void
|
||||
{
|
||||
$this->run(['add', '-A']);
|
||||
$this->run(['commit', '--quiet', '--allow-empty', '-m', $message]);
|
||||
}
|
||||
|
||||
public function switchTo(string $branch, bool $new = false): void
|
||||
{
|
||||
$this->run($new ? ['checkout', '--quiet', '-b', $branch] : ['checkout', '--quiet', $branch]);
|
||||
}
|
||||
|
||||
public function rename(string $from, string $to): void
|
||||
{
|
||||
$this->run(['branch', '-m', $from, $to]);
|
||||
}
|
||||
|
||||
public function detach(): void
|
||||
{
|
||||
$this->run(['checkout', '--quiet', '--detach']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers an `origin`, which also decides the graph's storage key:
|
||||
* {@see Storage::projectKey()} prefers the remote's
|
||||
* identity over the path, so two checkouts of one repository — a worktree,
|
||||
* say — share a single graph.
|
||||
*/
|
||||
public function addOrigin(string $url = 'git@github.com:pestphp/tia-fixture.git'): void
|
||||
{
|
||||
$this->run(['remote', 'add', 'origin', $url]);
|
||||
}
|
||||
|
||||
public function removeOrigin(): void
|
||||
{
|
||||
$this->run(['remote', 'remove', 'origin']);
|
||||
}
|
||||
|
||||
/**
|
||||
* Points `refs/remotes/origin/HEAD` at a local branch — what
|
||||
* `git remote set-head` would write, without a remote to talk to.
|
||||
*/
|
||||
public function setOriginHead(string $branch): void
|
||||
{
|
||||
$this->run(['update-ref', 'refs/remotes/origin/'.$branch, 'HEAD']);
|
||||
$this->run(['symbolic-ref', 'refs/remotes/origin/HEAD', 'refs/remotes/origin/'.$branch]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops `refs/remotes/origin/HEAD` while keeping the remote-tracking branch
|
||||
* — what a CI checkout looks like. `actions/checkout` builds its working
|
||||
* copy with `git init` plus a single-ref `fetch` rather than a `clone`, and
|
||||
* only a `clone` writes that symbolic ref.
|
||||
*/
|
||||
public function unsetOriginHead(): void
|
||||
{
|
||||
$this->run(['symbolic-ref', '--delete', 'refs/remotes/origin/HEAD']);
|
||||
}
|
||||
|
||||
public function config(string $key, string $value): void
|
||||
{
|
||||
$this->run(['config', '--local', $key, $value]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch and returns its path.
|
||||
*/
|
||||
public function worktree(string $path, string $branch): string
|
||||
{
|
||||
$this->run(['worktree', 'add', '--quiet', '-b', $branch, $path]);
|
||||
|
||||
return $path;
|
||||
}
|
||||
|
||||
public function sha(): string
|
||||
{
|
||||
return $this->output(['rev-parse', 'HEAD']);
|
||||
}
|
||||
|
||||
public function currentBranch(): string
|
||||
{
|
||||
return $this->output(['rev-parse', '--abbrev-ref', 'HEAD']);
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchNames(): array
|
||||
{
|
||||
$names = explode("\n", $this->output(['for-each-ref', '--format=%(refname:short)', 'refs/heads']));
|
||||
|
||||
return array_values(array_filter($names, fn (string $name): bool => $name !== ''));
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function output(array $arguments): string
|
||||
{
|
||||
return trim($this->process($arguments, mustSucceed: true)->getOutput());
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function run(array $arguments): void
|
||||
{
|
||||
$this->process($arguments, mustSucceed: true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
private function process(array $arguments, bool $mustSucceed): Process
|
||||
{
|
||||
$process = new Process(['git', ...$arguments], $this->path, self::ENV);
|
||||
$process->setTimeout(30.0);
|
||||
$process->run();
|
||||
|
||||
if ($mustSucceed && ! $process->isSuccessful()) {
|
||||
throw new RuntimeException(sprintf(
|
||||
"git %s failed in [%s]:\n%s",
|
||||
implode(' ', $arguments),
|
||||
$this->path,
|
||||
$process->getErrorOutput().$process->getOutput(),
|
||||
));
|
||||
}
|
||||
|
||||
return $process;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,339 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* What one run did to the graph.
|
||||
*
|
||||
* The three tiers a run may respect, from the conformance matrix:
|
||||
*
|
||||
* - COMPLETE — may change everything.
|
||||
* - RESULTS-ONLY — may change only `baselines[<branch>].results` for tests that
|
||||
* actually ran. It may never remove an entry, nor move `sha`, `tree`,
|
||||
* `edges`, `files` or `fingerprint`.
|
||||
* - HARD-SUPPRESSED — may change nothing at all.
|
||||
*
|
||||
* {@see self::writtenCount()} is the load-bearing measurement, and the reason
|
||||
* {@see Project::sentinel()} exists: without falsified cached values there is no
|
||||
* way to tell "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class GraphDelta
|
||||
{
|
||||
/**
|
||||
* @param array<string, mixed>|null $before
|
||||
* @param array<string, mixed>|null $after
|
||||
*/
|
||||
public function __construct(
|
||||
private ?array $before,
|
||||
private ?array $after,
|
||||
) {}
|
||||
|
||||
public function graphWasCreated(): bool
|
||||
{
|
||||
return $this->before === null && $this->after !== null;
|
||||
}
|
||||
|
||||
public function graphWasDeleted(): bool
|
||||
{
|
||||
return $this->before !== null && $this->after === null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries whose stored values actually moved.
|
||||
*/
|
||||
public function writtenCount(): int
|
||||
{
|
||||
$written = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$before = $this->results($this->before, $branch);
|
||||
$after = $this->results($this->after, $branch);
|
||||
|
||||
foreach ($before as $testId => $entry) {
|
||||
if (! isset($after[$testId])) {
|
||||
continue;
|
||||
}
|
||||
|
||||
foreach (['status', 'time', 'assertions', 'message'] as $field) {
|
||||
if (($entry[$field] ?? null) !== ($after[$testId][$field] ?? null)) {
|
||||
$written++;
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return $written;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that appeared.
|
||||
*/
|
||||
public function added(): int
|
||||
{
|
||||
$added = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$added += count(array_diff(
|
||||
array_keys($this->results($this->after, $branch)),
|
||||
array_keys($this->results($this->before, $branch)),
|
||||
));
|
||||
}
|
||||
|
||||
return $added;
|
||||
}
|
||||
|
||||
/**
|
||||
* Result entries that were pruned.
|
||||
*/
|
||||
public function removed(): int
|
||||
{
|
||||
$removed = 0;
|
||||
|
||||
foreach ($this->branchKeys() as $branch) {
|
||||
$removed += count(array_diff(
|
||||
array_keys($this->results($this->before, $branch)),
|
||||
array_keys($this->results($this->after, $branch)),
|
||||
));
|
||||
}
|
||||
|
||||
return $removed;
|
||||
}
|
||||
|
||||
/**
|
||||
* The baseline keys after the run — the headline signal for the
|
||||
* default-branch rows, where a phantom key is the defect.
|
||||
*
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeys(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->after));
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeysBefore(): array
|
||||
{
|
||||
return array_keys($this->baselines($this->before));
|
||||
}
|
||||
|
||||
public function branchKeysMoved(): bool
|
||||
{
|
||||
return $this->branchKeysBefore() !== $this->branchKeys();
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the named branch's baseline is byte-identical — how a row proves
|
||||
* the fallback is read-only.
|
||||
*/
|
||||
public function baselineUntouched(string $branch): bool
|
||||
{
|
||||
return ($this->baselines($this->before)[$branch] ?? null)
|
||||
=== ($this->baselines($this->after)[$branch] ?? null);
|
||||
}
|
||||
|
||||
public function shaMoved(): bool
|
||||
{
|
||||
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'sha', $this->before) !== $this->baselineField($branch, 'sha', $this->after));
|
||||
}
|
||||
|
||||
public function treeMoved(): bool
|
||||
{
|
||||
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'tree', $this->before) !== $this->baselineField($branch, 'tree', $this->after));
|
||||
}
|
||||
|
||||
/**
|
||||
* Compares edges by the file paths they resolve to, not by file id: ids are
|
||||
* an implementation detail that shifts whenever `files` is rebuilt in a
|
||||
* different order.
|
||||
*/
|
||||
public function edgesMoved(): bool
|
||||
{
|
||||
return $this->edgeSets($this->before) !== $this->edgeSets($this->after);
|
||||
}
|
||||
|
||||
public function filesMoved(): bool
|
||||
{
|
||||
return $this->section($this->before, 'files') !== $this->section($this->after, 'files');
|
||||
}
|
||||
|
||||
public function fingerprintMoved(): bool
|
||||
{
|
||||
return $this->section($this->before, 'fingerprint') !== $this->section($this->after, 'fingerprint');
|
||||
}
|
||||
|
||||
public function structureMoved(): bool
|
||||
{
|
||||
if ($this->edgesMoved()) {
|
||||
return true;
|
||||
}
|
||||
if ($this->filesMoved()) {
|
||||
return true;
|
||||
}
|
||||
if ($this->fingerprintMoved()) {
|
||||
return true;
|
||||
}
|
||||
|
||||
return $this->branchKeysMoved();
|
||||
}
|
||||
|
||||
/**
|
||||
* Nothing moved at all.
|
||||
*/
|
||||
public function isHardSuppressed(): bool
|
||||
{
|
||||
return $this->before === $this->after;
|
||||
}
|
||||
|
||||
/**
|
||||
* Results may have moved for tests that ran; nothing structural did.
|
||||
*/
|
||||
public function isResultsOnly(): bool
|
||||
{
|
||||
return ! $this->graphWasCreated()
|
||||
&& ! $this->graphWasDeleted()
|
||||
&& ! $this->structureMoved()
|
||||
&& ! $this->shaMoved()
|
||||
&& ! $this->treeMoved()
|
||||
&& $this->removed() === 0
|
||||
&& $this->added() === 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* A one-line verdict, for failure messages.
|
||||
*/
|
||||
public function summary(): string
|
||||
{
|
||||
if ($this->graphWasCreated()) {
|
||||
return 'graph created';
|
||||
}
|
||||
|
||||
if ($this->graphWasDeleted()) {
|
||||
return 'graph deleted';
|
||||
}
|
||||
|
||||
$moved = [];
|
||||
|
||||
foreach (['edges', 'files', 'fingerprint'] as $section) {
|
||||
if ($this->{$section.'Moved'}()) {
|
||||
$moved[] = $section;
|
||||
}
|
||||
}
|
||||
|
||||
if ($this->branchKeysMoved()) {
|
||||
$moved[] = sprintf(
|
||||
'branchkeys(%s->%s)',
|
||||
implode('|', $this->branchKeysBefore()),
|
||||
implode('|', $this->branchKeys()),
|
||||
);
|
||||
}
|
||||
|
||||
return sprintf(
|
||||
'w=%d +%d -%d %s%s%s',
|
||||
$this->writtenCount(),
|
||||
$this->added(),
|
||||
$this->removed(),
|
||||
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
|
||||
$this->shaMoved() ? ' sha:changed' : '',
|
||||
$this->treeMoved() ? ' tree:changed' : '',
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, array<int, string>>
|
||||
*/
|
||||
private function edgeSets(?array $graph): array
|
||||
{
|
||||
$files = $this->section($graph, 'files');
|
||||
$sets = [];
|
||||
|
||||
foreach ($this->section($graph, 'edges') as $test => $ids) {
|
||||
if (! is_string($test)) {
|
||||
continue;
|
||||
}
|
||||
if (! is_array($ids)) {
|
||||
continue;
|
||||
}
|
||||
$paths = array_map(
|
||||
fn (mixed $id): string => is_int($id) && isset($files[$id]) && is_string($files[$id])
|
||||
? $files[$id]
|
||||
: '?'.json_encode($id),
|
||||
$ids,
|
||||
);
|
||||
|
||||
sort($paths);
|
||||
$sets[$test] = $paths;
|
||||
}
|
||||
|
||||
ksort($sets);
|
||||
|
||||
return $sets;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<mixed>
|
||||
*/
|
||||
private function section(?array $graph, string $key): array
|
||||
{
|
||||
$section = $graph[$key] ?? null;
|
||||
|
||||
return is_array($section) ? $section : [];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function baselines(?array $graph): array
|
||||
{
|
||||
$baselines = [];
|
||||
|
||||
foreach ($this->section($graph, 'baselines') as $branch => $baseline) {
|
||||
if (is_string($branch)) {
|
||||
$baselines[$branch] = $baseline;
|
||||
}
|
||||
}
|
||||
|
||||
return $baselines;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
* @return array<string, array<string, mixed>>
|
||||
*/
|
||||
private function results(?array $graph, string $branch): array
|
||||
{
|
||||
$results = $this->baselines($graph)[$branch]['results'] ?? null;
|
||||
|
||||
if (! is_array($results)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
$entries = [];
|
||||
|
||||
foreach ($results as $testId => $entry) {
|
||||
if (is_string($testId) && is_array($entry)) {
|
||||
$entries[$testId] = $entry;
|
||||
}
|
||||
}
|
||||
|
||||
return $entries;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, mixed>|null $graph
|
||||
*/
|
||||
private function baselineField(string $branch, string $field, ?array $graph): mixed
|
||||
{
|
||||
return $this->baselines($graph)[$branch][$field] ?? null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
/**
|
||||
* The outcome of one `pest` invocation against a fixture project.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final readonly class PestResult
|
||||
{
|
||||
/**
|
||||
* The run's output, with the terminal's escape sequences taken back out.
|
||||
*/
|
||||
public string $output;
|
||||
|
||||
/**
|
||||
* @param array<int, string> $arguments
|
||||
*/
|
||||
public function __construct(
|
||||
public array $arguments,
|
||||
string $output,
|
||||
public int $exitCode,
|
||||
) {
|
||||
$this->output = (string) preg_replace([
|
||||
'#\x1b[[][^A-Za-z]*[A-Za-z]#', // colours, cursor moves
|
||||
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#', // hyperlinks
|
||||
], '', $output);
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests whose cached result was replayed instead of executed.
|
||||
*/
|
||||
public function replayed(): int
|
||||
{
|
||||
return $this->recapFragment('replayed');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because the graph held nothing for them — the count that
|
||||
* betrays a fallback which never resolved.
|
||||
*/
|
||||
public function uncached(): int
|
||||
{
|
||||
return $this->recapFragment('uncached');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tests that ran because a file they depend on changed.
|
||||
*/
|
||||
public function affected(): int
|
||||
{
|
||||
return $this->recapFragment('affected');
|
||||
}
|
||||
|
||||
/**
|
||||
* The `Tests:` summary line, without its label or leading whitespace.
|
||||
*/
|
||||
public function tally(): string
|
||||
{
|
||||
if (preg_match('/^\s*Tests:\s+(.+)$/m', $this->output, $matches) !== 1) {
|
||||
return '';
|
||||
}
|
||||
|
||||
return trim($matches[1]);
|
||||
}
|
||||
|
||||
public function contains(string $needle): bool
|
||||
{
|
||||
return str_contains($this->output, $needle);
|
||||
}
|
||||
|
||||
/**
|
||||
* A description of the run, for failure messages that would otherwise say
|
||||
* only that 0 !== 6.
|
||||
*/
|
||||
public function describe(): string
|
||||
{
|
||||
return sprintf(
|
||||
"pest %s exited %d:\n%s",
|
||||
implode(' ', $this->arguments),
|
||||
$this->exitCode,
|
||||
$this->output,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read off the `Tests:` line rather than the whole output: the TIA headline
|
||||
* counts affected *files*, and matching that instead would be a quietly
|
||||
* wrong number.
|
||||
*/
|
||||
private function recapFragment(string $label): int
|
||||
{
|
||||
if (preg_match('/(\d+) '.preg_quote($label, '/').'/', $this->tally(), $matches) !== 1) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
return (int) $matches[1];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,645 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Fixtures\Tia;
|
||||
|
||||
use FilesystemIterator;
|
||||
use Pest\Factories\TestCaseFactory;
|
||||
use Pest\Plugins\Tia;
|
||||
use Pest\Plugins\Tia\ChangedFiles;
|
||||
use Pest\Plugins\Tia\FileState;
|
||||
use Pest\Plugins\Tia\Fingerprint;
|
||||
use Pest\Plugins\Tia\Graph;
|
||||
use Pest\Plugins\Tia\Storage;
|
||||
use Pest\Support\Str;
|
||||
use RecursiveDirectoryIterator;
|
||||
use RecursiveIteratorIterator;
|
||||
use RuntimeException;
|
||||
use Symfony\Component\Process\Process;
|
||||
|
||||
/**
|
||||
* A throwaway Pest project the TIA scenario tests drive.
|
||||
*
|
||||
* Two facts about Pest shape everything here:
|
||||
*
|
||||
* 1. `bin/pest` derives the project root from **the autoloader it finds**, not
|
||||
* from the working directory — `dirname($autoloadPath, 2)`. So the project
|
||||
* owns a real `vendor/autoload.php` and a real copy of `bin/pest` at the path
|
||||
* a composer install would have put them. A symlinked `vendor` would resolve
|
||||
* `__DIR__` straight back to the Pest repository, and every scenario would
|
||||
* silently measure the wrong project.
|
||||
* 2. TIA cannot *record* without pcov or Xdebug, and CI has neither. So a
|
||||
* scenario never records: {@see self::seed()} writes the graph a recording
|
||||
* run would have written, and the run under test exercises the read path.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class Project
|
||||
{
|
||||
/**
|
||||
* Test file → the source files a recording run would have linked it to. The
|
||||
* self-edge every test file gets is added on top of these.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array EDGES = [
|
||||
'tests/Unit/CalculatorTest.php' => ['app/Calculator.php'],
|
||||
'tests/Unit/GreeterTest.php' => ['app/Greeter.php'],
|
||||
'tests/Feature/CoversCalculatorTest.php' => ['app/Calculator.php'],
|
||||
];
|
||||
|
||||
/**
|
||||
* Test file → the descriptions it declares, in declaration order.
|
||||
*
|
||||
* @var array<string, array<int, string>>
|
||||
*/
|
||||
public const array TESTS = [
|
||||
'tests/Unit/CalculatorTest.php' => ['adds two numbers', 'subtracts two numbers'],
|
||||
'tests/Unit/GreeterTest.php' => ['greets a person', 'greets the world'],
|
||||
'tests/Feature/CoversCalculatorTest.php' => ['adds within a feature test', 'subtracts within a feature test'],
|
||||
];
|
||||
|
||||
/**
|
||||
* Every test in the fixture suite.
|
||||
*/
|
||||
public const int TOTAL_TESTS = 6;
|
||||
|
||||
/**
|
||||
* Every project scaffolded so far, so a row cannot leak one by failing
|
||||
* before its own cleanup.
|
||||
*
|
||||
* @var array<int, self>
|
||||
*/
|
||||
private static array $created = [];
|
||||
|
||||
private ?GitRepo $repo = null;
|
||||
|
||||
/**
|
||||
* @var array<string, mixed>|null
|
||||
*/
|
||||
private ?array $snapshot = null;
|
||||
|
||||
/**
|
||||
* @var array<int, string>
|
||||
*/
|
||||
private array $extraPaths = [];
|
||||
|
||||
/**
|
||||
* The root whose graph this project reads and writes.
|
||||
*
|
||||
* Its own, except where a row seeds a worktree: a worktree's `.git` is a
|
||||
* file rather than a directory, so {@see Storage} cannot read the remote
|
||||
* from it and resolves a storage key of its own.
|
||||
*/
|
||||
private string $graphRoot;
|
||||
|
||||
private function __construct(public readonly string $path)
|
||||
{
|
||||
$this->graphRoot = $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scaffolds a project whose default branch is `$branch`, and hands it back
|
||||
* checked out there.
|
||||
*
|
||||
* The repository is realistic on purpose: it has an `origin`, and an
|
||||
* `origin/HEAD` naming `$branch`, which is what a checkout of a real project
|
||||
* looks like and what the default branch is autodetected from.
|
||||
*/
|
||||
public static function make(string $branch = 'master', ?string $overlay = null): self
|
||||
{
|
||||
$project = self::scaffold($overlay);
|
||||
|
||||
$project->git()->init($branch);
|
||||
$project->git()->addOrigin();
|
||||
$project->git()->setOriginHead($branch);
|
||||
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Destroys every project scaffolded so far. Belongs in an `afterEach`.
|
||||
*/
|
||||
public static function destroyAll(): void
|
||||
{
|
||||
while (self::$created !== []) {
|
||||
array_pop(self::$created)->destroy();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A project that is not a git repository at all — for the rows that assert
|
||||
* TIA still demands git, and that a plain run does not care.
|
||||
*
|
||||
* Lives in the system temp directory, so it is outside any enclosing
|
||||
* repository, and owns a real `vendor` rather than a symlinked one, so its
|
||||
* baseline key cannot collide with another fixture's.
|
||||
*/
|
||||
public static function withoutGit(?string $overlay = null): self
|
||||
{
|
||||
return self::scaffold($overlay);
|
||||
}
|
||||
|
||||
public function git(): GitRepo
|
||||
{
|
||||
return $this->repo ??= new GitRepo($this->path);
|
||||
}
|
||||
|
||||
public function path(string $relative = ''): string
|
||||
{
|
||||
return $relative === '' ? $this->path : $this->path.DIRECTORY_SEPARATOR.$relative;
|
||||
}
|
||||
|
||||
/**
|
||||
* Overwrites a file in the project.
|
||||
*
|
||||
* Edits must be semantic: TIA hashes PHP at the AST level, so a
|
||||
* comment-only change is not a change at all.
|
||||
*/
|
||||
public function write(string $relative, string $contents): void
|
||||
{
|
||||
$path = $this->path($relative);
|
||||
$directory = dirname($path);
|
||||
|
||||
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
|
||||
}
|
||||
|
||||
if (file_put_contents($path, $contents) === false) {
|
||||
throw new RuntimeException(sprintf('Unable to write [%s].', $path));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a worktree for a new branch, scaffolded so `pest` can run in it, and
|
||||
* returns its path.
|
||||
*
|
||||
* The graph is shared with the main checkout, which is the whole point: both
|
||||
* resolve the same storage key, because {@see Storage} prefers the `origin`
|
||||
* identity over the path.
|
||||
*/
|
||||
public function worktree(string $branch): string
|
||||
{
|
||||
$path = $this->path.'-worktree-'.preg_replace('/[^a-z0-9]+/i', '-', $branch);
|
||||
|
||||
$this->git()->worktree($path, $branch);
|
||||
$this->scaffoldVendor($path);
|
||||
$this->extraPaths[] = $path;
|
||||
|
||||
return $path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in the project and returns what happened.
|
||||
*/
|
||||
public function pest(string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestIn($this->path, ...$arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `pest` in `$directory` — a worktree, say — against this project's
|
||||
* graph.
|
||||
*/
|
||||
public function pestIn(string $directory, string ...$arguments): PestResult
|
||||
{
|
||||
return $this->pestWithEnvironment($directory, [], ...$arguments);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, string> $environment
|
||||
*/
|
||||
public function pestWithEnvironment(string $directory, array $environment, string ...$arguments): PestResult
|
||||
{
|
||||
$process = new Process(
|
||||
[PHP_BINARY, $directory.'/vendor/pestphp/pest/bin/pest', ...$arguments],
|
||||
$directory,
|
||||
[
|
||||
...GitRepo::ENV,
|
||||
'COLLISION_PRINTER' => 'DefaultPrinter',
|
||||
'COLLISION_IGNORE_DURATION' => 'true',
|
||||
'PARATEST' => '0',
|
||||
'PAO_DISABLE' => '1',
|
||||
'HOME' => $this->home(),
|
||||
// Blanked for the same reason `GitRepo::ENV` blanks git's own
|
||||
// config: the default branch a CI provider reports is the one
|
||||
// *its* build is for. Pest's suite runs on GitHub Actions, so
|
||||
// without this every scenario would autodetect Pest's default
|
||||
// branch instead of the fixture's.
|
||||
'GITHUB_EVENT_PATH' => '',
|
||||
'CI_DEFAULT_BRANCH' => '',
|
||||
...$environment,
|
||||
],
|
||||
);
|
||||
|
||||
$process->setTimeout(180.0);
|
||||
$process->run();
|
||||
|
||||
return new PestResult(
|
||||
array_values($arguments),
|
||||
$process->getOutput().$process->getErrorOutput(),
|
||||
(int) $process->getExitCode(),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the graph a clean, green recording run on `$branch` would have
|
||||
* written, and remembers it as the snapshot the next {@see self::delta()}
|
||||
* compares against.
|
||||
*
|
||||
* Sentinelled by default: a row almost always wants to know which entries
|
||||
* were written, and only a real recording run's values can answer that.
|
||||
*
|
||||
* `$failing` names descriptions to record as failures — a clean green run
|
||||
* can never cache one, so a row that needs a cached failure to re-run has to
|
||||
* be handed it.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seed(string $branch, bool $sentinel = true, array $failing = []): void
|
||||
{
|
||||
$this->seedFor($this->path, $branch, $sentinel, $failing);
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeds the graph belonging to `$root` — a worktree, say, which resolves a
|
||||
* storage key of its own.
|
||||
*
|
||||
* @param array<int, string> $failing
|
||||
*/
|
||||
public function seedFor(string $root, string $branch, bool $sentinel = true, array $failing = []): void
|
||||
{
|
||||
$this->graphRoot = $root;
|
||||
|
||||
$changedFiles = new ChangedFiles($root);
|
||||
$sha = new GitRepo($root)->sha();
|
||||
|
||||
$graph = new Graph($root);
|
||||
$graph->setFingerprint(Fingerprint::compute($root));
|
||||
$graph->setRecordedAtSha($branch, $sha);
|
||||
|
||||
// Hashes the tree as it stands, so the run under test sees nothing as
|
||||
// changed — the same call the recording path makes.
|
||||
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
|
||||
|
||||
$graph->markKnownTestFiles(array_keys(self::EDGES));
|
||||
|
||||
foreach (self::EDGES as $testFile => $sourceFiles) {
|
||||
$graph->link($testFile, $testFile);
|
||||
|
||||
foreach ($sourceFiles as $sourceFile) {
|
||||
$graph->link($testFile, $sourceFile);
|
||||
}
|
||||
}
|
||||
|
||||
foreach (self::TESTS as $testFile => $descriptions) {
|
||||
foreach ($descriptions as $description) {
|
||||
$failed = in_array($description, $failing, true);
|
||||
|
||||
$graph->setResult(
|
||||
$branch,
|
||||
self::testId($testFile, $description),
|
||||
$failed ? 7 : 0,
|
||||
$failed ? 'cached failure' : '',
|
||||
0.05,
|
||||
1,
|
||||
$testFile,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
$json = $graph->encode();
|
||||
|
||||
if ($json === null) {
|
||||
throw new RuntimeException('Unable to encode the seeded graph.');
|
||||
}
|
||||
|
||||
if (! $this->state()->write(Tia::KEY_GRAPH, $json)) {
|
||||
throw new RuntimeException('Unable to persist the seeded graph.');
|
||||
}
|
||||
|
||||
$sentinel ? $this->sentinel() : $this->snapshot();
|
||||
}
|
||||
|
||||
/**
|
||||
* The id PHPUnit reports for a test in the fixture suite.
|
||||
*
|
||||
* Mirrors how Pest names generated test classes
|
||||
* ({@see TestCaseFactory}): a wrong id here shows up as
|
||||
* `0 replayed`, which every scenario asserts against.
|
||||
*/
|
||||
public static function testId(string $testFile, string $description): string
|
||||
{
|
||||
$basename = basename($testFile, '.php');
|
||||
$dotPosition = strpos($basename, '.');
|
||||
|
||||
if ($dotPosition !== false) {
|
||||
$basename = substr($basename, 0, $dotPosition);
|
||||
}
|
||||
|
||||
$relative = dirname(ucfirst($testFile)).DIRECTORY_SEPARATOR.$basename;
|
||||
|
||||
return 'P\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relative).'::'.Str::evaluable($description);
|
||||
}
|
||||
|
||||
/**
|
||||
* Falsifies every cached value, so the next {@see self::delta()} can tell
|
||||
* "wrote the same values back" from "wrote nothing".
|
||||
*
|
||||
* `assertions` is only ever falsified where it is already non-zero: risky,
|
||||
* skipped and incomplete statuses are *derived* from "performed no
|
||||
* assertions", so patching those would rewrite the status on replay and
|
||||
* destroy the very discriminator this exists to provide.
|
||||
*/
|
||||
public function sentinel(): void
|
||||
{
|
||||
$graph = $this->graph();
|
||||
|
||||
if ($graph === null) {
|
||||
throw new RuntimeException('There is no graph to sentinel.');
|
||||
}
|
||||
|
||||
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
|
||||
foreach (array_keys($baseline['results'] ?? []) as $testId) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
|
||||
|
||||
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
|
||||
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($graph, JSON_UNESCAPED_SLASHES));
|
||||
|
||||
$this->snapshot();
|
||||
}
|
||||
|
||||
/**
|
||||
* The decoded graph, or `null` when there is none.
|
||||
*
|
||||
* @return array<string, mixed>|null
|
||||
*/
|
||||
public function graph(): ?array
|
||||
{
|
||||
$json = $this->state()->read(Tia::KEY_GRAPH);
|
||||
|
||||
if ($json === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$graph = json_decode($json, true);
|
||||
|
||||
return is_array($graph) ? $graph : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array<int, string>
|
||||
*/
|
||||
public function branchKeys(): array
|
||||
{
|
||||
$baselines = $this->graph()['baselines'] ?? [];
|
||||
|
||||
return is_array($baselines) ? array_keys($baselines) : [];
|
||||
}
|
||||
|
||||
public function graphDir(): string
|
||||
{
|
||||
return $this->withHome(fn (): string => Storage::tempDir($this->graphRoot));
|
||||
}
|
||||
|
||||
public function graphExists(): bool
|
||||
{
|
||||
return is_file($this->graphDir().DIRECTORY_SEPARATOR.Tia::KEY_GRAPH);
|
||||
}
|
||||
|
||||
/**
|
||||
* Remembers the graph as it stands now.
|
||||
*/
|
||||
public function snapshot(): void
|
||||
{
|
||||
$this->snapshot = $this->graph();
|
||||
}
|
||||
|
||||
/**
|
||||
* What has happened to the graph since the last snapshot.
|
||||
*/
|
||||
public function delta(): GraphDelta
|
||||
{
|
||||
return new GraphDelta($this->snapshot, $this->graph());
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the `vendor` a composer install would have produced.
|
||||
*
|
||||
* Pest is mirrored in at `vendor/pestphp/pest` rather than pointed at,
|
||||
* because Pest locates things from where its own files sit:
|
||||
* `bin/pest` finds the project root by walking up from the autoloader it
|
||||
* loads, and a parallel run picks its worker binary — and with it the
|
||||
* worker's project root — from the directory the runner class was loaded
|
||||
* from. Deferring to the repository's copy would resolve both back to the
|
||||
* Pest repository, and every scenario would quietly measure that instead.
|
||||
*
|
||||
* Hardlinked where the filesystem allows it, so the mirror costs almost
|
||||
* nothing and can never drift from the working tree.
|
||||
*/
|
||||
public function scaffoldVendor(string $directory): void
|
||||
{
|
||||
$pestRoot = dirname(__DIR__, 3);
|
||||
$pest = $directory.'/vendor/pestphp/pest';
|
||||
|
||||
// `overrides`, `resources` and `stubs` come along because Pest loads
|
||||
// them relative to `src` — the same list `BootExcludeList` walks.
|
||||
foreach (['src', 'overrides', 'resources', 'stubs'] as $tree) {
|
||||
self::mirror($pestRoot.'/'.$tree, $pest.'/'.$tree);
|
||||
}
|
||||
|
||||
foreach (['pest', 'worker.php'] as $binary) {
|
||||
self::mirror($pestRoot.'/bin/'.$binary, $pest.'/bin/'.$binary);
|
||||
}
|
||||
|
||||
self::mirror($pestRoot.'/composer.json', $pest.'/composer.json');
|
||||
|
||||
// Pest's own autoloader, with the mirrored copy taking precedence: the
|
||||
// repository's `vendor` supplies PHPUnit, Symfony and the plugin
|
||||
// packages, none of which care where they are loaded from.
|
||||
file_put_contents($directory.'/vendor/autoload.php', sprintf(
|
||||
"<?php\n\n\$loader = require %s;\n\$loader->addPsr4('Pest\\\\', __DIR__.'/pestphp/pest/src', true);\n\nreturn \$loader;\n",
|
||||
var_export($pestRoot.'/vendor/autoload.php', true),
|
||||
));
|
||||
|
||||
// Invoking the binary directly skips composer's bin proxy, which is
|
||||
// what would otherwise define `$GLOBALS['_composer_bin_dir']`. Without
|
||||
// it `Pest\Plugin\Loader` looks for `vendor/bin/../pest-plugins.json`
|
||||
// relative to the working directory — so that is where the plugin list
|
||||
// goes, and mirroring the repository's keeps it in step with
|
||||
// composer.json. `vendor/bin` has to exist for the `..` in that path to
|
||||
// resolve, empty though it is.
|
||||
if (! is_dir($directory.'/vendor/bin') && ! @mkdir($directory.'/vendor/bin', 0755, true)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory.'/vendor/bin'));
|
||||
}
|
||||
|
||||
self::mirror($pestRoot.'/vendor/pest-plugins.json', $directory.'/vendor/pest-plugins.json');
|
||||
}
|
||||
|
||||
public function destroy(): void
|
||||
{
|
||||
foreach ([...$this->extraPaths, $this->path] as $path) {
|
||||
$this->remove($path);
|
||||
}
|
||||
}
|
||||
|
||||
private function home(): string
|
||||
{
|
||||
return $this->path.DIRECTORY_SEPARATOR.'.home';
|
||||
}
|
||||
|
||||
private function state(): FileState
|
||||
{
|
||||
return new FileState($this->graphDir());
|
||||
}
|
||||
|
||||
/**
|
||||
* `Storage` reads `HOME` from the environment, so the graph lands inside the
|
||||
* throwaway project instead of the developer's real `~/.pest`.
|
||||
*
|
||||
* @template TReturn
|
||||
*
|
||||
* @param callable(): TReturn $callback
|
||||
* @return TReturn
|
||||
*/
|
||||
private function withHome(callable $callback): mixed
|
||||
{
|
||||
$original = getenv('HOME');
|
||||
|
||||
putenv('HOME='.$this->home());
|
||||
|
||||
try {
|
||||
return $callback();
|
||||
} finally {
|
||||
putenv($original === false ? 'HOME' : 'HOME='.$original);
|
||||
}
|
||||
}
|
||||
|
||||
private static function scaffold(?string $overlay): self
|
||||
{
|
||||
$path = sys_get_temp_dir().DIRECTORY_SEPARATOR.'pest-tia-'.bin2hex(random_bytes(8));
|
||||
|
||||
if (! @mkdir($path, 0755, true) && ! is_dir($path)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
|
||||
}
|
||||
|
||||
// Realpathed because Pest realpaths its own project root, and macOS
|
||||
// hands out a symlinked temp directory.
|
||||
$real = realpath($path);
|
||||
|
||||
$project = new self($real === false ? $path : $real);
|
||||
|
||||
self::$created[] = $project;
|
||||
|
||||
self::copy(__DIR__.'/app', $project->path);
|
||||
|
||||
if ($overlay !== null) {
|
||||
self::copy(__DIR__.'/overlays/'.$overlay, $project->path);
|
||||
}
|
||||
|
||||
// `ChangedFiles` asks git what changed, so anything the scaffold writes
|
||||
// but the project does not own has to be invisible to it.
|
||||
$project->write('.gitignore', implode("\n", ['/vendor/', '/.home/', '/.phpunit.cache/', '']));
|
||||
|
||||
$project->scaffoldVendor($project->path);
|
||||
@mkdir($project->home(), 0755, true);
|
||||
|
||||
return $project;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirrors a file or directory, hardlinking where the filesystem allows it
|
||||
* and copying where it does not.
|
||||
*/
|
||||
private static function mirror(string $from, string $to): void
|
||||
{
|
||||
if (is_dir($from)) {
|
||||
foreach (self::contentsOf($from) as $path) {
|
||||
self::mirror($path->getPathname(), $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1));
|
||||
}
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
$directory = dirname($to);
|
||||
|
||||
if (! is_dir($directory) && ! @mkdir($directory, 0755, true) && ! is_dir($directory)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $directory));
|
||||
}
|
||||
|
||||
if (@link($from, $to) || @copy($from, $to)) {
|
||||
return;
|
||||
}
|
||||
|
||||
throw new RuntimeException(sprintf('Unable to mirror [%s] into [%s].', $from, $to));
|
||||
}
|
||||
|
||||
/**
|
||||
* @return iterable<\SplFileInfo>
|
||||
*/
|
||||
private static function contentsOf(string $directory): iterable
|
||||
{
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($directory, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::SELF_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $path) {
|
||||
if (! $path->isDir()) {
|
||||
yield $path;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static function copy(string $from, string $to): void
|
||||
{
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($from, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::SELF_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $path) {
|
||||
$target = $to.DIRECTORY_SEPARATOR.substr($path->getPathname(), strlen($from) + 1);
|
||||
|
||||
if ($path->isDir()) {
|
||||
if (! is_dir($target) && ! @mkdir($target, 0755, true) && ! is_dir($target)) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', $target));
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (! is_dir(dirname($target)) && ! @mkdir(dirname($target), 0755, true) && ! is_dir(dirname($target))) {
|
||||
throw new RuntimeException(sprintf('Unable to create [%s].', dirname($target)));
|
||||
}
|
||||
|
||||
if (! @copy($path->getPathname(), $target)) {
|
||||
throw new RuntimeException(sprintf('Unable to copy [%s].', $path->getPathname()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private function remove(string $path): void
|
||||
{
|
||||
if (! is_dir($path)) {
|
||||
return;
|
||||
}
|
||||
|
||||
$paths = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($path, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::CHILD_FIRST,
|
||||
);
|
||||
|
||||
foreach ($paths as $entry) {
|
||||
$entry->isDir() && ! $entry->isLink() ? @rmdir($entry->getPathname()) : @unlink($entry->getPathname());
|
||||
}
|
||||
|
||||
@rmdir($path);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Calculator
|
||||
{
|
||||
public function add(int $a, int $b): int
|
||||
{
|
||||
return $a + $b;
|
||||
}
|
||||
|
||||
public function subtract(int $a, int $b): int
|
||||
{
|
||||
return $a - $b;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Fixture\App;
|
||||
|
||||
final class Greeter
|
||||
{
|
||||
public function greet(string $name): string
|
||||
{
|
||||
return sprintf('Hello, %s!', $name);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"name": "pest/tia-fixture",
|
||||
"description": "Throwaway project the TIA scenario tests scaffold into a temp directory.",
|
||||
"license": "MIT",
|
||||
"require": {},
|
||||
"config": {
|
||||
"lock": true
|
||||
}
|
||||
}
|
||||
Generated
+17
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"_readme": [
|
||||
"Hand-written: the fixture project installs nothing. It exists so the TIA",
|
||||
"fingerprint has a composer.lock to hash, the way a real project does."
|
||||
],
|
||||
"content-hash": "0000000000000000000000000000000000",
|
||||
"packages": [],
|
||||
"packages-dev": [],
|
||||
"aliases": [],
|
||||
"minimum-stability": "stable",
|
||||
"stability-flags": {},
|
||||
"prefer-stable": true,
|
||||
"prefer-lowest": false,
|
||||
"platform": {},
|
||||
"platform-dev": {},
|
||||
"plugin-api-version": "2.6.0"
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
|
||||
bootstrap="vendor/autoload.php"
|
||||
cacheDirectory=".phpunit.cache"
|
||||
colors="true"
|
||||
failOnRisky="true"
|
||||
failOnWarning="false"
|
||||
>
|
||||
<testsuites>
|
||||
<testsuite name="default">
|
||||
<directory suffix="Test.php">./tests</directory>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
<source>
|
||||
<include>
|
||||
<directory suffix=".php">./app</directory>
|
||||
</include>
|
||||
</source>
|
||||
</phpunit>
|
||||
@@ -0,0 +1,15 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// A second test file over the same source file, so a `Calculator` edit narrows
|
||||
// to two of the three test files rather than to one.
|
||||
test('adds within a feature test', function (): void {
|
||||
expect((new Calculator)->add(10, 5))->toBe(15);
|
||||
});
|
||||
|
||||
test('subtracts within a feature test', function (): void {
|
||||
expect((new Calculator)->subtract(10, 5))->toBe(5);
|
||||
});
|
||||
@@ -0,0 +1,9 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// The fixture project has no composer autoloader of its own — its `vendor/`
|
||||
// holds nothing but a shim pointing back at Pest's. Requiring the two classes
|
||||
// here is enough for every test file in the suite.
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
@@ -0,0 +1,16 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Calculator;
|
||||
|
||||
// `test()` rather than `it()`: the harness seeds results by test id, and `it()`
|
||||
// would prefix every description with `it `, leaving Project::TESTS a step away
|
||||
// from what is written here.
|
||||
test('adds two numbers', function (): void {
|
||||
expect((new Calculator)->add(1, 2))->toBe(3);
|
||||
});
|
||||
|
||||
test('subtracts two numbers', function (): void {
|
||||
expect((new Calculator)->subtract(3, 1))->toBe(2);
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Fixture\App\Greeter;
|
||||
|
||||
test('greets a person', function (): void {
|
||||
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
|
||||
});
|
||||
|
||||
test('greets the world', function (): void {
|
||||
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
|
||||
});
|
||||
@@ -0,0 +1,11 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// Declared default branch. Wins over whatever the repository autodetects — the
|
||||
// escape hatch for a checkout with no `origin/HEAD` and a misleading
|
||||
// `init.defaultBranch`.
|
||||
pest()->tia()->defaultBranch('master');
|
||||
@@ -0,0 +1,10 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
require_once __DIR__.'/../app/Calculator.php';
|
||||
require_once __DIR__.'/../app/Greeter.php';
|
||||
|
||||
// A branch the repository does not have. Accepted as configured — a name that
|
||||
// resolves to no baseline degrades to a full run, which is safe.
|
||||
pest()->tia()->defaultBranch('nope');
|
||||
@@ -19,6 +19,10 @@ pest()->in('PHPUnit/GlobPatternTests/SubFolder2/*AsPattern.php')->use(CustomTest
|
||||
|
||||
pest()->in('Visual')->group('integration');
|
||||
|
||||
// Every row scaffolds a throwaway project and runs `pest` in it as a
|
||||
// subprocess, which is far too slow for the unit suite.
|
||||
pest()->in('Features/Tia')->group('integration');
|
||||
|
||||
// NOTE: global test value container to be mutated and checked across files, as needed
|
||||
$_SERVER['globalHook'] = (object) ['calls' => (object) ['beforeAll' => 0, 'afterAll' => 0]];
|
||||
|
||||
|
||||
@@ -66,8 +66,26 @@ describe('applyMigrationChanges()', function (): void {
|
||||
});
|
||||
|
||||
describe('rerun tracking', function (): void {
|
||||
beforeEach(function (): void {
|
||||
// `hasUnlocatedTestsToRerun()` stats each recorded file to tell a
|
||||
// deleted test apart from a live one, so the files have to exist.
|
||||
$this->projectRoot = sys_get_temp_dir().'/pest-tia-rerun-'.bin2hex(random_bytes(4));
|
||||
mkdir($this->projectRoot.'/tests/Feature', 0755, true);
|
||||
|
||||
touch($this->projectRoot.'/tests/Feature/FooTest.php');
|
||||
touch($this->projectRoot.'/tests/Feature/BarTest.php');
|
||||
});
|
||||
|
||||
afterEach(function (): void {
|
||||
@unlink($this->projectRoot.'/tests/Feature/FooTest.php');
|
||||
@unlink($this->projectRoot.'/tests/Feature/BarTest.php');
|
||||
@rmdir($this->projectRoot.'/tests/Feature');
|
||||
@rmdir($this->projectRoot.'/tests');
|
||||
@rmdir($this->projectRoot);
|
||||
});
|
||||
|
||||
it('reruns cached failures via their file', function (): void {
|
||||
$graph = new Graph(sys_get_temp_dir());
|
||||
$graph = new Graph($this->projectRoot);
|
||||
$graph->setResult('main', 'Tests\FooTest::it fails', 7, 'boom', 0.1, 1, 'tests/Feature/FooTest.php');
|
||||
$graph->setResult('main', 'Tests\BarTest::it passes', 0, '', 0.1, 1, 'tests/Feature/BarTest.php');
|
||||
|
||||
@@ -76,7 +94,7 @@ describe('rerun tracking', function (): void {
|
||||
});
|
||||
|
||||
it('flags cached failures whose file is unknown', function (): void {
|
||||
$graph = new Graph(sys_get_temp_dir());
|
||||
$graph = new Graph($this->projectRoot);
|
||||
$graph->setResult('main', 'Tests\EvalTest::it fails', 7, 'boom', 0.1, 1);
|
||||
|
||||
expect($graph->testFilesToRerun('main'))->toBeEmpty()
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
use Pest\Plugins\Tia\ResultCollector;
|
||||
use Pest\Subscribers\EnsureTiaAssertionsAreRecordedOnFinished;
|
||||
use Pest\Subscribers\EnsureTiaResultsAreCollected;
|
||||
use PHPUnit\Event\Code\TestDox;
|
||||
use PHPUnit\Event\Code\TestMethod;
|
||||
use PHPUnit\Event\Telemetry\Duration;
|
||||
use PHPUnit\Event\Telemetry\Info;
|
||||
use PHPUnit\Event\Telemetry\MemoryUsage;
|
||||
use PHPUnit\Event\Telemetry\System;
|
||||
use PHPUnit\Event\Telemetry\SystemCpuTimeMeter;
|
||||
use PHPUnit\Event\Telemetry\SystemGarbageCollectorStatusProvider;
|
||||
use PHPUnit\Event\Telemetry\SystemMemoryMeter;
|
||||
use PHPUnit\Event\Telemetry\SystemStopWatch;
|
||||
use PHPUnit\Event\Test\Finished;
|
||||
use PHPUnit\Event\Test\PreparationStarted;
|
||||
use PHPUnit\Event\TestData\DataFromDataProvider;
|
||||
use PHPUnit\Event\TestData\TestDataCollection;
|
||||
use PHPUnit\Metadata\MetadataCollection;
|
||||
|
||||
function tiaResultKeyTestMethod(?string $dataSetName): TestMethod
|
||||
{
|
||||
$testData = $dataSetName === null
|
||||
? TestDataCollection::fromArray([])
|
||||
: TestDataCollection::fromArray([DataFromDataProvider::from($dataSetName, '', '')]);
|
||||
|
||||
return new TestMethod(
|
||||
'Tests\Feature\OrderTest',
|
||||
'it prices an order',
|
||||
'/project/tests/Feature/OrderTest.php',
|
||||
1,
|
||||
new TestDox('Order', 'it prices an order', 'it prices an order'),
|
||||
MetadataCollection::fromArray([]),
|
||||
$testData,
|
||||
);
|
||||
}
|
||||
|
||||
function tiaResultKeyTelemetryInfo(): Info
|
||||
{
|
||||
$system = new System(
|
||||
new SystemStopWatch,
|
||||
new SystemMemoryMeter,
|
||||
new SystemGarbageCollectorStatusProvider,
|
||||
new SystemCpuTimeMeter,
|
||||
);
|
||||
|
||||
$zeroDuration = Duration::fromSecondsAndNanoseconds(0, 0);
|
||||
$zeroMemory = MemoryUsage::fromBytes(0);
|
||||
$zeroCpuTime = $system->snapshot()->userCpuTime();
|
||||
|
||||
return new Info(
|
||||
$system->snapshot(),
|
||||
$zeroDuration,
|
||||
$zeroMemory,
|
||||
$zeroDuration,
|
||||
$zeroMemory,
|
||||
$zeroCpuTime,
|
||||
$zeroCpuTime,
|
||||
$zeroCpuTime,
|
||||
$zeroCpuTime,
|
||||
$zeroCpuTime,
|
||||
$zeroCpuTime,
|
||||
);
|
||||
}
|
||||
|
||||
it('keys a result per dataset row rather than per method', function (): void {
|
||||
$collector = new ResultCollector;
|
||||
$subscriber = new EnsureTiaResultsAreCollected($collector);
|
||||
|
||||
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')));
|
||||
$collector->testPassed();
|
||||
$collector->finishTest();
|
||||
|
||||
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('fake')));
|
||||
$collector->testSkipped('the fake driver does not report balances');
|
||||
$collector->finishTest();
|
||||
|
||||
// Without the dataset in the key both rows write to `Class::method`, so the
|
||||
// second one overwrites the first and a replay hands every row the same
|
||||
// status — a passing row reported as skipped, or a failing one as passed.
|
||||
expect(array_keys($collector->all()))->toBe([
|
||||
'Tests\Feature\OrderTest::it prices an order#opp',
|
||||
'Tests\Feature\OrderTest::it prices an order#fake',
|
||||
]);
|
||||
});
|
||||
|
||||
it('records assertions against the same per-dataset key', function (): void {
|
||||
$collector = new ResultCollector;
|
||||
|
||||
new EnsureTiaResultsAreCollected($collector)->notify(
|
||||
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')),
|
||||
);
|
||||
$collector->testPassed();
|
||||
|
||||
new EnsureTiaAssertionsAreRecordedOnFinished($collector)->notify(
|
||||
new Finished(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp'), 7),
|
||||
);
|
||||
|
||||
expect($collector->all()['Tests\Feature\OrderTest::it prices an order#opp']['assertions'])->toBe(7);
|
||||
});
|
||||
|
||||
it('leaves a test without a dataset keyed by class and method', function (): void {
|
||||
$collector = new ResultCollector;
|
||||
|
||||
new EnsureTiaResultsAreCollected($collector)->notify(
|
||||
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod(null)),
|
||||
);
|
||||
$collector->testPassed();
|
||||
|
||||
expect(array_keys($collector->all()))->toBe(['Tests\Feature\OrderTest::it prices an order']);
|
||||
});
|
||||
@@ -47,6 +47,15 @@ describe('fromSql()', function (): void {
|
||||
->and(TableExtractor::fromSql('select * from information_schema.tables'))->toBeEmpty();
|
||||
});
|
||||
|
||||
it('does not leak int keys for numeric identifiers', function (): void {
|
||||
// `substring(x FROM 1 FOR 3)` is standard SQL, and the `1` matches the
|
||||
// FROM pattern. Collecting names as array keys makes PHP coerce the
|
||||
// numeric string to an int, which then violates the declared
|
||||
// list<string> and blows up Recorder::linkTable(string).
|
||||
expect(TableExtractor::fromSql('select substring(name from 1 for 3) from users'))
|
||||
->each->toBeString();
|
||||
});
|
||||
|
||||
it('returns nothing for non-DML statements', function (): void {
|
||||
expect(TableExtractor::fromSql('PRAGMA foreign_keys = ON'))->toBeEmpty()
|
||||
->and(TableExtractor::fromSql(''))->toBeEmpty()
|
||||
@@ -89,6 +98,14 @@ describe('fromMigrationSource()', function (): void {
|
||||
->toBe(['audits', 'events', 'sessions', 'settings', 'users']);
|
||||
});
|
||||
|
||||
it('does not leak int keys for numeric table names', function (): void {
|
||||
// A table named `123` is a legal quoted identifier. Collecting names as
|
||||
// array keys makes PHP coerce it to an int, breaking the declared
|
||||
// list<string>, so it must survive as a string rather than be dropped.
|
||||
expect(TableExtractor::fromMigrationSource("DB::table('123')->insert([]);"))
|
||||
->toBe(['123']);
|
||||
});
|
||||
|
||||
it('extracts tables from DB::table calls', function (): void {
|
||||
expect(TableExtractor::fromMigrationSource("DB::table('permissions')->insert([]);"))
|
||||
->toBe(['permissions']);
|
||||
|
||||
@@ -15,7 +15,7 @@ it('does not allow to add the same test description twice', function (): void {
|
||||
$testSuite->tests->set($method);
|
||||
})->throws(
|
||||
TestAlreadyExist::class,
|
||||
sprintf('A test with the description `%s` already exists in the filename `%s`.', 'bar', 'foo'),
|
||||
sprintf('A test with the description [%s] already exists in the filename [%s].', 'bar', 'foo'),
|
||||
);
|
||||
|
||||
it('does not allow static closures', function (): void {
|
||||
|
||||
@@ -29,30 +29,30 @@ $run = function () {
|
||||
$normalizedPath = (fn (string $path): string => str_replace('/', DIRECTORY_SEPARATOR, $path));
|
||||
|
||||
test('junit output', function () use ($normalizedPath, $run): void {
|
||||
$result = $run('tests/.tests/SuccessOnly.php');
|
||||
$result = $run('tests/Fixtures/Suites/SuccessOnly.php');
|
||||
|
||||
expect($result['testsuite']['@attributes'])
|
||||
->name->toBe('Tests\tests\SuccessOnly')
|
||||
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
|
||||
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
|
||||
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
|
||||
->tests->toBe('4')
|
||||
->assertions->toBe('4')
|
||||
->errors->toBe('0')
|
||||
->failures->toBe('0')
|
||||
->skipped->toBe('0')
|
||||
->and($result['testsuite']['testcase'])->toHaveCount(2)
|
||||
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
|
||||
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
|
||||
});
|
||||
|
||||
test('junit with parallel', function () use ($normalizedPath, $run): void {
|
||||
$result = $run('tests/.tests/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
|
||||
$result = $run('tests/Fixtures/Suites/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
|
||||
|
||||
expect($result['testsuite']['@attributes'])
|
||||
->name->toBe('Tests\tests\SuccessOnly')
|
||||
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
|
||||
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
|
||||
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
|
||||
->tests->toBe('1')
|
||||
->assertions->toBe('1')
|
||||
->errors->toBe('0')
|
||||
->failures->toBe('0')
|
||||
->skipped->toBe('0')
|
||||
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
|
||||
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
|
||||
});
|
||||
|
||||
@@ -24,13 +24,13 @@ test('parallel', function () use ($run): void {
|
||||
$file = file_get_contents(__FILE__);
|
||||
$file = preg_replace(
|
||||
'/\$expected = \'.*?\';/',
|
||||
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';",
|
||||
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';",
|
||||
$file,
|
||||
);
|
||||
file_put_contents(__FILE__, $file);
|
||||
}
|
||||
|
||||
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';
|
||||
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';
|
||||
|
||||
expect($output)
|
||||
->toContain("Tests: {$expected}")
|
||||
@@ -42,8 +42,8 @@ test('a parallel test can extend another test with same name', function () use (
|
||||
})->skipOnWindows();
|
||||
|
||||
test('parallel reports invalid datasets as failures', function () use ($run): void {
|
||||
expect($run('tests/.tests/ParallelInvalidDataset'))
|
||||
->toContain("A dataset with the name `missing.dataset` does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
|
||||
expect($run('tests/Fixtures/Suites/ParallelInvalidDataset'))
|
||||
->toContain("A dataset with the name [missing.dataset] does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
|
||||
->toContain('Tests: 1 failed, 1 passed (1 assertions)')
|
||||
->toContain('Parallel: 3 processes');
|
||||
})->skipOnWindows();
|
||||
@@ -58,3 +58,11 @@ test('parallel can have multiple exclude-groups', function () use ($run): void {
|
||||
expect((int) $doubleMatch[1])->toBeLessThan((int) $singleMatch[1])
|
||||
->and($doubleExclude)->toContain('Parallel: 3 processes');
|
||||
})->skipOnWindows();
|
||||
|
||||
test('parallel can have multiple groups', function () use ($run): void {
|
||||
$output = $run('tests/Fixtures/Suites/MultipleGroups', '--group=one', '--group=two');
|
||||
|
||||
expect($output)
|
||||
->toContain('Tests: 2 passed (2 assertions)')
|
||||
->toContain('Parallel: 3 processes');
|
||||
})->skipOnWindows();
|
||||
|
||||
@@ -27,7 +27,7 @@ test('allows to run a single test', function () use ($run, $snapshot): void {
|
||||
})->skipOnWindows();
|
||||
|
||||
test('allows to run a directory', function () use ($run, $snapshot): void {
|
||||
expect($run('tests/Fixtures'))->toContain($snapshot('allows-to-run-a-directory'));
|
||||
expect($run('tests/Fixtures/DirectoryWithTests'))->toContain($snapshot('allows-to-run-a-directory'));
|
||||
})->skipOnWindows();
|
||||
|
||||
it('disable decorating printer when colors is set to never', function () use ($snapshot): void {
|
||||
|
||||
@@ -10,7 +10,7 @@ function normalize_windows_os_output(string $text): string
|
||||
}
|
||||
|
||||
test('visual snapshot of team city', function (string $testFile): void {
|
||||
$testsPath = dirname(__DIR__)."/.tests/$testFile";
|
||||
$testsPath = dirname(__DIR__)."/Fixtures/Suites/$testFile";
|
||||
|
||||
$snapshot = implode(DIRECTORY_SEPARATOR, [
|
||||
dirname(__DIR__),
|
||||
|
||||
@@ -8,7 +8,7 @@ test('filter works with unicode characters in filename', function (): void {
|
||||
$process = new Process([
|
||||
'php',
|
||||
'bin/pest',
|
||||
'tests/.tests/StraßenTest.php',
|
||||
'tests/Fixtures/Suites/StraßenTest.php',
|
||||
'--colors=never',
|
||||
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);
|
||||
|
||||
@@ -26,7 +26,7 @@ test('filter with unicode regex matches unicode filename', function (): void {
|
||||
'php',
|
||||
'bin/pest',
|
||||
'--filter=.*Straß.*',
|
||||
'tests/.tests/',
|
||||
'tests/Fixtures/Suites/',
|
||||
'--colors=never',
|
||||
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user