Compare commits

...

8 Commits

Author SHA1 Message Date
nuno maduro 7bfb2a6185 chore: upgrade snapshots 2026-08-06 22:24:26 +01:00
nuno maduro ca56ca5216 dw 2026-08-06 22:19:47 +01:00
nuno maduro b49ba062d5 wq 2026-08-06 22:05:15 +01:00
nuno maduro 09699847a2 wqd 2026-08-06 20:53:09 +01:00
nuno maduro 545d0c2784 wip 2026-08-06 20:48:22 +01:00
nuno maduro b795af3b1b wip 2026-08-06 17:52:04 +01:00
nuno maduro a40cc6bc0e more tests 2026-08-06 16:35:03 +01:00
nuno maduro 4d3d0105b7 chore: style 2026-08-06 16:02:36 +01:00
50 changed files with 3635 additions and 801 deletions
+16
View File
@@ -16,3 +16,19 @@ composer test:integration # visual and snapshot tests
composer test # everything CI runs, in CI's order
composer update:snapshots # only when a test was added or removed
```
## TIA scenario tests
`tests/Features/Tia/*` scaffold a throwaway git project, run a real `pest` subprocess against it, and diff the TIA graph it wrote. They exist because TIA's contract is about what a run *writes* — replay, branch keys, and the COMPLETE / RESULTS-ONLY / HARD-SUPPRESSED tiers are invisible to ordinary assertions, and every case used to be measured by hand against a playground app.
Add one whenever a change touches branch resolution, replay, filtered mode, or the write tiers. How:
- `Project::make('master')` scaffolds; `seed('master')` writes a graph and sentinels every cached result (`time=9.999`, `assertions=42`) so any rewrite shows up.
- `$project->pest('--tia', …)` runs it; `$project->delta()` compares against that snapshot. `writtenCount()` is the discriminator — `0` means "replayed", not "wrote the same values". `mutateGraph()` bends one entry; overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`.
- Keep expectations driver-independent: a cold recording run needs pcov/xdebug and behaves differently without one. Seed a graph instead of recording one.
Run them by file (a directory argument finds nothing) or by `--filter`:
```bash
php bin/pest tests/Features/Tia/PartialRunWriteTier.php
```
+3 -3
View File
@@ -90,9 +90,9 @@ these are one fix or two.
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
| Replay clobbers cached `time` | write path | `0.058``0.001`; timing data degrades toward zero across runs |
| `pest --parallel` without `--tia` writes nothing | G3, G6 | Sequential `pest --filter=…` does record. Parallel CI contributes nothing to the cached-failure replay path |
| `--tia --parallel --filter`/`--shard` record nothing | G2, G8 | More conservative than the results-only contract requires |
| ~~Replay clobbers cached `time`~~ | write path | **Struck in phase four — does not reproduce**, sequentially or in parallel. Both write paths route through `resultTime()`; a replay with every cached `time` sentinelled writes nothing |
| ~~`pest --parallel` without `--tia` writes nothing~~ | G3, G6 | **Closed in phase four.** Workers flush their results through `requestWorkerResults()`, so a parallel run refreshes and prunes exactly like the sequential run of the same command |
| ~~`--tia --parallel --filter`/`--shard` record nothing~~ | G2, G8 | **Closed in phase four** (partial parity landed first, complete parity with the row above) |
| `--min=50` without `--coverage` is a silent no-op | — | — |
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
+312
View File
@@ -0,0 +1,312 @@
# TIA deep audit — phase five
## Your task
Phases one through four fixed the defects that were *reported*. This phase is the opposite shape: go
looking. Read TIA's read/write path adversarially, find what is wrong or fragile, and leave behind a
scenario suite that covers the edges nobody has exercised yet.
Two deliverables, both required:
1. **A ranked findings list.** Every finding needs a *reproduction*, not a reading of the code. A
finding you cannot reproduce is a hypothesis — say so and rank it separately.
2. **New scenario tests** in `tests/Features/Tia/*`, covering the edges you probed. Rows that pass go
in the repo (they are the regression net). Rows that fail stay in your scratchpad until the fix
lands — **never commit a red test.**
Work in passes, and **report between passes** rather than at the very end:
- **Pass A** — reproduce the leads in Part 3 below. Report which are real.
- **Pass B** — your own hunt: the invariants in Part 2, attacked with inputs nobody tried.
- **Pass C** — fixes, smallest first, each with the row that pins it.
Fix what is clearly a defect with an obvious correct answer. **Stop and ask** when the fix is a
behaviour *choice* (what should TIA do when two runs race for one graph?) — those are Nuno's calls,
and a reproduction with a crisp yes/no question is worth more than a guessed fix.
---
## Part 1 — The harness
### 1.1 What TIA is
Test Impact Analysis. `pest --tia` records a dependency graph (test file → source files it touched)
plus a per-branch baseline of results, then on later runs replays the tests whose dependencies did not
change. State lives in a per-project dir under `~/.pest/tia/`; `graph.json` is the whole thing.
Source of truth: `src/Plugins/Tia.php` (the plugin, ~1900 lines) and `src/Plugins/Tia/Graph.php` (the
graph model + read/write API). Supporting: `src/Plugins/Tia/ChangedFiles.php` (git), `Fingerprint.php`
(environment/structure hashing), `State.php` (the state dir).
Read `PLAN.md`, `PLAN_PHASE_TWO.md`, `PLAN_PHASE_THREE.md`, `PLAN_PHASE_FOUR.md` first — in that
order. They carry the history, the tier contract, and the decisions already made. Struck-through rows
are fixed; do not re-report them.
### 1.2 The scenario harness
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the graph it wrote. Everything lives in `tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `seedFor($root, …)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `path($rel)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroy()`, `destroyAll()`, `SEQUENTIAL_AND_PARALLEL` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()` |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`); `Project::EDGES` and
`Project::TESTS` describe the graph `seed()` writes. `Project::testId($file, $description)` builds a
result key. Overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`
that is how you configure `pest()->tia()->…` for a scenario.
**If the fixture cannot express your case, extend the fixture.** A new `GitRepo` verb or a new overlay
is a legitimate part of the deliverable — several findings below need one. Do not water down a
scenario to fit the current helpers.
### 1.3 The sentinel discriminator — read before writing any assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`, `assertions=42`**
(only where assertions are non-zero — risky/skipped/incomplete statuses are *derived* from "performed
no assertions", so falsifying those would rewrite the status on replay), then snapshots. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
- A replayed suite reports inflated assertion counts (6 tests × 42). That is the sentinel, not a bug.
The three write tiers, unchanged since phase two:
- **COMPLETE** — may change everything.
- **RESULTS-ONLY** — may change only `baselines[<branch>].results` for tests that ran; never removes
an entry, never adds a result for a test file absent from `edges`, never touches
`sha`/`tree`/`edges`/`files`/`fingerprint`.
- **HARD-SUPPRESSED** — may change nothing.
### 1.4 Running them
They are in the `integration` group (`tests/Pest.php`), so `composer test:unit` skips them. **A
directory argument finds nothing** — pass files, space-separated, as separate argv entries:
```bash
PAO_DISABLE=1 php84 bin/pest tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/PartialRunWriteTier.php \
tests/Features/Tia/CompleteRunWriteTier.php tests/Features/Tia/FilteredMode.php
PAO_DISABLE=1 php bin/pest <same list>
```
**Both interpreters must be green.** `php84` is 8.4.x with **no pcov** — that is what CI has
(`.github/workflows/tests.yml` sets `coverage: none`). `php` is 8.5.x with pcov — that is the dev
machine. Any assertion that depends on a coverage driver passes locally and fails in CI. Concretely:
- A **cold recording run writes no graph at all** without pcov/xdebug (it prints `Running in TIA mode,
however TIA is skipped as it needs ext-pcov or Xdebug`). **Seed a graph, never record one**, unless
the driver *is* the point of the row.
- A **PHP source file edit** driverless triggers `Detected PHP source changes but no coverage driver
is available` → full suite, `affected=0`. Edit *test* files, not `app/` files.
`PAO_DISABLE=1` is mandatory on every pest invocation and every probe: `laravel/pao` emits JSON under
agents and corrupts the captured output.
Baseline before you touch anything: **72 passed** on both interpreters, at `HEAD` plus the phase-four
working-tree changes. If that number does not reproduce, stop and say so.
### 1.5 Measure before you assert
Never guess an expectation from reading the code. Probe first, in your scratchpad:
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`; `Project::destroyAll()`
at the end. Run every probe on **both** interpreters before believing it.
### 1.6 Ground rules
- **Do not** run `composer test`. It takes minutes and you do not need it.
- **Do not** run `composer update:snapshots`. `tests/.snapshots/success.txt` and the tally in
`tests/Visual/Parallel.php` encode the whole suite's result, so every row you add breaks them.
That is expected — **report that they need regenerating and let Nuno run it.**
- **Do not commit.** Leave the tree dirty.
- **Do not touch the playground's `vendor/`.** If a finding truly needs the playground (25 real tests,
real timings), say so and ask — syncing it is a manual step Nuno owns.
- Run `vendor/bin/phpstan analyse <the files you touched> --memory-limit=-1 --no-progress` and
`vendor/bin/pint <the files you touched>` before reporting. **Scope both to files you changed** —
Nuno edits `src/` live, and a repo-wide fixer will revert his work.
- The fixture projects **hardlink `src/`**. If you edit `src/` while a scenario run is in flight,
those rows go red for no reason. Finish the edit, then run.
- Keep scratch probes out of the repo.
- **Never weaken an existing assertion to make something pass.** If an existing row contradicts your
fix, that is a finding: report the contradiction and ask.
---
## Part 2 — The invariants to attack
These are the properties TIA is supposed to have. Each one is a place to hunt: construct the input
that breaks it.
1. **Parity.** `pest <args>` and `pest --parallel --processes=N <args>` must leave the **same graph**
and reach the same tally. This is a hard rule from Nuno — sequential and parallel must *always*
agree. `Project::SEQUENTIAL_AND_PARALLEL` is the dataset that encodes it; consider making every
new write-path row use it. Vary `--processes` (1, 2, 8 — more processes than test files).
2. **The tiers hold.** Every command lands in exactly one of COMPLETE / RESULTS-ONLY /
HARD-SUPPRESSED, and stays inside it. Combinations are where this frays: `--fresh --parallel
--filter`, `--bail --shard`, `--filtered` plus an explicit path, `--tia --no-tia`, `--retry`.
3. **Replay is faithful.** A replayed test reports the same status, message, time and assertion count
as the recorded run — and replay itself writes nothing. Statuses beyond pass/fail are the soft
spot: skipped, incomplete, risky, notice, deprecation, warning, todo, and a test that failed with a
multi-line message.
4. **Reads never write.** No read path may mint a baseline key, move a `sha`, or create the state dir.
A project that has never run TIA must gain nothing from a plain `pest` run.
5. **A branch never corrupts another branch's baseline.** Writes land on the branch that ran, and
only there. Reads may *layer* the default branch under the current one (phase four, B1) — but that
layering must not leak into a write.
6. **Nothing is unbounded.** Baseline keys, `files`, `edges`, worker partials, state files: something
must eventually reclaim them, or the graph grows forever.
7. **A hostile state dir cannot break a run.** Corrupt, truncated, empty, wrong-schema, read-only,
absent, or *someone else's* `graph.json` — the suite still runs and exits on the tests' merit.
8. **Git shapes are all handled.** Detached HEAD (read-only, per phase four B2), worktrees, no commits
yet, no `origin`, no `origin/HEAD`, submodules, a repo whose root is above the pest project (that
one panics deliberately — `TiaRequiresRepositoryRoot`), renamed branches, deleted branches.
---
## Part 3 — Leads to reproduce first (Pass A)
These came out of reading the phase-four diff. **Each is a hypothesis, not a finding** — several may
turn out to be fine. Reproduce or refute each, in order, and report the measured delta either way.
### L1 — the per-entry fallback may resurrect a pruned or deleted test · **highest value**
Phase four made `Graph::baselineFor()` layer the branch's results **over** the default branch's. Two
consequences worth probing:
- `pruneStaleResults()` unsets an entry from `baselines[branch].results`. The very next read layers the
**fallback's** entry for that same test id back in. So a delete may not stick from a branch's point
of view.
- `hasUnlocatedTestsToRerun()` returns true when a *failing* cached result names a file that no longer
exists on disk — and that forces a **full suite**. If a feature branch deletes a test file that
fails on the default branch, the merged read still carries master's entry pointing at the now-absent
file. Suspected symptom: **that branch runs the full suite forever.**
Probe: seed master with a failing test (`seed('master', failing: [...])`), branch off, delete the test
file that holds it, run `--tia`, and compare `replayed`/`uncached`/`affected` against the same shape
where the failure is on the branch instead. Then the mirror case with a green deleted test.
### L2 — environment drift clears one branch's results and the fallback serves them right back
`reconcileFingerprint()` on environmental drift calls `$graph->clearResults($this->branch)` and warns
`results dropped, edges reused`. On a feature branch that clears only the *branch's* results — the
layered read then re-serves the default branch's results, which were recorded under the **old**
environment. Suspected symptom: the drop is a no-op on any branch that is not the default one.
Probe: `pestWithEnvironment()` to shift whatever `Fingerprint` reads as environmental (check
`Fingerprint::environmentalDrift()` for the exact keys), on the default branch vs a feature branch,
and compare what survives.
### L3 — `sha`/`tree` may be taken from a different commit than the results
`baselineFor()` takes `sha` from the branch when non-null and otherwise from the fallback; `tree`
likewise when the branch's is empty. So a branch can end up computing "what changed since" against the
**default branch's** recorded sha while reading its own results — or vice versa. Is there a shape where
that under-reports changed files (a test replays that should have run)? That is the dangerous
direction: a false replay is a lie about a passing test.
Probe: seed master, commit a test edit on the branch so the shas genuinely differ, and check whether
the edited test is treated as affected.
### L4 — pruning from merged worker partials
Phase four made a complete `--parallel` run write and prune from merged worker results. The stated
safety net is that `pruneStaleResults()` only prunes files it saw results for, and that a truncated
worker sets results-only. Try to defeat it: a worker that reports results for a test file it did not
finish. Candidate shapes — a fatal error mid-file (not an assertion failure), `exit()` inside a test,
an uncaught error in an `afterEach`, a test that kills its own process, `--stop-on-failure` variants,
`--processes` greater than the number of test files.
### L5 — two runs racing for one `graph.json`
`State::write()` has no locking. Two pest processes on one project (a watcher plus a manual run, two CI
jobs sharing a cache dir, `--parallel` where the parent writes while a straggler worker flushes) can
lose an update or interleave. Probe by launching two `pest --tia` subprocesses concurrently against one
project and diffing. **This is likely a design decision, not a bug** — if you reproduce a lost update,
report it as a question (accept last-writer-wins, or lock?), do not invent a locking scheme.
### L6 — a detached HEAD still purges on structural drift
Phase four made a detached HEAD read-only *for writes* (`saveGraph()` refuses). But
`reconcileFingerprint()` deletes the whole graph on structural drift (`Tia.php`, the
`state->delete(KEY_GRAPH)` in the structural branch) before any write happens. So `--tia` from a
detached checkout with a changed `composer.lock` can still wipe the default branch's baseline. Confirm
it, then ask: should the detached-HEAD guard cover the purge too?
### L7 — statuses that may not round-trip
`PLAN.md` §5 claims warnings and deprecations record as `status=0`, and that codes `6` and `4` look
unreachable. `Graph::getResult()` maps 08 to `TestStatus`. Verify each status end to end: record it,
replay it, and check the replayed run reports the same thing — including the message, the exit code,
and whether `shouldRerunStatus()` decides to re-execute it. `failOnRisky` / `failOnSkipped` /
`displayDetailsOn*` change that decision, so an overlay that flips those config flags is part of this.
A status that replays as a pass would be the most serious class of bug in TIA.
### L8 — branch-key hygiene
Nothing appears to reclaim baseline keys. Probe: create and delete 5 branches, rename one
(`GitRepo::rename()`), and check what `branchKeys()` holds afterwards. Also try names that stress the
JSON keying: `feature/x/y` (already covered), a name differing from another only in case (macOS is
case-insensitive — does the key match the ref?), a name with a space or a unicode character, a branch
literally called `HEAD`, and a very long name. Then: is unbounded growth acceptable, or does this need
a cap / GC? Ask rather than build.
### L9 — `soleRecordedBranch()` as a fallback source
When config, CI env and git all fail to name a default branch, resolution falls back to "the only
branch in the graph". If that sole key was minted by a *narrowed* run on a feature branch (which phase
four's B1 made a live possibility), the fallback now names a feature branch, and every other branch
layers **its** results underneath. Probe: `withoutGit()` or `removeOrigin()` + no config, with a graph
whose only key is `feature-x`.
### L10 — the state dir as an adversary
Beyond corrupt JSON (fixed in phase four by deleting it): a valid-JSON graph with `schema: 2`; a graph
whose `files` and `edges` disagree; `results` entries with a `file` pointing outside the project root
or at an absolute path from another machine; a `graph.json` that is a directory; a state dir with no
write permission; `$HOME` unset. Each should degrade to "run the tests", never crash and never write
garbage.
---
## Part 4 — Reporting
**Between passes**, not just at the end. Per finding:
- **Reproduced (yes / no / hypothesis only)**, with the exact command and the measured
`tally` + `delta()->summary()` on **both** interpreters.
- **Severity**, and say why in one line. The scale that matters here: *a test wrongly replayed as
passing* (worst) > *cache silently useless, full suite forever* > *graph grows / stale data* >
*cosmetic*.
- **Fixed / deferred / needs-a-decision**, and the row that pins it.
- For anything needing a decision: **one yes/no question**, no essay.
Close with:
1. The count of `tests/Features/Tia/*` green on **both** `php84` (no pcov) and `php85` (pcov), against
the 72 baseline.
2. Every row you added, and what invariant from Part 2 it defends.
3. Which findings are still open, as yes/no questions.
4. That `tests/.snapshots/success.txt` and the `tests/Visual/Parallel.php` tally need regenerating —
**do not regenerate them.**
5. **What you looked at and found solid.** A list of attacks that did not break anything is a real
result: it tells the next phase where not to spend its time.
+351
View File
@@ -0,0 +1,351 @@
# TIA defect sweep — phase four
## Your task
Five defects in TIA's read/write path, found while building the repo's TIA scenario suite. One is
confirmed and load-bearing (**B1**); four need a decision before a fix (**B2****B5**).
Work **one bug at a time, in order**, and for each:
1. **Reproduce it as a repo test first.** The reproduction is the deliverable even when the fix is
deferred — a red test that pins the exact symptom is worth more than a prose report. Do not commit
a red test to the suite; keep it in a scratch file until the fix lands (see Part 1.4).
2. Confirm the measured numbers in this file still hold. They were taken at commit `4d3d0105` plus
the two uncommitted changes described in Part 2. If a number has moved, say so and stop.
3. Fix, then re-run **the whole `tests/Features/Tia/*` set on two interpreters** (Part 1.3).
4. **HARD STOP after B1.** Report the diff and wait — B1's fix changes `Graph`'s read semantics for
every caller, and Nuno wants to see it before B2B5 pile on top.
Per `CLAUDE.md`: do not run `composer test`, and do not regenerate snapshots unless told. Do not
commit. Do not touch the playground's `vendor/`.
---
## Part 1 — The harness
### 1.1 What exists
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the TIA graph it wrote. Everything lives in
`tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroyAll()` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()` |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`), with `Project::EDGES` and
`Project::TESTS` describing the graph `seed()` writes. `Project::testId($file, $description)` builds
a result key.
### 1.2 The sentinel discriminator — read this before writing an assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`,
`assertions=42`** (non-zero assertion counts only — risky/skipped/incomplete statuses are *derived*
from "performed no assertions", so falsifying those would rewrite the status on replay), and takes a
snapshot. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
Tiers, unchanged since phase two: **COMPLETE** may change everything · **RESULTS-ONLY** may change
only `baselines[<branch>].results` for tests that ran, never removing an entry, never adding a result
for a test file absent from `edges`, never touching `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
### 1.3 Running them
They are in the `integration` group (`tests/Pest.php:22`), so `composer test:unit` skips them. A
**directory argument finds nothing** — pass files:
```bash
F="tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/PartialRunWriteTier.php \
tests/Features/Tia/CompleteRunWriteTier.php tests/Features/Tia/FilteredMode.php"
PAO_DISABLE=1 php84 bin/pest $F # 8.4.23, NO pcov — this is what CI has
PAO_DISABLE=1 php bin/pest $F # 8.5.8, pcov — this is what your machine has
```
**Both must be green.** `.github/workflows/tests.yml` sets `coverage: none`, so any assertion that
depends on a coverage driver fails in CI while passing locally. Two tests were already caught by
this. Concretely: a **cold recording run writes no graph at all** without pcov/xdebug (it prints
`Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug`), and a **PHP source
file edit** triggers `Detected PHP source changes but no coverage driver is available` → full suite,
`affected=0`. Seed a graph instead of recording one, and edit *test* files rather than `app/` files,
unless the point of the row is the driver itself.
### 1.4 Measure before you assert
Do not guess expectations from reading the code — every number in Part 3 came from a scratch probe.
The pattern (put it in your scratchpad, not in the repo):
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`.
---
## Part 2 — What the code looks like right now
Phase three landed default-branch resolution: `ChangedFiles::defaultBranch()`, the
`pest()->tia()->defaultBranch()` config surface, `Graph::setFallbackBranch()` + `?string
$fallbackBranch = null` on the seven read methods, and `Tia::resolveFallbackBranch()`
(`Tia.php:~1776`) resolving **config → CI env (`CiDefaultBranch`) → git (`origin/HEAD`, then
`init.defaultBranch` if the branch exists) → `soleRecordedBranch()`**, failing loudly when nothing
can name it.
On top of that, **two uncommitted changes** you will see in `git diff`:
1. `TIA_RESULTS_ONLY` global — a *partial* parallel run with an existing graph now purges stale
worker partials, sets the global, and workers flush their results through the existing
`flushWorkerReplay()` / `mergeWorkerReplayPartials()` path; the parent writes them with
`complete: false`. Gated on a graph already existing, so a TIA-less project still creates no
baseline dir. This gave `--parallel --filter` parity with sequential — **and, per B1, handed it
the shadowing bug too.**
2. `loadGraph()` emits `WARN The dependency graph could not be read — it will be rebuilt.` once per
parent process when `graph.json` exists but will not decode. Previously silent.
62 scenario tests cover this and pass on both interpreters.
---
## Part 3 — The defects
### B1 — a thin baseline key permanently shadows the default-branch fallback · **confirmed, priority 1**
**Symptom.** Any *narrowed* run on a new branch (`--filter`, `--group`, a path, `--bail`, `--shard`,
and now `--parallel --filter`) writes a baseline key holding only the tests that ran. From then on
`--tia` on that branch reads that thin key instead of falling back to the default branch, so
everything else is uncached — **one full suite per branch, forever**, which is the exact cost
issue [#1823](https://github.com/pestphp/pest/issues/1823) was about, re-entering through a side door.
**Measured** (fixture: 6 tests, graph seeded on `master`):
```
switch -c feature-x; pest --filter="adds two numbers"
→ keys=[master,feature-x] (feature-x holds 1 result)
pest --tia
→ 6 passed (6 assertions, 5 uncached, 1 replayed) ← want: 6 replayed, 0 uncached
same via: pest --parallel --processes=2 --filter="adds two numbers" → identical
```
**Where.** `src/Plugins/Tia/Graph.php::baselineFor()` (~line 814):
```php
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
```
The fallback is all-or-nothing: it fires only when the branch has **no** key at all. The key itself
is minted by `Graph::setResult()``ensureBaseline($branch)` (~599 / ~829), reached from
`Tia::snapshotTestResults()` on partial runs.
**Reproduction to add** (`tests/Features/Tia/DefaultBranchReplay.php`):
```php
test('a narrowed run on a new branch does not cost the fallback', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--filter=adds two numbers');
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
```
Add the `--parallel --processes=2 --filter=…` variant as a second row (dataset), since the two write
paths are different code.
**Fix direction.** Make the fallback **per entry** rather than per baseline: in `baselineFor()`,
when the branch has its own baseline *and* a distinct fallback baseline exists, return
`results` = the branch's results **layered over** the fallback's (branch wins per test id), and take
`sha`/`tree` from the fallback when the branch's are `null`/empty. `baselineFor()` is the single
funnel for `recordedAtSha()`, `lastRunTree()`, `getResult()`, `getTime()`, `getAssertions()`,
`testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so one change covers every reader. An
alternative — never mint a key from a partial run — is smaller but loses the executed result
entirely, which regresses the parity just gained.
**Done when.** Both reproduction rows are green on both interpreters, and none of these move:
- `the branch that ran gets its own key and the default branch keeps its baseline` — writes stay on
the real branch; the merge must be **read-only** and must not leak into `ensureBaseline()`/`setResult()`.
- `a declared default branch that does not exist degrades to a full run` — with a fallback that names
nothing, a branch's own thin results must still be all you get.
- `a detached HEAD replays without minting a branch key`, `writes nothing on a second run on the same
branch`, `filtered mode finds nothing to do…` (both rows) — a merged read must not make a clean
replay start writing.
- The whole `PartialRunWriteTier.php` / `CompleteRunWriteTier.php` set — tier semantics are unchanged
by this fix.
---
### B2 — a partial run on detached HEAD writes into the default branch's baseline · **needs a decision**
**Symptom.** With `HEAD` detached, `Tia::resolveBranch()` (`Tia.php:~1756`) sets
`$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch` — and that branch is used
for **writes**. A `--tia` run in this state happens to be harmless (a clean replay writes nothing),
but any run that *executes* tests writes their results into the default branch's baseline.
**Measured.**
```
seed on master; git checkout --detach; pest --filter="adds two numbers"
→ keys=[master] w=1 struct:ok ← master's baseline rewritten from a detached checkout
seed on master; git checkout --detach; pest --tia
→ keys=[master] w=0 ← read-only, as intended
```
**The decision.** `PLAN_PHASE_THREE.md` §2.3 **D3** recommended detached HEAD be *read-only*. If that
still stands, suppress writes when `currentBranch()` is `null` (a dedicated flag — note
`resultsOnlyWrites` is **not** enough, it still writes results). If Nuno prefers the current
behaviour, add a test pinning it and close this out.
**Reproduction** (`tests/Features/Tia/DefaultBranchWriteTier.php`), written for the read-only answer:
```php
test('a detached HEAD does not write into the default branch baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$project->pest('--filter=adds two numbers');
$delta = $project->delta();
expect($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($project->branchKeys())->toBe(['master']);
})->skipOnWindows();
```
---
### B3 — an unreadable graph is never repaired on a machine with no coverage driver · **needs a decision**
**Symptom.** A corrupt `graph.json` is now *reported* (Part 2, change 2) but only *rebuilt* when a
coverage driver is present, because rebuilding means recording. Driverless, the file stays corrupt
run after run and TIA is silently inert until someone deletes it by hand — while the WARN claims
`it will be rebuilt`.
**Measured** (`php84`, no pcov):
```
overwrite graph.json with '{not json'
run 1: exit=0, 6 passed, file still '{not json'
run 2: exit=0, 6 passed, file still '{not json'
headline: "Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug"
```
**Options.** (a) delete the file when it cannot be decoded, so the next drivered run starts clean and
the state dir does not carry a permanent landmine; (b) keep the file but reword the WARN when no
driver is available. (a) is the honest one and costs one `State::delete()`.
**Reproduction** (`tests/Features/Tia/FilteredMode.php`, extending the existing corrupt-graph row):
```php
expect($result->output)->toContain('The dependency graph could not be read')
->and(file_get_contents($project->graphDir().'/graph.json'))->not->toBe('{not json');
```
Must pass on **both** interpreters — that is the whole point of the row.
---
### B4 — a complete `--parallel` run writes nothing and prunes nothing · **needs a decision**
**Symptom.** With a graph present and TIA not flagged, a sequential run refreshes results and applies
the prune; the same run under `--parallel` does neither, because the parent's `ResultCollector` is
empty (results live in the workers) and workers only flush when they were told to record, replay, or
— since Part 2's change — results-only for a *partial* run. So parallel CI contributes nothing to the
cache, and a deleted test's entry survives forever.
**Measured** (graph seeded on `master`, sentinelled):
```
pest → w=6 +0 -0 struct:ok (sequential baseline)
pest --parallel --processes=2 → w=0 +0 -0 struct:ok ← writes nothing
delete a test, then:
pest --parallel --processes=2 → w=0 +0 -0 struct:ok ← and does not prune (sequential gives -1)
pest --tia --parallel --processes=2 → w=0 ← correct: everything replayed
```
**The decision.** Extending the `TIA_RESULTS_ONLY` mechanism to complete parallel runs is
mechanically easy, but a *complete* run also prunes, and pruning from merged worker partials is the
risky half: a worker that dies, or a shard that never ran, would look like "these tests no longer
exist". If it is done, the prune must key off "every worker reported" and fall back to
results-only when it cannot prove that. `PLAN.md` §5 lists this as a known gap, not a regression.
**Reproduction** (`tests/Features/Tia/CompleteRunWriteTier.php`) — mirror the two sequential rows
that already exist (`a complete run prunes a deleted test`, `--no-tia refreshes results…`) with
`--parallel --processes=2` added, and assert the same deltas.
---
### B5 — G4 ("parallel replay clobbers cached `time`") no longer reproduces · **verify, then correct the record**
**Symptom.** `PLAN_PHASE_THREE.md` §4.6 lists as still-present: *"parallel replay clobbers cached
`time` on all non-executed tests — `mergeWorkerReplayPartials()` takes `$result['time']` verbatim,
never routing through `resultTime()`"*. The repo fixture disagrees: after a parallel replay the
sentinelled `time=9.999` / `assertions=42` survive on every non-executed test.
**Measured.** `a parallel run merges worker results into the parent baseline` edits one test file,
then runs `--tia --parallel --processes=2`: `2 affected, 4 replayed, w=2`. If replayed times were
being clobbered, `w` would be `6`.
**Why the record may be stale.** `flushWorkerReplay()` (`Tia.php:~1286`) already applies
`resultTime()` **worker-side** before writing the partial, so the parent's verbatim read is reading
values that were already corrected.
**What to do.** Either find a shape where it still reproduces (the playground has 25 tests and real
timings; the fixture has 6 and may be too small), or confirm it is fixed and strike it from §4.6.
Add a direct row either way:
```php
test('a parallel replay keeps the recorded time of tests that did not run', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->pest('--tia', '--parallel', '--processes=2');
$delta = $project->delta();
expect($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
```
---
## Part 4 — Reporting
Per bug: **reproduced (yes/no)** with the measured delta, **fixed (yes/no/deferred)**, and the test
that now pins it. Close with:
1. Whether all `tests/Features/Tia/*` are green on **both** `php84` (no pcov) and `php85` (pcov).
2. Which of B2B5 still need Nuno's decision, phrased as a yes/no question each.
3. Whether `tests/.snapshots/success.txt` and the `tests/Visual/Parallel.php` tally now need
regenerating (they will, if you added rows) — **do not run `composer update:snapshots` unless
asked.**
4. Anything you found that is not in this file.
Leave the tree uncommitted and the scratch probes out of the repo.
+386
View File
@@ -0,0 +1,386 @@
# TIA deep audit — phase six
## Your task
Phase five went looking rather than fixing what was reported, and found nine defects in the
**read/write path and the state dir**. All nine are fixed and pinned by rows in the repo. It stopped
there deliberately: roughly half of TIA was never opened. Phase six is that other half.
Two deliverables, both required:
1. **A ranked findings list.** Every finding needs a *reproduction*, not a reading of the code. A
finding you cannot reproduce is a hypothesis — say so and rank it separately.
2. **New scenario tests** in `tests/Features/Tia/*`. Rows that pass go in the repo (they are the
regression net). Rows that fail stay in your scratchpad until the fix lands — **never commit a red
test.**
Work in passes, and **report between passes** rather than at the very end:
- **Pass A** — reproduce the leads in Part 3. Report which are real.
- **Pass B** — your own hunt: the invariants in Part 2, attacked with inputs nobody tried.
- **Pass C** — fixes, smallest first, each with the row that pins it.
Fix what is clearly a defect with an obvious correct answer. **Stop and ask** when the fix is a
behaviour *choice* — those are Nuno's calls, and a reproduction with a crisp yes/no question is worth
more than a guessed fix.
---
## Part 1 — Where things stand
### 1.1 What TIA is
Test Impact Analysis. `pest --tia` records a dependency graph (test file → source files it touched)
plus a per-branch baseline of results, then on later runs replays the tests whose dependencies did not
change. State lives in a per-project dir under `~/.pest/tia/`; `graph.json` is the whole thing.
Source of truth: `src/Plugins/Tia.php` (the plugin) and `src/Plugins/Tia/Graph.php` (the graph model +
read/write API). Supporting: `ChangedFiles.php` (git), `Fingerprint.php` (environment/structure
hashing), `Storage.php` / `FileState.php` (the state dir), `ResultCollector.php` (what a run observed),
`Recorder.php` + `CoverageCollector.php` (how edges are recorded).
Read `PLAN.md`, then `PLAN_PHASE_TWO.md``THREE``FOUR``FIVE` for the history and the tier
contract. Struck-through rows are fixed; do not re-report them. **Section 1.2 below supersedes any
phase-five row that contradicts it.**
### 1.2 What phase five settled — do not re-report these
Nine defects, all fixed, each with a row that fails if it comes back:
| # | Defect | Fix | Pinned by |
|---|---|---|---|
| 1 | `pruneStaleResults()` unset an entry; the next read layered the default branch's entry for the same test id back in, so a renamed/removed test stayed "previously unsuccessful" on a feature branch forever | `Graph::baselineFor()` layers per *file* once a branch has had a complete run (new `complete` flag on the baseline); a key minted by a narrowed run keeps the per-test-id merge | `StateReclamation`*a pruned result does not come back from the fallback*, *the fallback still reaches a branch that has never run a test file* |
| 2 | A cached failure whose test file was deleted was never reclaimed, so `--filtered` degraded to a full replay on every later run | `hasUnlocatedTestsToRerun()` widens only for a path it cannot *address*; `Graph::pruneResultsForMissingFiles()` + `pruneMissingTests()` run on every complete write | `StateReclamation`*a cached failure whose test file was deleted stops widening later runs*, *a complete run reclaims the entry and the edge of a deleted test file* |
| 3 | A detached HEAD is read-only for writes, but three paths still *deleted*: structural drift, `--fresh` (`Storage::purge`), and the corrupt-graph discard. The checkout that wiped the baseline could never rebuild it | `Tia::deleteState()` no-ops when detached; `Storage::purge` guarded | `StateReclamation` → the three *a detached HEAD does not purge…* rows |
| 4 | A status int outside 08 became `TestStatus::unknown()`, which `ReplayType` folded into `Failure` — a green test went red, exit 1 | `Graph::getResult()` returns `null` for an unknown status (re-run, don't replay); `shouldRerunStatus()` treats unknown as re-run | `HostileState`*a cached status this build cannot interpret is re-run, not replayed* |
| 5 | Notice/deprecation/warning (3/4/6) decode fine but `ReplayType` had no case, so they also folded into `Failure` | Explicit `Pass` cases — those statuses only reach replay when the configured `failOn*` / `displayDetailsOn*` policies say they do not matter | `HostileState`*a cached status with no replay of its own does not fail the run* |
| 6 | `Graph::decode()` took `baselines`, `edges` and `files` verbatim; one malformed entry raised a `TypeError` inside a test | `decodeBaselines()` / `decodeResults()` / `decodeEdges()` / `decodeFiles()` validate every field. Numeric-looking keys are cast, not filtered — a branch named `12345` decodes as an `int` | `HostileState`*a graph whose shape is wrong everywhere is repaired rather than trusted* |
| 7 | Baseline keys grew forever — one full copy of the suite per branch ever created | `ChangedFiles::branchNames()` (local + remote refs) + `Graph::pruneMissingBranches()`, on complete writes only, and only when the fallback branch is visible in the refs | `BranchShapes`*deleting many branches reclaims every one of their baselines* and the four rows around it |
| 8 | A test that triggered a deprecation was recorded as `status=0`, because PHPUnit emits `Passed` for it and TIA had no issue subscribers. A fresh `--fail-on-deprecation` run exited 1; the replayed one exited 0 | Six subscribers (`Notice`/`PhpNotice`/`Deprecation`/`PhpDeprecation`/`Warning`/`PhpWarning`) feed `ResultCollector`; most-important-status-wins; a plain `Passed` does not downgrade a triggered issue; `@`-suppressed issues are ignored | `IssueStatuses`*a triggered issue is recorded as itself, not as a pass*, *a cached deprecation still fails the run that asked to fail on one* |
| 9 | A replay wrote back the status it *looked* like from outside, so a cached deprecation replaying as a pass was persisted as `0` — defect 8's fix eroded after one run | `Tia::replayedAsRecorded()` writes back the cached status and message for replayed tests, in the sequential path and in the worker flush | `IssueStatuses`*replaying a cached issue does not downgrade it to a pass* |
| 10 | A run torn down mid-file (an `exit()` inside a test) still flushed what it had, and the parent read that as licence to prune the siblings it never reached. Sequential and parallel disagreed | `ResultCollector::hasUnfinishedTest()` demotes such a run to results-only, in `terminate()` (the shutdown path) and in `addOutput()` | `StateReclamation`*a run torn down mid-file does not prune the tests it never reached* |
**One existing assertion was changed.** `DefaultBranchWriteTier > filtered mode falls back to a full
replay when a cached failure cannot be located` used `tests/Unit/DeletedTest.php` — a path that
resolves but does not exist, which is precisely the shape behind defect 2. It now points at
`/build/agent/…`, so it still pins the widening safety net for the case where widening can help. If
you disagree with that reading, that is a finding, not a licence to change it back quietly.
### 1.3 Attacked in phase five and found solid — do not spend time here again
- **Parity.** Sequential vs `--processes=1/2/8` across replay-with-edit, `--filtered` with a cached
failure, a first run on a feature branch, `--bail`, `--stop-on-failure`, `--compact`: identical
tally *and* identical graph delta every time.
- **Hostile state dir.** Empty / truncated / not-JSON / JSON scalar / JSON list / `null` / `{}` / NUL
bytes; `graph.json` as a directory; a read-only state dir; dangling and negative edge ids; a result
`file` pointing outside the project; `schema: 2`. All degrade to "run the tests", exit 0.
- **Status int mapping.** `ResultCollector` (`asInt()`), `Graph::getResult()` and `TestStatus::from()`
agree exactly on 08. No off-by-one.
- **`FileState::write`** is tmp + rename, so concurrent runs cannot tear a file. Racing runs are
last-writer-wins **by design** — treat as a decision, not a bug, unless you can show data loss
worse than that.
- **Branch names.** Slashes, dots, unicode, digits-only, 180 characters, case-only differences,
remote-only branches, worktree branches: all keyed and reclaimed correctly.
- **`sha`/`tree` layering.** No shape found that under-reports changed files. The fallback `tree` only
drops a file whose current content hashes identically to what the fallback ran with, which is sound.
### 1.4 Known and deliberately unfixed
| Item | Why it is still open |
|---|---|
| Reclamation is skipped when a complete run executes **zero** tests | Stale edges linger until a run that executes at least one test. Writing the graph from a run that produced nothing costs more than it buys. Revisit if you find a real project stuck there. |
| SIGINT never reaches the suite (`PcovRestarter` re-exec) | Real, but signal handling outside the TIA read/write path. Needs its own pass. |
| Orphaned state dirs when the origin URL changes | `Storage::projectKey()` keys on the origin identity so clones share a graph. Changing the remote silently moves TIA to a fresh dir and nothing reclaims the old one. Unbounded growth in `~/.pest/tia`. |
| `environmental` fingerprint holds only `php_minor` | Nuno declined. The `clearResults()` drift path is therefore latent — it becomes live the moment that bucket grows. **Do not re-flag the `PHP_MAJOR_VERSION` line itself.** |
| PLAN.md §3 (raw coverage flags leave filtered mode on) | **No longer reproduces**`coverageReportActive()` consults `COVERAGE_REPORT_FLAGS` now. Struck. |
---
## Part 2 — The invariants to attack
Same list as phase five; 1, 2, 4, 5, 7 and 8 are now well covered, so weight your effort toward 3 and
6 and toward the *recording* half of the system, which nothing below the plugin has ever probed.
1. **Parity.** `pest <args>` and `pest --parallel --processes=N <args>` must leave the same graph and
reach the same tally. Hard rule. `Project::SEQUENTIAL_AND_PARALLEL` encodes it; use it on every new
write-path row.
2. **The tiers hold.** COMPLETE / RESULTS-ONLY / HARD-SUPPRESSED, exactly one each, no leaking.
3. **Replay is faithful.***weak spot.* A replayed test reports the same status, message, time and
assertion count as the recorded run. Phase five fixed statuses; **edges** are unproven.
4. **Reads never write.**
5. **A branch never corrupts another branch's baseline.**
6. **Nothing is unbounded.***weak spot.* Baseline keys are reclaimed now; `files`, `edges`, worker
partials, coverage caches and orphaned state dirs are not.
7. **A hostile state dir cannot break a run.**
8. **Git shapes are all handled.**
---
## Part 3 — Leads to reproduce first (Pass A)
Ranked by expected value. Everything here is **unexplored**, not merely unfixed — phase five never
opened these files.
### M1 — the recording path with a real coverage driver · **highest value**
Everything phase five did was driver-independent by design (seed a graph, never record one). Nothing
verified that a *recorded* graph is correct. This is the biggest blind spot in the audit.
- Do the recorded edges match what the test actually touched? Record with pcov, then hand-check
`edges` against the source files each fixture test uses.
- `PLAN.md` §4: **`pest --tia --coverage` narrows edges** — `Feature/ExampleTest` recorded with 2
files instead of 16, dropping self-edges. Same observable shape as the parallel bug G12 that phase
four closed, likely a different mechanism (the piggyback collector is scoped by `phpunit.xml
<include>`, the pcov-restarted recorder is not). **Confirm whether these are one fix or two.**
- `Recorder::activateLinkTracking()` (piggyback) vs `activate()` (pcov restart) must produce the same
edge set for the same suite. Compare them directly.
- `keepExisting: $this->piggybackCoverage` in `replaceEdges()` — what happens to a test whose edges
genuinely shrank while piggybacking?
Rows for this **must** be `->skipOnPhpVersionsWithoutCoverage()`-style guarded, or seeded, or CI goes
red on `php84`. That constraint is why phase five skipped it; solve it deliberately rather than by
accident. Adding a coverage-driver guard helper to the fixture is a legitimate deliverable.
### M2 — `BaselineSync` (621 lines, never opened)
The remote-baseline fetch is the only path where a graph arrives from **another machine**, which is
exactly where the hostile-state work matters most and where none of it has been exercised.
- A fetched baseline whose `fingerprint` matches but whose `files`/`edges` describe a different tree.
- A fetched baseline recorded on a branch this checkout does not have.
- `fetchIfAvailable()` under a broken network, a 404, a truncated download, a non-gzip body.
- `KEY_FETCH_COOLDOWN` — does it bound retries, and does a corrupt cooldown file break a run?
- Interaction with defect 7's branch GC: a fetched baseline carries branch keys this clone has never
heard of. **They will be pruned on the next complete write.** Is that right, or must fetched keys be
exempt? This is a real question, answer it.
### M3 — selection paths nobody has probed
`Graph::affected()` is ~600 lines and phase five only exercised the plain PHP-edge path.
- **Migrations** → `TableExtractor``testTables` intersection. What happens with an unparseable
migration, a migration that drops a table, a squashed schema dump?
- **Blade** — `bladeAncestorsFor()` walks `@include`/`@extends`/`<x-*>` transitively. Cycles?
Depth? A component referenced only dynamically?
- **Inertia** — `componentForInertiaPage()`, `jsFileToComponents`, `JsModuleGraph::buildStrict()`.
What if `vite` is missing, or the resolver returns garbage?
- **`usesSiblingHeuristicForUnknownPhp()`** — a hard-coded list of Laravel directories. A changed file
in `app/Providers/` widens to every test whose deps share that directory. Measure how much that
over-selects on a real tree.
- **Arch tests** — `testSourceDeclaresArchGroup()` greps the source with three regexes. False
positives (the string `arch(` in a comment) select the file on *every* PHP source change.
### M4 — git shapes phase five left alone
- A repo with **no commits yet** (`currentSha()` returns null / git fails).
- **Submodules** — a changed file inside one; `git status --porcelain` reports the submodule path.
- A repo whose root is **above** the pest project — `TiaRequiresRepositoryRoot` panics deliberately;
confirm it still panics and writes nothing.
- **Shallow / single-branch CI checkouts.** Defect 7's guard (`fallbackBranch` must be visible in the
refs) was reasoned about, not measured. Build one and check nothing is over-pruned.
- A branch **behind** the recorded sha, so `merge-base --is-ancestor` fails and the graph is declared
unreachable. Does it recover, or thrash?
### M5 — a real concurrent race
Phase five established `FileState::write` is atomic and called it last-writer-wins by design. Nobody
launched two runs. Launch them: a watcher plus a manual run, two `--tia` processes on one project,
`--parallel` where the parent writes while a straggler worker flushes. Look for **loss worse than
last-writer-wins** — a partially-merged baseline, a pruned entry from a run that never saw the file,
worker partials from run A consumed by run B (`KEY_WORKER_*` are not namespaced per run).
That last one is the sharpest: `purgeWorkerPartials()` deletes *all* partials by prefix, so two
concurrent parallel runs will eat each other's. Probe it.
### M6 — the unbounded remainder
Defect 7 reclaimed baseline keys. These still grow:
- `files` and `edges` — a deleted *source* file's entry is never removed (`pruneMissingTests()` only
covers test files). Every rename leaves an orphan id forever.
- Orphaned state dirs under `~/.pest/tia` (see 1.4).
- `KEY_COVERAGE_CACHE` / `KEY_COVERAGE_MARKER` — who deletes them, and when?
- Worker partials when a worker dies before `terminate()`.
Measure the growth on a realistic tree before proposing anything. Then **ask** — a cap or a GC is a
behaviour choice.
---
## Part 4 — The harness
### 4.1 The scenario suite
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the graph it wrote. Everything lives in `tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `seedFor($root, …)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `path($rel)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroy()`, `destroyAll()`, `SEQUENTIAL_AND_PARALLEL` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `commit($msg)`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()`, `run([...])` for anything else |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`); `Project::EDGES` and
`Project::TESTS` describe the graph `seed()` writes. `Project::testId($file, $description)` builds a
result key. Overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`
that is how you configure `pest()->tia()->…` for a scenario.
**If the fixture cannot express your case, extend the fixture.** A new `GitRepo` verb, a new overlay,
or a coverage-driver guard is a legitimate part of the deliverable. Do not water down a scenario to
fit the current helpers. Phase five added `GitRepo::commit()` usage, numeric-key handling in
`Project::branchKeys()` and `GraphDelta`, and used `$project->write()` to author test files inline —
follow that pattern.
### 4.2 The sentinel discriminator — read before writing any assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`, `assertions=42`**
(only where assertions are non-zero — risky/skipped/incomplete statuses are *derived* from "performed
no assertions", so falsifying those would rewrite the status on replay), then snapshots. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
- A replayed suite reports inflated assertion counts (6 tests × 42). That is the sentinel, not a bug.
The three write tiers, unchanged since phase two:
- **COMPLETE** — may change everything.
- **RESULTS-ONLY** — may change only `baselines[<branch>].results` for tests that ran; never removes
an entry, never adds a result for a test file absent from `edges`, never touches
`sha`/`tree`/`edges`/`files`/`fingerprint`.
- **HARD-SUPPRESSED** — may change nothing.
One addition from phase five: a complete run on a **non-default** branch also sets
`baselines[<branch>].complete = true`. It is deliberately *not* set on the default branch, so a clean
green run there stays byte-identical.
### 4.3 Running them
They are in the `integration` group (`tests/Pest.php`), so `composer test:unit` skips them. **A
directory argument finds nothing** — pass files, space-separated, as separate argv entries:
```bash
PAO_DISABLE=1 php84 bin/pest tests/Features/Tia/BranchShapes.php tests/Features/Tia/CompleteRunWriteTier.php \
tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/FilteredMode.php \
tests/Features/Tia/HostileState.php tests/Features/Tia/IssueStatuses.php \
tests/Features/Tia/PartialRunWriteTier.php tests/Features/Tia/StateReclamation.php
PAO_DISABLE=1 php bin/pest <same list>
```
**Both interpreters must be green.** `php84` is 8.4.x with **no pcov** — that is what CI has
(`.github/workflows/tests.yml` sets `coverage: none`). `php` is 8.5.x with pcov — that is the dev
machine. Any assertion that depends on a coverage driver passes locally and fails in CI. Concretely:
- A **cold recording run writes no graph at all** without pcov/xdebug (it prints `Running in TIA mode,
however TIA is skipped as it needs ext-pcov or Xdebug`). **Seed a graph, never record one**, unless
the driver *is* the point of the row — see M1, which has to solve this properly.
- A **PHP source file edit** driverless triggers `Detected PHP source changes but no coverage driver
is available` → full suite, `affected=0`. Edit *test* files, not `app/` files.
- The `terminate()` path differs by driver: with pcov the plugin reaches the complete write through
the shutdown handler, without it the run exits earlier. Defect 10 only reproduced on `php`. **Run
every probe on both before believing it.**
`PAO_DISABLE=1` is mandatory on every pest invocation and every probe: `laravel/pao` emits JSON under
agents and corrupts the captured output.
**Baseline: 161 scenario rows green on both interpreters**, at `b49ba062`. Per file:
| File | Rows |
|---|---|
| `StateReclamation.php` | 37 |
| `HostileState.php` | 25 |
| `CompleteRunWriteTier.php` | 17 |
| `DefaultBranchResolution.php` | 14 |
| `BranchShapes.php` | 14 |
| `DefaultBranchReplay.php` | 13 |
| `IssueStatuses.php` | 13 |
| `PartialRunWriteTier.php` | 12 |
| `DefaultBranchWriteTier.php` | 10 |
| `FilteredMode.php` | 6 |
Plus 80 unit/arch rows (`tests/Unit/Plugins/Tia/*`, `tests/Arch.php`). If those numbers do not
reproduce, stop and say so.
### 4.4 Measure before you assert
Never guess an expectation from reading the code. Probe first, in your scratchpad:
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`; `Project::destroyAll()`
at the end. Run every probe on **both** interpreters before believing it.
---
## Part 5 — Ground rules
- **Do not** run `composer test`. It takes minutes and you do not need it.
- **`tests/.snapshots/success.txt` and the tally in `tests/Visual/Parallel.php` are stale right now** —
phase five added 89 rows and did not regenerate them. **Report that they need regenerating and let
Nuno run `composer update:snapshots`.** Do not run it yourself.
- **Do not commit.** Leave the tree dirty.
- **Do not touch the playground's `vendor/`.** If a finding truly needs the playground (25 real tests,
real timings), say so and ask — syncing it is a manual step Nuno owns.
- Run `vendor/bin/phpstan analyse <the files you touched> --memory-limit=-1 --no-progress` and
`vendor/bin/pint <the files you touched>` before reporting. **Scope both to files you changed** —
Nuno edits `src/` live, and a repo-wide fixer will revert his work.
- The fixture projects **hardlink `src/`**. If you edit `src/` while a scenario run is in flight,
those rows go red for no reason. Finish the edit, then run.
- Keep scratch probes out of the repo.
- **Never weaken an existing assertion to make something pass.** If an existing row contradicts your
fix, that is a finding: report the contradiction and ask. Phase five hit this once (see 1.2) and
rewrote the row to pin the *narrower* contract rather than deleting it — that is the bar.
---
## Part 6 — Reporting
**Between passes**, not just at the end. Per finding:
- **Reproduced (yes / no / hypothesis only)**, with the exact command and the measured
`tally` + `delta()->summary()` on **both** interpreters.
- **Severity**, and say why in one line. The scale that matters here: *a test wrongly replayed as
passing* (worst) > *cache silently useless, full suite forever* > *graph grows / stale data* >
*cosmetic*.
- **Fixed / deferred / needs-a-decision**, and the row that pins it.
- For anything needing a decision: **one yes/no question**, no essay.
Close with:
1. The count of `tests/Features/Tia/*` green on **both** `php84` (no pcov) and `php` (pcov), against
the 161 baseline.
2. Every row you added, and what invariant from Part 2 it defends.
3. Which findings are still open, as yes/no questions.
4. That the snapshots need regenerating — **do not regenerate them.**
5. **What you looked at and found solid.** A list of attacks that did not break anything is a real
result: it tells the next phase where not to spend its time.
---
## Part 7 — Open questions carried into this phase
Answer these before or during Pass C; they change what the fixes should be.
1. A fetched remote baseline carries branch keys this clone has never heard of, and the branch GC will
prune them on the next complete write. **Should fetched keys be exempt?**
2. Two concurrent parallel runs share the `worker-edges-*` / `worker-results-*` prefixes and
`purgeWorkerPartials()` deletes by prefix. **Should partials be namespaced per run, or is "do not
run two TIA suites at once" the contract?**
3. `files` and `edges` never lose a deleted *source* file. **Cap, GC, or accept?**
4. Orphaned state dirs accumulate under `~/.pest/tia` whenever a project's origin URL changes.
**Reclaim them, or accept?**
+1 -1
View File
@@ -430,7 +430,7 @@ Most rows batch cheaply — phase two ran C1C20 in one call at roughly one li
| item | status |
|---|---|
| **G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests. `mergeWorkerReplayPartials()` takes `$result['time']` verbatim at `Tia.php:1451`, never routing through `resultTime()` as the sequential sites (1681, 1762) do. Assertions survive because workers replay those themselves. | Pre-existing (pre-fix had no preservation at all), **out of scope for phase three**. Report it as still-present; do not fix it unless Nuno asks. |
| ~~**G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests.~~ | **Struck in phase four — does not reproduce.** `flushWorkerReplay()` applies `resultTime()` worker-side before writing the partial, so the parent's verbatim read of `$result['time']` is reading values that were already corrected. Pinned by `a parallel replay keeps the recorded time of tests that did not run` (`tests/Features/Tia/CompleteRunWriteTier.php`), which sentinels every cached `time` and asserts a parallel replay writes nothing. |
| **C19**`--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
| **J11**`--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
| **J10**`--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
-2
View File
@@ -28,8 +28,6 @@ return RectorConfig::configure()
->withSkip([
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
__DIR__.'/tests/Fixtures/Arch',
// Fixture suites are pinned by the TeamCity / JUnit snapshots, down to
// the line numbers — rewriting their source would break them.
__DIR__.'/tests/Fixtures/Suites',
ReturnNeverTypeRector::class,
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
+6
View File
@@ -35,6 +35,12 @@ final readonly class BootSubscribers implements Bootstrapper
Subscribers\EnsureTiaResultIsRecordedOnSkipped::class,
Subscribers\EnsureTiaResultIsRecordedOnIncomplete::class,
Subscribers\EnsureTiaResultIsRecordedOnRisky::class,
Subscribers\EnsureTiaResultIsRecordedOnNoticeTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpNoticeTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnDeprecationTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpDeprecationTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnWarningTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpWarningTriggered::class,
Subscribers\EnsureTiaAssertionsAreRecordedOnFinished::class,
];
-3
View File
@@ -286,9 +286,6 @@ trait Testable
if ($replay !== ReplayType::None) {
assert($status !== null);
// Marks the replay before the branches below throw, so `tearDown`
// short-circuits for every replayed result — the throwing branches
// never reach `parent::setUp`, so no user hook may run after them.
$this->__replay = $replay;
match ($replay) {
@@ -205,15 +205,6 @@ final class WrapperRunner implements RunnerInterface
}
/**
* Widens pcov's instrumentation scope to the whole project for workers that
* record TIA edges.
*
* pcov's default scope is a single source directory it auto-detects, so
* `config/`, `routes/`, `bootstrap/` and every test's own file never reach
* the recorder — a worker-recorded graph selects a fraction of what a
* sequential one does. `pcov.directory` is only settable at startup, hence
* the command line rather than an `ini_set()` inside the worker.
*
* @param array<int, non-empty-string> $parameters
* @return array<int, non-empty-string>
*/
+229 -306
View File
@@ -63,11 +63,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
private const string BASELINE_PATH_OPTION = '--baseline';
/**
* Set by the mutation plugin on the subprocess running a single mutant,
* and nowhere else. Its own `--mutate` flag is popped before the argv is
* handed to that subprocess, so the flag cannot be matched instead.
*/
private const string ENV_MUTATION_TESTING = 'PEST_MUTATION_TESTING';
private const string ENV_TIA = 'PEST_TIA';
@@ -100,21 +95,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
private const string FILTERED_GLOBAL = 'TIA_FILTERED';
private const string WORKER_RESULTS_GLOBAL = 'TIA_WORKER_RESULTS';
private const string PIGGYBACK_COVERAGE_GLOBAL = 'TIA_PIGGYBACK_COVERAGE';
/**
* The parent's resolved fallback branch, handed to the workers.
*
* A worker cannot resolve it for itself: the restarters run before
* `tests/Pest.php` is loaded, so a `defaultBranch()` declared there is
* invisible to it — and autodetecting again would spend a git call per
* worker to reach the answer the parent already has.
*/
private const string FALLBACK_BRANCH_GLOBAL = 'TIA_FALLBACK_BRANCH';
/**
* The branch assumed when a repository cannot name its own default.
*/
private const string DEFAULT_BRANCH = 'main';
/**
@@ -138,19 +124,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
];
/**
* PHPUnit flags that make this run produce a coverage report.
*
* Pest's own `--coverage` is tracked by the Coverage plugin, but a raw
* PHPUnit report flag never reaches it. A run that reports coverage must
* not be narrowed to the affected tests — the report would then describe a
* subset of the suite — and must let PHPUnit own the coverage driver rather
* than have the TIA recorder clear it mid-collection.
*
* Flags that only shape collection or an existing report — `--coverage-filter`,
* `--path-coverage`, `--warm-coverage-cache`, `--only-summary-for-coverage-text`,
* `--show-uncovered-for-coverage-text`, `--disable-coverage-ignore` — produce no
* report on their own, so they are deliberately absent.
*
* @var list<string>
*/
private const array COVERAGE_REPORT_FLAGS = [
@@ -160,24 +133,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
];
/**
* Flags that narrow this run to a subset of the suite.
*
* Only user-supplied, per-run narrowing belongs here. A filter that is
* always in force — `<groups>` in phpunit.xml, or a plugin registering a
* test case filter from `boot()` — applies equally to the runs that build
* the baseline, so it does not make this run narrower than the baseline
* and must not disable baseline writes.
*
* `--shard` is rewritten to `--filter` before this plugin sees the
* arguments, so it is covered here too. The `bin/pest`-only flags are
* stripped from the handled arguments, so they are matched against the
* original argv instead.
*
* Flags that cut a run short instead of narrowing it — `--bail`, `--retry`,
* `--stop-on-*` — do not belong here either. They only narrow the run when
* something actually fails, and that is not known until it is over, so they
* are handled by stoppedEarly() from addOutput().
*
* @var list<string>
*/
private const array PARTIAL_SELECTION_FLAGS = [
@@ -187,15 +142,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
];
/**
* Options that cannot be combined with Tia mode.
*
* `--covers` and `--uses` select on coverage metadata Tia does not model,
* so they resolve to no tests at all rather than to the ones the user meant.
* `--random-order-seed` exits non-zero on its own, with or without Tia.
* Either way the run cannot honour both things it was asked for, so it says
* so instead of silently dropping Tia and running something else.
*/
private const array UNSUPPORTED_OPTIONS = [
'--covers', '--uses', '--random-order-seed',
];
@@ -214,46 +160,26 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
private array $cachedAssertionsByTestId = [];
/**
* Recorded durations of the tests this run replayed rather than executed.
*
* A replayed test never runs, so the duration PHPUnit reports for it is the
* cost of replaying it — near zero. Writing that back would decay every
* cached timing toward zero one run at a time.
* The status a replayed test was replayed *as*, so the write-back records
* what was cached rather than what the replay looked like from the
* outside. A cached deprecation replays as a pass — recording that pass
* would erase the deprecation from the baseline on the very next run.
*
* @var array<string, array{status: int, message: string}>
*/
private array $cachedStatusByTestId = [];
/**
* @var array<string, float>
*/
private array $cachedTimeByTestId = [];
private ?Graph $replayGraph = null;
/**
* The baseline this run reads from and writes to.
*
* The repository's default branch is only the fallback for a checkout whose
* branch cannot be read — a detached HEAD. It is also the branch every
* other baseline falls back to reading, so writing there by accident
* corrupts the shared baseline. Resolved through resolveBranch() rather
* than at every use site, because the git call it needs is not free.
*/
private string $branch = self::DEFAULT_BRANCH;
/**
* The baseline branches with none of their own read from.
*
* Read-only, and the whole point of the exercise: without it the first run
* on every new branch re-runs a suite whose results the default branch
* already holds.
*/
private string $fallbackBranch = self::DEFAULT_BRANCH;
/**
* Whether anything actually named the branch above.
*
* When nothing did, the value is a guess, and a guess is what the TIA path
* refuses to run on: an unresolved fallback reads no baseline at all, which
* looks exactly like a hit in the output. Runs that never asked for TIA
* still have to write somewhere, so the guess stands for them.
*/
private bool $fallbackBranchResolved = false;
private bool $branchResolved = false;
@@ -272,28 +198,18 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
private bool $baselineFetchAttemptedForDrift = false;
private bool $freshRebuild = false;
private bool $filteredMode = false;
/**
* Bars this run from touching the graph at all, results included.
*
* Reserved for runs whose results describe something other than the code
* in the working tree, which is nothing the baseline can ever use.
*/
private bool $writesSuppressed = false;
/**
* Narrows this run's writes to the results of the tests it actually ran.
*
* A run that covered only part of the suite still learns something true
* about the tests it did reach. What it cannot do is speak for the rest:
* pruning results, advancing the recorded sha and replacing the edge map
* all claim the whole suite reported, so they stay behind a complete run.
*/
private bool $resultsOnlyWrites = false;
private bool $flushesWorkerResults = false;
private bool $unreadableGraphReported = false;
private bool $detachedHead = false;
/** @var array<int, string> */
private array $originalArguments = [];
@@ -342,15 +258,70 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph = Graph::decode($json, $projectRoot);
// Every read of a baseline goes through a graph loaded here, so this is
// the one place the resolved fallback has to reach.
$graph?->setFallbackBranch($this->fallbackBranch);
if (! $graph instanceof Graph) {
$this->discardUnreadableGraph();
return null;
}
$graph->setFallbackBranch($this->fallbackBranch);
return $graph;
}
/**
* Drop a graph that will not decode, so the next run that can record starts
* clean instead of tripping over the same file forever — rebuilding needs a
* coverage driver, and without one the file would stay corrupt for good.
*/
private function discardUnreadableGraph(): void
{
if (Parallel::isWorker()) {
return;
}
if (! $this->deleteState(self::KEY_GRAPH)) {
return;
}
if ($this->unreadableGraphReported) {
return;
}
$this->unreadableGraphReported = true;
$this->output->writeln('');
$this->renderBadge('WARN', 'The dependency graph could not be read — it will be rebuilt.');
}
/**
* Delete a state file, unless this checkout may not write.
*
* A detached HEAD names no branch, so {@see self::saveGraph()} refuses to
* write — which means anything deleted here could never be rebuilt from
* this checkout. Read-only has to mean deletes too, or a drifted
* `composer.lock` on a detached CI checkout wipes the whole team's baseline.
*
* @return bool Whether the delete happened.
*/
private function deleteState(string $key): bool
{
if ($this->detachedHead) {
return false;
}
return $this->state->delete($key);
}
private function saveGraph(Graph $graph): bool
{
// A detached HEAD names no branch of its own, so `$this->branch` is the
// fallback — writing here would land this checkout's results in the
// default branch's baseline. Leave the graph exactly as it was.
if ($this->detachedHead) {
return true;
}
$json = $graph->encode();
if ($json === null) {
@@ -388,19 +359,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return ! self::argumentPresent('--ci', $arguments);
}
/**
* Whether the workers of this run record their own coverage edges.
*
* Stamped by the parent before paratest spawns anything, because a worker
* cannot tell on its own: its argv carries no `--tia`, and the restarters
* run before `tests/Pest.php` is loaded, so {@see self::isEnabledForRun()}
* sees an empty {@see WatchPatterns} too. Left unanswered, pcov keeps its
* default scope — a single auto-detected source directory — and every edge
* outside it, test self-edges included, is silently dropped.
*
* Piggyback runs are excluded: their edges come from PHPUnit's own coverage
* session, so widening pcov there costs time and buys nothing.
*/
public static function recordsEdgesInWorkers(): bool
{
return (string) Parallel::getGlobal(self::RECORDING_GLOBAL) === '1'
@@ -483,6 +441,10 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
}
$this->replayedCount++;
$this->cachedStatusByTestId[$testId] = [
'status' => $result->asInt(),
'message' => $result->message(),
];
$assertions = $this->replayGraph->getAssertions($this->branch, $testId);
$this->cachedAssertionsByTestId[$testId] = $assertions ?? 0;
@@ -541,11 +503,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
$disabled = $disabled || $partial;
// A mutation subprocess runs the suite against source the mutation
// plugin has deliberately broken. Its failures describe the mutant, not
// the working tree, so unlike every other narrowed run there is nothing
// in its results worth keeping. The parent `--mutate` run is untouched
// by this: it runs the whole suite against real source.
if (getenv(self::ENV_MUTATION_TESTING) !== false) {
$this->writesSuppressed = true;
}
@@ -565,19 +522,11 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$arguments = $this->popArgument(self::BASELINED_OPTION, $arguments);
if ($disabled) {
$this->requestWorkerResults();
if ($partial) {
// TIA cannot choose what runs here — the user already did — but
// the tests they picked still report honestly, so their results
// are kept and everything that would speak for the excluded ones
// is not. `--no-tia` needs none of this: it still runs the whole
// suite, so it remains a complete run.
$this->resultsOnlyWrites = true;
// `$this->filteredMode` counts as asking for it: reaching here
// means the narrowing came from the command line while filtered
// mode came from the environment or the config, and a run that
// silently declines what the config asked for is the one most
// in need of the explanation.
if ($cliEnabled || $freshRequested || $this->forceRefetch || $this->filteredMode) {
$this->output->writeln('');
$this->renderChild('TIA does not apply to partial runs — running the selected tests directly.');
@@ -586,15 +535,22 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$this->forceRefetch = false;
$this->filteredMode = false;
$this->freshRebuild = false;
return $arguments;
}
if ($isWorker && (string) Parallel::getGlobal(self::WORKER_RESULTS_GLOBAL) === '1') {
$this->flushesWorkerResults = true;
$this->resultsOnlyWrites = true;
return $arguments;
}
$forceRebuild = $freshRequested && ($enabled || $recordingGlobal || $replayingGlobal);
$this->freshRebuild = $forceRebuild;
if (! $enabled && ! $this->forceRefetch && ! $recordingGlobal && ! $replayingGlobal) {
$this->requestWorkerResults();
return $arguments;
}
@@ -617,16 +573,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return;
}
if (Parallel::isWorker() && ($this->replayGraph instanceof Graph || $this->recordingActive)) {
if (Parallel::isWorker() && ($this->replayGraph instanceof Graph || $this->recordingActive || $this->flushesWorkerResults)) {
$this->flushWorkerReplay();
}
// Both only ever set for the parent — addOutput() returns early in
// workers, whose partials are ephemeral and only reach the baseline if
// the parent consumes them. Everything this method goes on to write is
// whole-suite by nature — the edge map above all — so a narrowed run
// stops here too, its results already persisted by addOutput().
if ($this->writesSuppressed || $this->resultsOnlyWrites) {
// `terminate()` also runs from the shutdown handler, which is how a run
// that `exit()`s inside a test gets here — with a test prepared and
// never finished, and so with no right to a complete write.
if ($this->writesSuppressed || $this->resultsOnlyWrites || $this->hasUnfinishedTest()) {
$this->recorder->reset();
$this->coverageCollector->reset();
@@ -699,10 +653,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph->replaceTestInertiaComponents($perTestInertia);
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
if ($this->freshRebuild) {
$graph->pruneMissingTests();
}
$this->seedResultsInto($graph);
if (! $this->saveGraph($graph)) {
@@ -722,20 +672,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return $exitCode;
}
// `->only()` narrows the executed set exactly like `--filter` does, but
// is only knowable once the suite has been collected — too late to turn
// TIA off up front. Sampled in addOutput() because Only's lock file is
// already gone by the time terminate() runs (its plugin terminates
// first). Whether the run was cut short is likewise only knowable now.
if (Only::isEnabled() || $this->stoppedEarly()) {
if (Only::isEnabled() || $this->stoppedEarly() || $this->hasUnfinishedTest()) {
$this->resultsOnlyWrites = true;
}
$this->reportMissingWorkerDrivers();
// Runs before the checks below: it is what fills the parent's result
// collector in parallel, and a worker that stopped early narrows the
// whole run.
if (Parallel::isEnabled()) {
$this->mergeWorkerReplayPartials();
}
@@ -815,10 +757,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph->replaceTestInertiaComponents($finalisedInertia);
$graph->replaceJsFileToComponents(JsModuleGraph::build($projectRoot));
if ($this->freshRebuild) {
$graph->pruneMissingTests();
}
if (! $this->saveGraph($graph)) {
$this->renderBadge('ERROR', 'Could not write the dependency graph.');
@@ -861,8 +799,8 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return $this->reconcileFingerprint($rebuilt, $current);
}
$this->state->delete(self::KEY_GRAPH);
$this->state->delete(self::KEY_COVERAGE_CACHE);
$this->deleteState(self::KEY_GRAPH);
$this->deleteState(self::KEY_COVERAGE_CACHE);
return null;
}
@@ -878,7 +816,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph->clearResults($this->branch);
$graph->setFingerprint($current);
$this->saveGraph($graph);
$this->state->delete(self::KEY_COVERAGE_CACHE);
$this->deleteState(self::KEY_COVERAGE_CACHE);
}
return $graph;
@@ -900,12 +838,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$this->resolveBranch($projectRoot);
// After resolveBranch(), so a directory that is no repository at all
// still reports the missing git dependency rather than an unresolved
// default branch. Nothing named the branch every other baseline reads
// through, so every new branch would re-run the whole suite while the
// output called it a hit. A repository with no remote is the likeliest
// reason and gets said out loud.
if (! $this->fallbackBranchResolved) {
Panic::with(new ChangedFiles($projectRoot)->hasRemote()
? new TiaRequiresDefaultBranch
@@ -915,7 +847,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$fingerprint = Fingerprint::compute($projectRoot);
$this->startFingerprint = $fingerprint;
if ($forceRebuild) {
if ($forceRebuild && ! $this->detachedHead) {
Storage::purge($projectRoot);
}
@@ -948,11 +880,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
}
}
// Both of these belong to the coverage cache, which only Pest's own
// `--coverage` ever writes or merges. A raw PHPUnit report flag takes
// the piggyback path — it must not drive the driver itself — but must
// not leave a marker behind, nor force a recording run to prime a cache
// that nothing on its path will fill.
$coverageCacheOwned = $this->piggybackCoverage && $this->pestCoverageActive();
if ($coverageCacheOwned) {
@@ -1157,9 +1084,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
if (! Parallel::isEnabled()) {
if ($canRefreshReplayEdges) {
// Piggyback runs read PHPUnit's own coverage session. Driving
// the driver alongside it would clear the data PHPUnit is about
// to read, so only link tracking may run here.
if ($this->piggybackCoverage) {
$this->recorder->activateLinkTracking();
} else {
@@ -1329,9 +1253,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$recorder->activate();
$this->recordingActive = true;
// Why this run is rebuilding is worth saying whenever there is a reason
// for it — the parallel and piggyback branches above already do. Runs
// that are simply recording for the first time have nothing to explain.
if ($this->driftLabel !== null || $this->freshGraphReason !== null) {
$this->output->writeln('');
$this->renderFreshGraph();
@@ -1347,8 +1268,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
private function renderFreshGraph(): void
{
if ($this->driftLabel === null && $this->freshGraphReason !== null) {
// The reason is only ever set for a run that keeps its graph and
// records alongside it, so "fresh graph" would be a lie here.
$headline = sprintf('Experimental TIA mode enabled / %s.', $this->freshGraphReason);
} else {
$headline = 'Experimental TIA mode enabled / fresh graph';
@@ -1423,6 +1342,30 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$this->renderChild('Install / enable pcov or xdebug (mode: coverage) in the worker PHP and rerun.');
}
/**
* A parallel run keeps its results in the workers, so the parent's collector
* is empty and nothing would ever reach the graph. Ask the workers to flush
* what they ran, so a parallel run refreshes — and prunes — exactly like the
* sequential run of the same command.
*
* Gated on a graph already existing: a project that has never run TIA must
* not gain a baseline from a plain `--parallel` run.
*/
private function requestWorkerResults(): void
{
if (Parallel::isWorker() || ! Parallel::isEnabled() || $this->writesSuppressed) {
return;
}
if ($this->state->read(self::KEY_GRAPH) === null) {
return;
}
$this->purgeWorkerPartials();
Parallel::setGlobal(self::WORKER_RESULTS_GLOBAL, '1');
}
private function purgeWorkerPartials(): void
{
foreach ($this->collectWorkerEdgesPartials() as $key) {
@@ -1444,14 +1387,8 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return;
}
// A replayed result carries the duration PHPUnit measured for a test
// that never ran — near zero. Unlike the assertion count, which the
// replay injects into the result itself, the cached duration lives only
// in this process: the parent replayed nothing of its own, so once the
// partial is written the real value is unrecoverable. Launder it here
// and the parent's verbatim read is correct by construction.
foreach ($results as $testId => $result) {
$results[$testId]['time'] = $this->resultTime($testId, $result['time']);
$results[$testId] = $this->replayedAsRecorded($testId, $result);
}
$json = json_encode([
@@ -1459,9 +1396,7 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
'replayed' => $this->replayedCount,
'affected' => $this->affectedCount,
'executed' => $this->executedCount,
// Only the worker knows it stopped early — the parent runs no tests
// of its own, so its own check would always come back clean.
'truncated' => $this->stoppedEarly(),
'truncated' => $this->stoppedEarly() || $collector->hasUnfinishedTest(),
], JSON_UNESCAPED_SLASHES);
if ($json === false) {
@@ -1498,9 +1433,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
continue;
}
// One worker stopping early leaves the whole suite incomplete: the
// tests it never reached are missing from the merged result set just
// as if they had been filtered out.
if (($decoded['truncated'] ?? false) === true) {
$this->resultsOnlyWrites = true;
}
@@ -1728,15 +1660,29 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return $coverage;
}
/**
* The duration to record for a test: its own, unless it was replayed rather
* than executed, in which case the duration it was recorded with stands.
*/
private function resultTime(string $testId, float $time): float
{
return $this->cachedTimeByTestId[$testId] ?? $time;
}
/**
* @param array{status: int, message: string, time: float, assertions: int, file?: string} $result
* @return array{status: int, message: string, time: float, assertions: int, file?: string}
*/
private function replayedAsRecorded(string $testId, array $result): array
{
$result['time'] = $this->resultTime($testId, $result['time']);
$cached = $this->cachedStatusByTestId[$testId] ?? null;
if ($cached !== null) {
$result['status'] = $cached['status'];
$result['message'] = $cached['message'];
}
return $result;
}
private function seedResultsInto(Graph $graph): void
{
/** @var ResultCollector $collector */
@@ -1756,12 +1702,14 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$touchedFiles[$file] = true;
}
$result = $this->replayedAsRecorded($testId, $result);
$graph->setResult(
$this->branch,
$testId,
$result['status'],
$result['message'],
$this->resultTime($testId, $result['time']),
$result['time'],
$result['assertions'],
$file,
);
@@ -1769,17 +1717,45 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph->markKnownTestFiles(array_keys($touchedFiles));
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
$this->reclaim($graph);
$collector->reset();
}
/**
* Folds the run's results into the existing graph.
*
* An incomplete run passes `$complete: false`, which keeps the additive
* half — the results of the tests it did run — and drops the half that
* speaks for the suite as a whole.
* Give back what the graph no longer needs. Only ever called from a
* complete write — the RESULTS-ONLY and HARD-SUPPRESSED tiers may not
* remove an entry, and a narrowed run has not seen enough to judge.
*/
private function reclaim(Graph $graph): void
{
// The fallback branch never layers under itself, so marking it would
// write the graph for no reader's benefit — and cost a clean green run
// its "wrote nothing at all".
if ($this->branch !== $this->fallbackBranch) {
$graph->markBaselineComplete($this->branch);
}
$graph->pruneMissingTests();
$graph->pruneResultsForMissingFiles($this->branch);
$branches = new ChangedFiles(TestSuite::getInstance()->rootPath)->branchNames();
if ($branches === null) {
return;
}
// A shallow, single-branch CI checkout can see almost no refs, and
// "git has never heard of it" would then mean "this clone is narrow",
// not "that branch is gone". Only reclaim from a checkout that can at
// least see the branch everything else falls back to.
if (! in_array($this->fallbackBranch, $branches, true)) {
return;
}
$graph->pruneMissingBranches([...$branches, $this->branch, $this->fallbackBranch]);
}
private function snapshotTestResults(bool $markKnownTestFiles = false, bool $complete = true): void
{
/** @var ResultCollector $collector */
@@ -1802,23 +1778,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
try {
$this->resolveBranch($projectRoot);
} catch (MissingDependency) {
// This run never asked for TIA, so a missing git must not turn it
// into a failure the way it does on the TIA path. Writing to the
// fallback baseline is the lesser of the two evils.
}
// The graph above was loaded before the branch was known — this path
// only writes, but a graph carrying an unresolved fallback is the exact
// bug this whole change is about.
$graph->setFallbackBranch($this->fallbackBranch);
$touchedFiles = [];
// Whether this run is the one that records the edges its results will be
// invalidated through. A recording run's edges are written after this
// (terminate() runs last), and a parallel one's arrive with the worker
// partials that ask for $markKnownTestFiles — either way the graph on
// disk cannot be asked yet, so the run is taken at its word.
$recordsEdges = $complete && ($markKnownTestFiles || $this->recordingActive);
foreach ($results as $testId => $result) {
@@ -1832,22 +1797,18 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$touchedFiles[$file] = true;
}
// A result is only ever invalidated through the edges of the test
// that produced it, so one recorded for a test the graph has no
// edges for could never be invalidated again — it would be replayed
// as settled however far the code around it moved. A run that
// records no edges leaves such a test exactly as unknown as it
// found it, whether or not it ran the whole suite.
if (! $recordsEdges && (! is_string($file) || ! $graph->knowsTest($file))) {
continue;
}
$result = $this->replayedAsRecorded($testId, $result);
$graph->setResult(
$this->branch,
$testId,
$result['status'],
$result['message'],
$this->resultTime($testId, $result['time']),
$result['time'],
$result['assertions'],
$file,
);
@@ -1857,10 +1818,9 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$graph->markKnownTestFiles(array_keys($touchedFiles));
}
// Pruning reads the absence of a test from this run as the test being
// gone. That only holds if every test was invited to report.
if ($complete) {
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
$this->reclaim($graph);
}
$this->saveGraph($graph);
@@ -1898,15 +1858,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return null;
}
/**
* Whether this run produces a coverage report, however it was asked for.
*
* The original argv, not the handled arguments: Pest's own Coverage plugin
* appends `--coverage-php <path>` to those and runs before this one, and a
* paratest worker's arguments always carry it too. `bin/worker.php` never
* hands over the original argv, so a worker sees `[]` here and keeps taking
* this from {@see self::PIGGYBACK_COVERAGE_GLOBAL} instead.
*/
private function coverageReportActive(): bool
{
if ($this->pestCoverageActive()) {
@@ -1916,10 +1867,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return array_any(self::COVERAGE_REPORT_FLAGS, fn (string $flag): bool => $this->hasArgument($flag, $this->originalArguments));
}
/**
* Whether Pest's own `--coverage` was given — the only entry point that
* writes the coverage cache these two flags read and clean up.
*/
private function pestCoverageActive(): bool
{
$coverage = Container::getInstance()->get(Coverage::class);
@@ -1929,11 +1876,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
}
/**
* Panics when the run asks for Tia alongside an option Tia cannot honour.
*
* Checked against the original argv as well, because `bin/pest` consumes
* some of these itself before PHPUnit ever sees them.
*
* @param array<int, string> $arguments
*/
private function guardUnsupportedOptions(array $arguments): void
@@ -1952,11 +1894,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
}
/**
* Whether a selection-narrowing flag was given, either among the arguments
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
* the original argv. Explicit path arguments and `->only()` are detected
* separately.
*
* @param array<int, string> $arguments
*/
private function hasPartialSelection(array $arguments): bool
@@ -1974,36 +1911,23 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
return false;
}
/**
* Whether the run stopped before reaching every test it had queued.
*
* Covers `--bail`, `--retry` and every `--stop-on-*` flag, the equivalent
* `phpunit.xml` attributes, and an interrupted run — none of which narrow
* the selection up front, so hasPartialSelection() cannot see them. The
* tests queued behind the defect that halted the run never reported, and
* folding what did report into the baseline prunes the cached results of
* their siblings in every file the run had already entered.
*
* Deliberately unguarded. Both callers run only once PHPUnit's
* configuration is registered — the kernel reads it unguarded itself just
* before dispatching addOutput(), and flushWorkerReplay() bails out unless
* the worker actually executed something. Swallowing a failure here would
* report every truncated run as complete, which is the corruption this
* guards against in the first place.
*/
private function stoppedEarly(): bool
{
return TestResultFacade::shouldStop();
}
/**
* Resolves the baselines this run reads from and writes to, once.
*
* Results are written on runs where TIA itself took no part, and those
* never reach handleParent(). Without this the default would stand and
* every such run would write its results to `main`, whatever branch it
* actually ran on.
* A test that was prepared and never finished means this process is being
* torn down mid-file, so it has not seen enough of that file to prune it.
*/
private function hasUnfinishedTest(): bool
{
$collector = Container::getInstance()->get(ResultCollector::class);
assert($collector instanceof ResultCollector);
return $collector->hasUnfinishedTest();
}
private function resolveBranch(string $projectRoot): void
{
if ($this->branchResolved) {
@@ -2014,9 +1938,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
$changedFiles = new ChangedFiles($projectRoot);
// Resolved before the current branch, which throws where git is
// missing: the fallback is advisory, so a run that cannot name its
// branch at all should still carry the best answer available.
$resolved = $this->resolveFallbackBranch($changedFiles);
$this->fallbackBranchResolved = $resolved !== null;
@@ -2024,24 +1945,12 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
Parallel::setGlobal(self::FALLBACK_BRANCH_GLOBAL, $this->fallbackBranch);
// A detached HEAD has no branch of its own to write to. The default
// branch is the honest key there — it is the commit the checkout most
// likely sits on, and it keeps a phantom baseline from being minted
// under a branch name the repository never had.
$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch;
$currentBranch = $changedFiles->currentBranch();
$this->detachedHead = $currentBranch === null;
$this->branch = $currentBranch ?? $this->fallbackBranch;
}
/**
* The branch every other baseline falls back to reading, or null when
* nothing in the checkout can name it.
*
* Ordered by how much the source actually knows. Configuration first: it is
* the escape hatch for a repository whose git-side answers disagree with its
* branches. Then the CI provider, which states the answer outright where git
* is at its least informed. Then git itself. Then the recorded graph, whose
* single baseline can only have come from the branch this repository
* integrates on.
*/
private function resolveFallbackBranch(ChangedFiles $changedFiles): ?string
{
$inherited = Parallel::getGlobal(self::FALLBACK_BRANCH_GLOBAL);
@@ -2056,14 +1965,6 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
?? $this->soleRecordedBranch();
}
/**
* The one branch a recorded graph holds a baseline for.
*
* Last in the chain and deliberately narrow: with a single baseline on disk
* there is only one branch whose results can be read at all, so naming it is
* strictly better than resolving to a branch that holds nothing. Two or more
* baselines carry no such implication and are left alone.
*/
private function soleRecordedBranch(): ?string
{
$json = $this->state->read(self::KEY_GRAPH);
@@ -2098,11 +1999,15 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
if (str_starts_with($arg, '-')) {
continue;
}
if ($index > 0) {
$previous = $arguments[$index - 1] ?? '';
if (in_array($previous, self::VALUE_TAKING_FLAGS, true)) {
continue;
}
if ($index === 0) {
continue;
}
$previous = $arguments[$index - 1] ?? '';
if (in_array($previous, self::VALUE_TAKING_FLAGS, true)) {
continue;
}
$candidate = $this->resolveArgumentPath($arg, $projectRoot);
@@ -2111,16 +2016,34 @@ final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArgument
continue;
}
foreach ($testPaths as $testPath) {
if ($candidate === $testPath || str_starts_with($candidate, $testPath.DIRECTORY_SEPARATOR)) {
return true;
}
if ($this->narrowsSuite($candidate, $testPaths)) {
return true;
}
}
return false;
}
/**
* @param array<int, string> $testPaths
*/
private function narrowsSuite(string $candidate, array $testPaths): bool
{
foreach ($testPaths as $testPath) {
if ($candidate === $testPath || str_starts_with($candidate, $testPath.DIRECTORY_SEPARATOR)) {
return true;
}
}
foreach ($testPaths as $testPath) {
if (str_starts_with($testPath, $candidate.DIRECTORY_SEPARATOR)) {
return false;
}
}
return true;
}
private function resolveArgumentPath(string $arg, string $projectRoot): ?string
{
$candidates = [$arg, rtrim($projectRoot, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR.ltrim($arg, DIRECTORY_SEPARATOR)];
+47 -17
View File
@@ -219,16 +219,6 @@ final readonly class ChangedFiles
return $branch === '' || $branch === 'HEAD' ? null : $branch;
}
/**
* The repository's default branch — the one every other branch's baseline
* falls back to reading.
*
* Advisory, unlike {@see self::currentBranch()}: a repository that cannot
* answer the question is not a broken repository. A remote-less checkout
* has no `origin/HEAD`, and plenty of CI checkouts never run
* `git remote set-head`, so every step here fails soft and the caller is
* left to pick its own default.
*/
public function defaultBranch(): ?string
{
$head = $this->gitOutput(['git', 'symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
@@ -241,10 +231,6 @@ final readonly class ChangedFiles
}
}
// `init.defaultBranch` is a setting of the machine, not of the
// repository — it names what `git init` would have called the first
// branch here, which is worth nothing once the repository disagrees.
// Taken only when a branch by that name actually exists.
$configured = $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
if ($configured === null) {
@@ -258,11 +244,55 @@ final readonly class ChangedFiles
}
/**
* Whether the repository has any remote configured.
* Every branch name this checkout knows, local and remote alike. Remotes
* count: a branch that only lives on the origin is still a branch someone
* will check out, and its baseline must survive.
*
* Advisory like {@see self::defaultBranch()} — a `git` that cannot answer
* is reported as "no remote", and the caller decides what that means.
* @return list<string>|null `null` when git cannot answer.
*/
public function branchNames(): ?array
{
$process = new Process(
['git', 'for-each-ref', '--format=%(refname)', 'refs/heads', 'refs/remotes'],
$this->projectRoot,
);
$process->setTimeout(5.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
$names = [];
foreach ($this->splitLines($process->getOutput()) as $ref) {
if (str_starts_with($ref, 'refs/heads/')) {
$names[substr($ref, strlen('refs/heads/'))] = true;
continue;
}
if (! str_starts_with($ref, 'refs/remotes/')) {
continue;
}
$tail = substr($ref, strlen('refs/remotes/'));
$slash = strpos($tail, '/');
if ($slash === false) {
continue;
}
$branch = substr($tail, $slash + 1);
if ($branch !== '' && $branch !== 'HEAD') {
$names[$branch] = true;
}
}
return array_keys($names);
}
public function hasRemote(): bool
{
return $this->gitOutput(['git', 'remote']) !== null;
-18
View File
@@ -5,23 +5,10 @@ declare(strict_types=1);
namespace Pest\Plugins\Tia;
/**
* The default branch as the CI provider itself reports it.
*
* Worth asking before git: a CI checkout is the one place where git knows the
* least. `actions/checkout` builds the working copy with `git init` plus a
* single-ref `fetch` rather than a `clone`, so `origin/HEAD` is never set and
* `init.defaultBranch` — a setting of the runner image, not of the repository —
* is all git has left to offer. The provider, meanwhile, states the answer
* outright in the environment it handed us.
*
* @internal
*/
final class CiDefaultBranch
{
/**
* Advisory, like every other source in the chain: anything unreadable,
* unparsable, or simply absent means "no answer", never a failure.
*/
public static function detect(): ?string
{
return self::fromGitLab() ?? self::fromGitHubEvent();
@@ -32,11 +19,6 @@ final class CiDefaultBranch
return self::environment('CI_DEFAULT_BRANCH');
}
/**
* GitHub publishes no default-branch variable, but every repository-scoped
* event payload carries `repository.default_branch`, and the path to that
* payload is in the environment.
*/
private static function fromGitHubEvent(): ?string
{
$path = self::environment('GITHUB_EVENT_PATH');
-6
View File
@@ -61,12 +61,6 @@ final class Configuration
}
/**
* The branch whose baseline every other branch falls back to reading.
*
* Autodetected from the repository when left unset; declare it here when
* the repository cannot answer for itself — no `origin/HEAD`, or an
* `init.defaultBranch` that disagrees with reality.
*
* @return $this
*/
public function defaultBranch(string $branch): self
+7 -1
View File
@@ -29,7 +29,13 @@ enum ReplayType
$status->isRisky() => self::Risky,
$status->isSkipped() => self::Skipped,
$status->isIncomplete() => self::Incomplete,
default => self::Failure,
// A recorded notice, deprecation or warning only reaches replay when
// the configured failOn* / displayDetailsOn* policies say it is not
// worth re-running — which means the test passed. Folding it into
// Failure below would turn a green run red on cache alone.
$status->isNotice(), $status->isDeprecation(), $status->isWarning() => self::Pass,
$status->isFailure(), $status->isError() => self::Failure,
default => self::None,
};
}
}
+305 -48
View File
@@ -43,20 +43,12 @@ final class Graph
* @var array<string, array{
* sha: ?string,
* tree: array<string, string>,
* complete?: bool,
* results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
* }>
*/
private array $baselines = [];
/**
* The baseline a branch with none of its own reads from.
*
* Only ever read from: a branch writes to its own key, so a fallback that
* leaked into the write path would corrupt the baseline every other branch
* depends on. Resolved once per run by the plugin — see
* {@see self::setFallbackBranch()} — because the git calls it takes are not
* free and the read path runs per test.
*/
private string $fallbackBranch = 'main';
private readonly string $projectRoot;
@@ -659,6 +651,9 @@ final class Graph
$r = $baseline['results'][$testId];
// A status this build does not know — a graph written by a newer Pest,
// or a corrupt one — is not a result. Returning null re-executes the
// test rather than replaying an outcome nobody can interpret.
return match ($r['status']) {
0 => TestStatus::success(),
1 => TestStatus::skipped($r['message']),
@@ -669,7 +664,7 @@ final class Graph
6 => TestStatus::warning($r['message']),
7 => TestStatus::failure($r['message']),
8 => TestStatus::error($r['message']),
default => TestStatus::unknown(),
default => null,
};
}
@@ -696,7 +691,9 @@ final class Graph
$rel = $this->relative($file);
if ($rel !== null) {
// A test file that is no longer on disk cannot be re-run by anyone,
// so selecting it would only widen the run for nothing.
if ($rel !== null && is_file($this->projectRoot.'/'.$rel)) {
$files[$rel] = true;
}
}
@@ -705,13 +702,13 @@ final class Graph
}
/**
* Whether any cached result due a re-run points at a test file that is not
* on disk — deleted, or never locatable in the first place (`eval()`'d code,
* a path outside the project).
* Whether a cached result due a re-run names a test file this project
* cannot address — an empty path, or one that resolves outside the project
* root. Those are genuinely lost, so the caller widens to the full suite.
*
* A filtered run cannot honour such an entry: it would select a file that
* collects no tests, so the run reports green without ever re-running the
* failure — and does so again on every subsequent invocation.
* A path that resolves fine but is simply absent is *deleted*, not lost:
* widening would not run it either, and treating it as unlocated used to
* strand `--filtered` on a full replay for good.
*/
public function hasUnlocatedTestsToRerun(string $branch, ?string $fallbackBranch = null): bool
{
@@ -728,12 +725,7 @@ final class Graph
return true;
}
$rel = $this->relative($file);
// Results are stored relative, so `relative()` answers "is this
// inside the project" without ever touching the filesystem. The
// stat is what tells a deleted test file apart from a live one.
if ($rel === null || ! is_file($this->projectRoot.'/'.$rel)) {
if ($this->relative($file) === null) {
return true;
}
}
@@ -757,6 +749,10 @@ final class Graph
return true;
}
if ($testStatus->isUnknown()) {
return true;
}
$configuration = Registry::get();
if ($testStatus->isRisky()) {
@@ -830,21 +826,82 @@ final class Graph
}
/**
* @return array{sha: ?string, tree: array<string, string>, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
* The baseline a read sees for this branch: its own entries layered over the
* default branch's, so a key minted by a narrowed run — which only holds the
* handful of tests that ran — does not shadow the fallback for everything else.
*
* Once this branch has had a complete run, the layering becomes per *file*
* rather than per test id: the branch's entries for a file it executed are
* the whole truth, so the fallback's entries for that same file are dropped
* rather than merged. Without that, a test the branch renamed or removed —
* and {@see self::pruneStaleResults()} therefore unset — is resurrected by
* the default branch on the very next read, and never stops coming back.
*
* A branch whose key was minted by a *narrowed* run holds only the handful
* of tests that ran, and has no business speaking for the rest of their
* file, so it keeps the per-test-id merge.
*
* Read-only: the layering never reaches `$this->baselines`, so writes stay on
* the branch that ran.
*
* @return array{sha: ?string, tree: array<string, string>, complete?: bool, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}
*/
private function baselineFor(string $branch, ?string $fallbackBranch): array
{
$fallbackBranch ??= $this->fallbackBranch;
if (isset($this->baselines[$branch])) {
return $this->baselines[$branch];
$fallback = $branch !== $fallbackBranch ? ($this->baselines[$fallbackBranch] ?? null) : null;
$own = $this->baselines[$branch] ?? null;
if ($own === null) {
return $fallback ?? ['sha' => null, 'tree' => [], 'results' => []];
}
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) {
return $this->baselines[$fallbackBranch];
if ($fallback === null) {
return $own;
}
return ['sha' => null, 'tree' => [], 'results' => []];
$under = ($own['complete'] ?? false) === true
? $this->withoutFilesCoveredBy($fallback['results'], $own['results'])
: $fallback['results'];
return [
'sha' => $own['sha'] ?? $fallback['sha'],
'tree' => $own['tree'] !== [] ? $own['tree'] : $fallback['tree'],
'results' => array_replace($under, $own['results']),
];
}
/**
* @param array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}> $results
* @param array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}> $authoritative
* @return array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
*/
private function withoutFilesCoveredBy(array $results, array $authoritative): array
{
$covered = [];
foreach ($authoritative as $entry) {
$file = $entry['file'] ?? null;
if (is_string($file) && $file !== '') {
$covered[$file] = true;
}
}
if ($covered === []) {
return $results;
}
foreach ($results as $testId => $entry) {
$file = $entry['file'] ?? null;
if (is_string($file) && isset($covered[$file])) {
unset($results[$testId]);
}
}
return $results;
}
private function ensureBaseline(string $branch): void
@@ -856,13 +913,7 @@ final class Graph
/**
* @param array<string, array<int, string>> $testToFiles
* @param bool $keepExisting Leave already-recorded edge sets alone. For runs
* whose edges are piggybacked off a PHPUnit coverage
* session: that data is scoped by `<source>`, so it
* can only ever be narrower than what the TIA
* recorder sees — it never contains the test's own
* file, for one — and a narrower edge set silently
* stops selecting the tests it used to select.
* @param bool $keepExisting Leave already-recorded edge sets alone.
*/
public function replaceEdges(array $testToFiles, bool $keepExisting = false): void
{
@@ -873,8 +924,6 @@ final class Graph
continue;
}
// An empty set means "known, covers nothing", so piggyback data is
// still an improvement there — only a populated set is protected.
if ($keepExisting && ($this->edges[$testRel] ?? []) !== []) {
continue;
}
@@ -1437,6 +1486,67 @@ final class Graph
}
}
/**
* Record that this branch has run the whole suite at least once, which is
* what lets {@see self::baselineFor()} treat its entries as authoritative
* for the files they cover. Never mints a key: a run that recorded nothing
* has nothing to be authoritative about.
*/
public function markBaselineComplete(string $branch): void
{
if (isset($this->baselines[$branch])) {
$this->baselines[$branch]['complete'] = true;
}
}
/**
* Drop this branch's result entries whose test file is no longer on disk.
*
* Without this nothing but `--fresh` ever reclaims them, and a *failing*
* one keeps `--filtered` widened to a full replay on every later run.
*/
public function pruneResultsForMissingFiles(string $branch): void
{
if (! isset($this->baselines[$branch]['results'])) {
return;
}
$root = rtrim($this->projectRoot, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR;
foreach ($this->baselines[$branch]['results'] as $testId => $result) {
$file = $result['file'] ?? null;
if (! is_string($file) || $file === '') {
continue;
}
$rel = $this->relative($file);
if ($rel === null || is_file($root.$rel)) {
continue;
}
unset($this->baselines[$branch]['results'][$testId]);
}
}
/**
* Drop baselines for branches git no longer knows, so the graph does not
* carry one full copy of the suite per branch ever created.
*
* @param array<int, string> $keep Branch names that must survive.
*/
public function pruneMissingBranches(array $keep): void
{
$survivors = array_fill_keys($keep, true);
foreach (array_keys($this->baselines) as $branch) {
if (! isset($survivors[$branch])) {
unset($this->baselines[$branch]);
}
}
}
/**
* Prune baseline result entries whose test files were just executed but whose
* test IDs are no longer present (e.g. the test method was removed or renamed).
@@ -1483,13 +1593,6 @@ final class Graph
}
/**
* The branches a recorded graph holds baselines for, read straight from the
* encoded form.
*
* Answerable before the graph is hydrated because the default branch has to
* be resolved first: the fallback is what every hydrated graph reads its
* baselines through.
*
* @return list<string>
*/
public static function branchesIn(string $json): array
@@ -1521,10 +1624,10 @@ final class Graph
$graph = new self($projectRoot);
$graph->fingerprint = is_array($data['fingerprint'] ?? null) ? $data['fingerprint'] : [];
$graph->files = is_array($data['files'] ?? null) ? array_values($data['files']) : [];
$graph->files = self::decodeFiles($data['files'] ?? null);
$graph->fileIds = array_flip($graph->files);
$graph->edges = is_array($data['edges'] ?? null) ? $data['edges'] : [];
$graph->baselines = is_array($data['baselines'] ?? null) ? $data['baselines'] : [];
$graph->edges = self::decodeEdges($data['edges'] ?? null);
$graph->baselines = self::decodeBaselines($data['baselines'] ?? null);
$graph->testTables = self::decodeStringMap($data['test_tables'] ?? null);
$graph->testInertiaComponents = self::decodeStringMap($data['test_inertia_components'] ?? null);
@@ -1533,6 +1636,160 @@ final class Graph
return $graph;
}
/**
* @return array<int, string>
*/
private static function decodeFiles(mixed $section): array
{
if (! is_array($section)) {
return [];
}
$files = [];
foreach ($section as $path) {
if (is_string($path) && $path !== '') {
$files[] = $path;
}
}
return $files;
}
/**
* @return array<string, array<int, int>>
*/
private static function decodeEdges(mixed $section): array
{
if (! is_array($section)) {
return [];
}
$edges = [];
foreach ($section as $key => $ids) {
$testFile = (string) $key;
if ($testFile === '') {
continue;
}
if (! is_array($ids)) {
continue;
}
$clean = [];
foreach ($ids as $id) {
if (is_int($id)) {
$clean[] = $id;
}
}
$edges[$testFile] = $clean;
}
return $edges;
}
/**
* A graph is state on disk that any process may have written: a newer Pest,
* a half-finished write, a hand edit. Every branch, every entry and every
* field is checked here so that a malformed one is dropped rather than
* reaching a read path and taking the run down with it.
*
* @return array<string, array{sha: ?string, tree: array<string, string>, complete?: bool, results: array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>}>
*/
private static function decodeBaselines(mixed $section): array
{
if (! is_array($section)) {
return [];
}
$baselines = [];
foreach ($section as $key => $baseline) {
// A branch named `12345` decodes as an integer key, and must not be
// mistaken for a malformed one.
$branch = (string) $key;
if ($branch === '') {
continue;
}
if (! is_array($baseline)) {
continue;
}
$sha = $baseline['sha'] ?? null;
$tree = [];
if (is_array($baseline['tree'] ?? null)) {
foreach ($baseline['tree'] as $path => $hash) {
if (is_string($path) && is_string($hash)) {
$tree[$path] = $hash;
}
}
}
$baselines[$branch] = [
'sha' => is_string($sha) ? $sha : null,
'tree' => $tree,
'results' => self::decodeResults($baseline['results'] ?? null),
];
if (($baseline['complete'] ?? null) === true) {
$baselines[$branch]['complete'] = true;
}
}
return $baselines;
}
/**
* @return array<string, array{status: int, message: string, time: float, assertions?: int, file?: string}>
*/
private static function decodeResults(mixed $section): array
{
if (! is_array($section)) {
return [];
}
$results = [];
foreach ($section as $key => $entry) {
$testId = (string) $key;
if ($testId === '') {
continue;
}
if (! is_array($entry)) {
continue;
}
if (! is_int($entry['status'] ?? null)) {
continue;
}
$time = $entry['time'] ?? null;
$result = [
'status' => $entry['status'],
'message' => is_string($entry['message'] ?? null) ? $entry['message'] : '',
'time' => is_int($time) || is_float($time) ? (float) $time : 0.0,
];
if (is_int($entry['assertions'] ?? null)) {
$result['assertions'] = $entry['assertions'];
}
if (is_string($entry['file'] ?? null) && $entry['file'] !== '') {
$result['file'] = $entry['file'];
}
$results[$testId] = $result;
}
return $results;
}
/**
* @return array<string, list<string>>
*/
+78
View File
@@ -16,6 +16,9 @@ final class ResultCollector
*/
private array $results = [];
/** @var array<string, true> */
private array $triggered = [];
private ?string $currentTestId = null;
private ?string $currentTestFile = null;
@@ -35,9 +38,35 @@ final class ResultCollector
return;
}
// PHPUnit reports a test that triggered a notice, deprecation or
// warning as passed, and emits Passed for it. Recording success here
// would erase the issue from the baseline, and a later replay under
// --fail-on-deprecation (and friends) would come back green where a
// fresh run fails. Keep the issue; only refresh what it cannot know.
if (isset($this->triggered[$this->currentTestId])) {
$this->refreshTime();
return;
}
$this->record(TestStatus::success());
}
public function testTriggeredNotice(string $message): void
{
$this->recordIssue(TestStatus::notice($message));
}
public function testTriggeredDeprecation(string $message): void
{
$this->recordIssue(TestStatus::deprecation($message));
}
public function testTriggeredWarning(string $message): void
{
$this->recordIssue(TestStatus::warning($message));
}
public function testFailed(string $message): void
{
if ($this->currentTestId === null) {
@@ -91,6 +120,17 @@ final class ResultCollector
return $this->results;
}
/**
* Whether a test was prepared but never finished — the process is being
* torn down in the middle of it (an `exit()` inside a test, a killed
* worker). What it collected is therefore a partial view of that test
* file, and must not license pruning the siblings it never reached.
*/
public function hasUnfinishedTest(): bool
{
return $this->currentTestId !== null;
}
public function recordAssertions(string $testId, int $assertions): void
{
if (isset($this->results[$testId])) {
@@ -111,6 +151,7 @@ final class ResultCollector
public function reset(): void
{
$this->results = [];
$this->triggered = [];
$this->currentTestId = null;
$this->currentTestFile = null;
$this->startTime = null;
@@ -123,6 +164,43 @@ final class ResultCollector
$this->startTime = null;
}
/**
* Record an issue raised while the test was running. The most important
* one wins, exactly as PHPUnit ranks them, so a deprecation does not
* shadow the warning that followed it — or the failure.
*/
private function recordIssue(TestStatus $status): void
{
if ($this->currentTestId === null) {
return;
}
$existing = $this->results[$this->currentTestId]['status'] ?? null;
if (is_int($existing) && $existing >= $status->asInt()) {
return;
}
$this->triggered[$this->currentTestId] = true;
$this->record($status);
}
private function refreshTime(): void
{
if ($this->currentTestId === null) {
return;
}
if (! isset($this->results[$this->currentTestId])) {
return;
}
if ($this->startTime === null) {
return;
}
$this->results[$this->currentTestId]['time'] = round(microtime(true) - $this->startTime, 3);
}
private function record(TestStatus $status): void
{
if ($this->currentTestId === null) {
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\DeprecationTriggered;
use PHPUnit\Event\Test\DeprecationTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnDeprecationTriggered implements DeprecationTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(DeprecationTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredDeprecation($event->message());
}
}
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\NoticeTriggered;
use PHPUnit\Event\Test\NoticeTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnNoticeTriggered implements NoticeTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(NoticeTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredNotice($event->message());
}
}
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\PhpDeprecationTriggered;
use PHPUnit\Event\Test\PhpDeprecationTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnPhpDeprecationTriggered implements PhpDeprecationTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(PhpDeprecationTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredDeprecation($event->message());
}
}
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\PhpNoticeTriggered;
use PHPUnit\Event\Test\PhpNoticeTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnPhpNoticeTriggered implements PhpNoticeTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(PhpNoticeTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredNotice($event->message());
}
}
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\PhpWarningTriggered;
use PHPUnit\Event\Test\PhpWarningTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnPhpWarningTriggered implements PhpWarningTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(PhpWarningTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredWarning($event->message());
}
}
@@ -0,0 +1,26 @@
<?php
declare(strict_types=1);
namespace Pest\Subscribers;
use Pest\Plugins\Tia\ResultCollector;
use PHPUnit\Event\Test\WarningTriggered;
use PHPUnit\Event\Test\WarningTriggeredSubscriber;
/**
* @internal
*/
final readonly class EnsureTiaResultIsRecordedOnWarningTriggered implements WarningTriggeredSubscriber
{
public function __construct(private ResultCollector $collector) {}
public function notify(WarningTriggered $event): void
{
if ($event->wasSuppressed()) {
return;
}
$this->collector->testTriggeredWarning($event->message());
}
}
+154 -1
View File
@@ -1568,6 +1568,41 @@
PASS Tests\Features\Tia
✓ it does not run user hooks when replaying cached skipped and incomplete results
PASS Tests\Features\Tia\BranchShapes
✓ a branch name git allows is a branch key TIA can hold with dataset "slashes"
✓ a branch name git allows is a branch key TIA can hold with dataset "dots"
✓ a branch name git allows is a branch key TIA can hold with dataset "unicode"
✓ a branch name git allows is a branch key TIA can hold with dataset "digits"
✓ a branch name git allows is a branch key TIA can hold with dataset "underscores"
✓ a branch name git allows is a branch key TIA can hold with dataset "very long"
✓ a branch differing from the default only in case gets its own key
✓ a branch that only lives on the remote keeps its baseline
✓ a branch checked out in a worktree keeps its baseline
✓ deleting many branches reclaims every one of their baselines with dataset "sequential"
✓ deleting many branches reclaims every one of their baselines with dataset "parallel"
✓ a narrowed run does not reclaim anything
✓ a detached HEAD does not reclaim anything either
✓ the default branch baseline survives every branch that comes and goes
PASS Tests\Features\Tia\CompleteRunWriteTier
✓ a complete run prunes a deleted test with dataset "sequential"
✓ a complete run prunes a deleted test with dataset "parallel"
✓ a complete run records nothing for a test file the graph does not know
✓ a partial run records nothing for a test file the graph does not know
✓ a truncated run does not prune with dataset "sequential"
✓ a truncated run does not prune with dataset "parallel"
✓ a green bail run is complete
✓ --no-tia refreshes results without enabling tia with dataset "sequential"
✓ --no-tia refreshes results without enabling tia with dataset "parallel"
✓ a plain run refreshes the results it executed with dataset "sequential"
✓ a plain run refreshes the results it executed with dataset "parallel"
✓ a parallel replay keeps the recorded time of tests that did not run
✓ a run that never enables tia creates no graph with dataset "plain"
✓ a run that never enables tia creates no graph with dataset "filtered"
✓ a run that never enables tia creates no graph with dataset "parallel filtered"
✓ a test edit narrows to the affected file and replays the rest
✓ a parallel run merges worker results into the parent baseline
PASS Tests\Features\Tia\DefaultBranchReplay
✓ replays the default branch baseline on a new branch
✓ replays whatever the default branch is called with ('main')
@@ -1577,12 +1612,23 @@
✓ replays on a second new branch too
✓ writes nothing on a second run on the same branch
✓ replays on a branch whose name contains slashes
✓ replays again once back on the default branch
✓ replays on a new branch when tia is enabled by configuration
✓ a narrowed run on a new branch does not cost the fallback with dataset "sequential"
✓ a narrowed run on a new branch does not cost the fallback with dataset "parallel"
✓ replays inside a worktree on a new branch
PASS Tests\Features\Tia\DefaultBranchResolution
✓ a declared default branch beats autodetection
✓ a declared default branch that does not exist degrades to a full run
✓ a renamed default branch replays and writes under its new name
✓ the CI provider names the default branch where the checkout cannot
✓ GitLab names the default branch through its own variable
✓ a lone recorded baseline names the default branch
✓ a default branch nothing can name is refused rather than guessed
✓ an init.defaultBranch naming a branch that exists is still trusted
✓ a repository with no remote is refused rather than silently re-run
✓ a remote-less repository holding one baseline is not refused
✓ a declared default branch stands in for a missing remote
✓ tia still requires git
✓ a plain run outside a repository creates no baseline
@@ -1592,10 +1638,117 @@
✓ narrows to the affected tests on a new branch
✓ filtered mode reads the fallback too
✓ filtered mode finds nothing to do on a clean green feature branch
✓ filtered mode falls back to a full replay when a cached failure cannot be located
✓ filtered mode finds nothing to do on the default branch itself
✓ a detached HEAD replays without minting a branch key
✓ a detached HEAD does not write into the default branch baseline with dataset "sequential"
✓ a detached HEAD does not write into the default branch baseline with dataset "parallel"
✓ the branch that ran gets its own key and the default branch keeps its baseline
✓ the fallback reaches parallel workers
PASS Tests\Features\Tia\FilteredMode
✓ re-runs a cached failure on a clean tree
✓ an explicit path turns filtered mode off
✓ filtered mode runs the whole suite when there is no baseline
✓ filtered mode finds nothing to do in parallel either
✓ a corrupt graph is reported and does not crash the run
✓ --parallel --retry is refused and leaves the graph alone
PASS Tests\Features\Tia\HostileState
✓ a graph mangled beyond use still lets the suite run with dataset "empty"
✓ a graph mangled beyond use still lets the suite run with dataset "truncated"
✓ a graph mangled beyond use still lets the suite run with dataset "not json"
✓ a graph mangled beyond use still lets the suite run with dataset "json scalar"
✓ a graph mangled beyond use still lets the suite run with dataset "json list"
✓ a graph mangled beyond use still lets the suite run with dataset "json null"
✓ a graph mangled beyond use still lets the suite run with dataset "empty object"
✓ a graph mangled beyond use still lets the suite run with dataset "nul bytes"
✓ a graph whose shape is wrong everywhere is repaired rather than trusted with dataset "sequential"
✓ a graph whose shape is wrong everywhere is repaired rather than trusted with dataset "parallel"
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "below the range"
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "one past the range"
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "far past the range"
✓ a cached status this build cannot interpret is re-run, not replayed with dataset "huge"
✓ a cached status with no replay of its own does not fail the run with dataset "notice"
✓ a cached status with no replay of its own does not fail the run with dataset "deprecation"
✓ a cached status with no replay of its own does not fail the run with dataset "warning"
✓ a cached skip or todo replays with its message intact with dataset "skipped"
✓ a cached skip or todo replays with its message intact with dataset "incomplete"
✓ a cached failure with a multi-line message re-runs rather than replaying the text
✓ a result pointing outside the project is not addressable and widens the run
✓ an edge pointing at a file id that does not exist is ignored
✓ a graph from a schema this build does not know is rebuilt, not read
✓ graph.json being a directory does not stop the run
✓ a state dir it cannot write to still replays
PASS Tests\Features\Tia\IssueStatuses
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "deprecation"
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "notice"
✓ a triggered issue is recorded as itself, not as a pass with dataset "sequential" / dataset "warning"
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "deprecation"
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "notice"
✓ a triggered issue is recorded as itself, not as a pass with dataset "parallel" / dataset "warning"
✓ a cached deprecation still fails the run that asked to fail on one with dataset "sequential"
✓ a cached deprecation still fails the run that asked to fail on one with dataset "parallel"
✓ replaying a cached issue does not downgrade it to a pass with dataset "sequential"
✓ replaying a cached issue does not downgrade it to a pass with dataset "parallel"
✓ a failure outranks an issue triggered on the way to it
✓ a skip outranks an issue triggered on the way to it
✓ a suppressed issue is not recorded
PASS Tests\Features\Tia\PartialRunWriteTier
✓ a filtered run rewrites only the test that ran
✓ a filtered run under --tia announces that tia does not apply
✓ a test suffix narrows the tier even though every test runs
✓ a dirty run narrows to the uncommitted test edit
✓ filtered mode yields to an explicit filter
✓ an env flag narrows exactly like the option it mirrors with ('PEST_TIA')
✓ an env flag narrows exactly like the option it mirrors with ('PEST_TIA_FILTERED')
✓ a partial run does not purge the graph even with --fresh
✓ --no-tia does not stop a partial run from refreshing its own entry
✓ two partial runs each keep the other entry
✓ a shard is a partial run
✓ a parallel partial run records the test that ran, like a sequential one
PASS Tests\Features\Tia\StateReclamation
✓ a detached HEAD does not purge the graph on structural drift with dataset "sequential"
✓ a detached HEAD does not purge the graph on structural drift with dataset "parallel"
✓ a detached HEAD does not purge the graph with --fresh either with dataset "sequential"
✓ a detached HEAD does not purge the graph with --fresh either with dataset "parallel"
✓ a detached HEAD leaves an unreadable graph for a checkout that can rebuild it
✓ a cached failure whose test file was deleted stops widening later runs with dataset "sequential"
✓ a cached failure whose test file was deleted stops widening later runs with dataset "parallel"
✓ a complete run reclaims the entry and the edge of a deleted test file with dataset "sequential"
✓ a complete run reclaims the entry and the edge of a deleted test file with dataset "parallel"
✓ a pruned result does not come back from the fallback with dataset "sequential"
✓ a pruned result does not come back from the fallback with dataset "parallel"
✓ the fallback still reaches a branch that has never run a test file
✓ a branch that git no longer knows loses its baseline
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "unknown"
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "future"
✓ an unknown cached status is re-run rather than replayed as a failure with dataset "garbage"
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "notice"
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "deprecation"
✓ a cached notice, deprecation or warning does not replay as a failure with dataset "warning"
✓ a malformed baseline entry cannot break the run with dataset "sequential"
✓ a malformed baseline entry cannot break the run with dataset "parallel"
✓ a run torn down mid-file does not prune the tests it never reached with dataset "sequential"
✓ a run torn down mid-file does not prune the tests it never reached with dataset "parallel"
✓ a fatal error mid-file is a test error, not a truncation with dataset "sequential"
✓ a fatal error mid-file is a test error, not a truncation with dataset "parallel"
✓ a green complete run leaves the graph exactly as it found it with dataset "bail"
✓ a green complete run leaves the graph exactly as it found it with dataset "stop-on-failure"
✓ a green complete run leaves the graph exactly as it found it with dataset "compact"
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel bail"
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel one process"
✓ a green complete run leaves the graph exactly as it found it with dataset "parallel more processes than files"
✓ --tia --no-tia is a plain run that still refreshes what it executed with dataset "sequential"
✓ --tia --no-tia is a plain run that still refreshes what it executed with dataset "parallel"
✓ --fresh on a partial run neither purges nor prunes with dataset "sequential"
✓ --fresh on a partial run neither purges nor prunes with dataset "parallel"
✓ a second green run on a feature branch writes nothing at all with dataset "sequential"
✓ a second green run on a feature branch writes nothing at all with dataset "parallel"
PASS Tests\Features\Ticket
✓ it may be associated with an ticket #1, #2
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
@@ -2225,4 +2378,4 @@
✓ pass with dataset with ('my-datas-set-value')
✓ within describe → pass with dataset with ('my-datas-set-value')
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1571 passed (3446 assertions)
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1710 passed (3845 assertions)
+1 -1
View File
@@ -49,7 +49,7 @@ it('does not run user hooks when replaying cached skipped and incomplete results
expect($storage->write(Tia::KEY_GRAPH, (string) $json))->toBeTrue();
$process = new Process(
['php', 'bin/pest', $fixture, '--tia'],
['php', 'bin/pest', '--configuration', 'tests/Fixtures/Suites/TiaReplayHooks.xml', '--tia'],
$projectRoot,
[
'COLLISION_PRINTER' => 'DefaultPrinter',
+161
View File
@@ -0,0 +1,161 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
/*
* Invariant 5 — writes land on the branch that ran and only there — and
* invariant 6 — nothing is unbounded — under every branch shape git allows.
*/
test('a branch name git allows is a branch key TIA can hold', function (string $branch): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo($branch, new: true);
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', $branch])
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary());
})->with([
'slashes' => 'feature/deep/nesting',
'dots' => 'release.1.2.x',
'unicode' => 'feature-café-日本',
'digits' => '12345',
'underscores' => 'feature_x_y',
'very long' => 'feature-'.str_repeat('x', 180),
])->skipOnWindows();
test('a branch differing from the default only in case gets its own key', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('MASTER-2', new: true);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($project->branchKeys())->toBe(['master', 'MASTER-2']);
})->skipOnWindows();
test('a branch that only lives on the remote keeps its baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('remote-only', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->run(['update-ref', 'refs/remotes/origin/remote-only', 'HEAD']);
$project->git()->run(['branch', '-D', 'remote-only']);
$project->pest('--tia');
expect($project->branchKeys())->toBe(['master', 'remote-only']);
})->skipOnWindows();
test('a branch checked out in a worktree keeps its baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('wt-branch', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->run(['worktree', 'add', '--quiet', $project->path().'-wt', 'wt-branch']);
$project->pest('--tia');
$project->git()->run(['worktree', 'remove', '--force', $project->path().'-wt']);
expect($project->branchKeys())->toBe(['master', 'wt-branch']);
})->skipOnWindows();
test('deleting many branches reclaims every one of their baselines', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
foreach (range(1, 4) as $index) {
$project->git()->switchTo('feature-'.$index, new: true);
$project->pest('--tia', ...$arguments);
}
expect($project->branchKeys())->toHaveCount(5);
$project->git()->switchTo('master');
foreach (range(1, 4) as $index) {
$project->git()->run(['branch', '-D', 'feature-'.$index]);
}
$project->pest('--tia', ...$arguments);
expect($project->branchKeys())->toBe(['master']);
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a narrowed run does not reclaim anything', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->run(['branch', '-D', 'feature-x']);
$project->snapshot();
$project->pest('--tia', '--filter=adds two numbers');
$delta = $project->delta();
expect($project->branchKeys())->toBe(['master', 'feature-x'])
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a detached HEAD does not reclaim anything either', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->run(['branch', '-D', 'feature-x']);
$project->git()->detach();
$project->snapshot();
$project->pest('--tia');
$delta = $project->delta();
expect($project->branchKeys())->toBe(['master', 'feature-x'])
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
test('the default branch baseline survives every branch that comes and goes', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->git()->run(['branch', '-D', 'feature-x']);
$project->snapshot();
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->added())->toBe(0, $delta->summary())
->and($delta->removed())->toBe(0, $delta->summary())
->and($project->graph()['baselines']['master']['results'])->toHaveCount(Project::TOTAL_TESTS);
})->skipOnWindows();
+234
View File
@@ -0,0 +1,234 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
test('a complete run prunes a deleted test', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
});
PHP);
$result = $project->pest(...$arguments);
$delta = $project->delta();
expect($result->tally())->toContain('5 passed')
->and($delta->removed())->toBe(1, $delta->summary())
->and($delta->added())->toBe(0, $delta->summary())
->and($delta->structureMoved())->toBeFalse($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a complete run records nothing for a test file the graph does not know', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/BrandNewTest.php', <<<'PHP'
<?php
declare(strict_types=1);
test('brand new thing', function (): void {
expect(true)->toBeTrue();
});
PHP);
$result = $project->pest();
$delta = $project->delta();
expect($result->tally())->toContain('7 passed')
->and($delta->added())->toBe(0, $delta->summary())
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary());
})->skipOnWindows();
test('a partial run records nothing for a test file the graph does not know', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/BrandNewTest.php', <<<'PHP'
<?php
declare(strict_types=1);
test('brand new thing', function (): void {
expect(true)->toBeTrue();
});
PHP);
$result = $project->pest('--filter=brand new thing');
$delta = $project->delta();
expect($result->tally())->toContain('1 passed')
->and($delta->added())->toBe(0, $delta->summary())
->and($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a truncated run does not prune', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Goodbye, Nuno!');
});
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
PHP);
$result = $project->pest('--bail', ...$arguments);
$delta = $project->delta();
expect($result->exitCode)->toBe(1, $result->describe())
->and($result->tally())->toContain('1 failed')
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->structureMoved())->toBeFalse($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a green bail run is complete', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--bail');
$delta = $project->delta();
expect($result->exitCode)->toBe(0, $result->describe())
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('--no-tia refreshes results without enabling tia', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--no-tia', ...$arguments);
$delta = $project->delta();
expect($result->output)->not->toContain('Experimental TIA mode enabled')
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a plain run refreshes the results it executed', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest(...$arguments);
$delta = $project->delta();
expect($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a parallel replay keeps the recorded time of tests that did not run', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->pest('--tia', '--parallel', '--processes=2');
$delta = $project->delta();
expect($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a run that never enables tia creates no graph', function (array $arguments): void {
$project = Project::make('master');
$result = $project->pest(...$arguments);
expect($result->exitCode)->toBe(0, $result->describe())
->and($project->graphExists())->toBeFalse();
})->with([
'plain' => [[]],
'filtered' => [['--filter=adds two numbers']],
'parallel filtered' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
])->skipOnWindows();
test('a test edit narrows to the affected file and replays the rest', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
expect((new Greeter)->greet('Nuno'))->toBeString();
});
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
PHP);
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->affected())->toBe(2, $result->describe())
->and($result->replayed())->toBe(4, $result->describe())
->and($delta->writtenCount())->toBe(2, $delta->summary())
->and($delta->edgesMoved())->toBeFalse($delta->summary())
->and($delta->removed())->toBe(0, $delta->summary());
})->skipOnWindows();
test('a parallel run merges worker results into the parent baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
expect((new Greeter)->greet('Nuno'))->toBeString();
});
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
PHP);
$result = $project->pest('--tia', '--parallel', '--processes=2');
$delta = $project->delta();
expect($result->affected())->toBe(2, $result->describe())
->and($result->replayed())->toBe(4, $result->describe())
->and($delta->writtenCount())->toBe(2, $delta->summary())
->and($delta->edgesMoved())->toBeFalse($delta->summary())
->and($delta->removed())->toBe(0, $delta->summary());
})->skipOnWindows();
+48 -17
View File
@@ -4,15 +4,6 @@ declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
/**
* Reading a baseline recorded on another branch.
*
* Without the default-branch fallback, the first `--tia` run on every new branch
* re-runs a whole suite whose results the default branch already holds — once
* per branch, forever, on any repository not named `main`.
*
* @see https://github.com/pestphp/pest/issues/1823
*/
afterEach(function (): void {
Project::destroyAll();
});
@@ -54,8 +45,6 @@ test('replays on a second new branch too', function (): void {
$result = $project->pest('--tia');
// The toll is one full run per new branch. It must not come back for the
// second branch either.
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
@@ -89,16 +78,58 @@ test('replays on a branch whose name contains slashes', function (): void {
->and($project->branchKeys())->toContain('feature/x/y');
})->skipOnWindows();
test('replays again once back on the default branch', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
$project->git()->switchTo('master');
$project->snapshot();
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('replays on a new branch when tia is enabled by configuration', function (): void {
$project = Project::make('master', overlay: 'always-enabled');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', 'feature-x']);
})->skipOnWindows();
test('a narrowed run on a new branch does not cost the fallback', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest(...$arguments);
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->with([
'sequential' => [['--filter=adds two numbers']],
'parallel' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
])->skipOnWindows();
test('replays inside a worktree on a new branch', function (): void {
$project = Project::make('master');
$worktree = $project->worktree('feature-worktree');
// Seeded against the worktree rather than the main checkout: a worktree's
// `.git` is a file, so `Storage::originIdentity()` cannot read the remote
// from it and the worktree resolves a storage key of its own. That gap is
// separate from the branch fallback, and it is the fallback this row is
// about — the worktree is checked out on a branch the baseline does not
// name, which is the scenario from the issue.
$project->seedFor($worktree, 'master');
$result = $project->pestIn($worktree, '--tia');
+19 -33
View File
@@ -5,17 +5,11 @@ declare(strict_types=1);
use Symfony\Component\Process\ExecutableFinder;
use Tests\Fixtures\Tia\Project;
/**
* How the default branch gets named: declared in `tests/Pest.php`, autodetected
* from the repository, or not answerable at all.
*/
afterEach(function (): void {
Project::destroyAll();
});
test('a declared default branch beats autodetection', function (): void {
// The repository autodetects `develop`, which holds no baseline. Only the
// declaration in `tests/Pest.php` can reach the `master` one.
$project = Project::make('develop', overlay: 'configured-default-branch');
$project->seed('master');
@@ -35,20 +29,32 @@ test('a declared default branch that does not exist degrades to a full run', fun
$result = $project->pest('--tia');
// Nothing to fall back to, so everything runs — and no baseline is minted
// under the name that resolved to nothing.
expect($result->uncached())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->exitCode)->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', 'feature-x']);
})->skipOnWindows();
test('a renamed default branch replays and writes under its new name', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->rename('master', 'main');
$result = $project->pest('--tia');
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', 'main'])
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($delta->writtenCount())->toBe(0, $delta->summary());
})->skipOnWindows();
test('the CI provider names the default branch where the checkout cannot', function (): void {
// A CI checkout: `actions/checkout` fetches a single ref instead of cloning,
// so there is no `origin/HEAD` for git to read the default branch from. The
// event payload GitHub hands the job says it outright.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->seed('master');
$project->addBaseline('legacy');
$project->git()->switchTo('feature-x', new: true);
@@ -68,6 +74,7 @@ test('GitLab names the default branch through its own variable', function (): vo
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->seed('master');
$project->addBaseline('legacy');
$project->git()->switchTo('feature-x', new: true);
@@ -80,10 +87,6 @@ test('GitLab names the default branch through its own variable', function (): vo
})->skipOnWindows();
test('a lone recorded baseline names the default branch', function (): void {
// Nothing left to ask: no `origin/HEAD`, no CI provider, and an
// `init.defaultBranch` that names a branch this repository does not have.
// The graph holds exactly one baseline, and it is the only one any branch
// could read — so it is the answer.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'main');
@@ -98,9 +101,6 @@ test('a lone recorded baseline names the default branch', function (): void {
})->skipOnWindows();
test('a default branch nothing can name is refused rather than guessed', function (): void {
// Same checkout as above, without the graph that answered it. Guessing here
// is what made this bug expensive: the guess reads no baseline at all, and
// the output calls that a hit.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'main');
@@ -116,9 +116,6 @@ test('a default branch nothing can name is refused rather than guessed', functio
})->skipOnWindows();
test('an init.defaultBranch naming a branch that exists is still trusted', function (): void {
// The setting is the machine's, not the repository's — worth taking only
// where the repository has a branch by that name. It does here, and with no
// graph on disk it is the only source left, so the run must not be refused.
$project = Project::make('master');
$project->git()->unsetOriginHead();
$project->git()->config('init.defaultBranch', 'master');
@@ -129,7 +126,7 @@ test('an init.defaultBranch naming a branch that exists is still trusted', funct
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->output)->not->toContain('could not determine the default branch')
->and($project->branchKeys())->toBe(['feature-x']);
->and($project->branchKeys())->not->toContain('master');
})->skipOnWindows();
test('a repository with no remote is refused rather than silently re-run', function (): void {
@@ -141,16 +138,11 @@ test('a repository with no remote is refused rather than silently re-run', funct
$result = $project->pest('--tia');
// Nothing can name the default branch, so every new branch would re-run the
// whole suite with no explanation. Saying so beats doing that quietly. The
// missing remote is the likeliest reason and gets named as such.
expect($result->output)->toContain('Tia mode requires a repository with a remote.')
->and($result->exitCode)->toBe(1, $result->describe());
})->skipOnWindows();
test('a remote-less repository holding one baseline is not refused', function (): void {
// The refusal above exists to stop a guess, not to demand a remote for its
// own sake. With a baseline on disk there is nothing left to guess at.
$project = Project::make('master');
$project->git()->removeOrigin();
@@ -165,8 +157,6 @@ test('a remote-less repository holding one baseline is not refused', function ()
})->skipOnWindows();
test('a declared default branch stands in for a missing remote', function (): void {
// The escape hatch the refusal above points at: with the branch named by
// hand there is nothing left for a remote to answer.
$project = Project::make('master', overlay: 'configured-default-branch');
$project->git()->removeOrigin();
@@ -185,8 +175,6 @@ test('tia still requires git', function (): void {
$result = $project->pest('--tia');
// The soft default-branch resolver runs before the branch is named, and it
// must not swallow this.
expect($result->output)->toContain('The feature "Tia mode" requires "git".')
->and($result->exitCode)->not->toBe(0);
})->skipOnWindows();
@@ -212,8 +200,6 @@ test('the default branch is resolved once per run, not once per test', function
expect($git)->not->toBeNull();
// A `git` first on `PATH` that records what it was asked before handing over
// to the real one.
$log = $project->path('git-calls.log');
$project->write('shim/git', implode("\n", [
'#!/bin/sh',
+69 -39
View File
@@ -4,12 +4,6 @@ declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
/**
* What the fallback is allowed to touch.
*
* Reading another branch's baseline must stay a read: writes belong to the
* branch that ran, and a branch that has no name of its own must not mint one.
*/
afterEach(function (): void {
Project::destroyAll();
});
@@ -20,58 +14,43 @@ test('narrows to the affected tests on a new branch', function (): void {
$project->git()->switchTo('feature-x', new: true);
// Semantic, not cosmetic: PHP is hashed at the AST level, so a comment
// would not register as a change at all.
$project->write('app/Calculator.php', <<<'PHP'
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
namespace Fixture\App;
use Fixture\App\Greeter;
final class Calculator
{
public function add(int $a, int $b): int
{
return $a + $b;
}
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
expect((new Greeter)->greet('Nuno'))->toBeString();
});
public function subtract(int $a, int $b): int
{
return $a - $b;
}
public function multiply(int $a, int $b): int
{
return $a * $b;
}
}
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
PHP);
$result = $project->pest('--tia');
$delta = $project->delta();
// Two of the three test files cover `Calculator`; the third replays from the
// default branch's baseline.
expect($result->affected())->toBe(4, $result->describe())
->and($result->replayed())->toBe(2, $result->describe())
->and($result->exitCode)->toBe(0, $result->describe());
expect($result->affected())->toBe(2, $result->describe())
->and($result->replayed())->toBe(4, $result->describe())
->and($result->exitCode)->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master', 'feature-x'])
->and($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($delta->edgesMoved())->toBeFalse($delta->summary());
})->skipOnWindows();
test('filtered mode reads the fallback too', function (): void {
$project = Project::make('master');
// A failure cached on the default branch. Filtered mode asks the graph which
// test files are due a re-run, and that read has to reach the fallback as
// well: without it a new branch sees nothing to do, reports green, and never
// re-runs the failure — on every subsequent invocation.
$project->seed('master', failing: ['adds two numbers']);
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia', '--filtered');
// Selection is by file, so the failure's two siblings in `CalculatorTest`
// come along; the other two test files stay out of the run entirely.
expect($result->output)->toContain('from 1 previously unsuccessful test')
->and($result->output)->not->toContain('No affected tests found')
->and($result->tally())->toContain('2 passed');
@@ -91,6 +70,43 @@ test('filtered mode finds nothing to do on a clean green feature branch', functi
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
test('filtered mode falls back to a full replay when a cached failure cannot be located', function (): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
// A path this project cannot address at all — recorded on another machine.
// A path that merely no longer exists is a *deleted* test, not a lost one,
// and widening the run would not find it either; see StateReclamation.
$project->mutateGraph(function (array $graph): array {
$testId = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$testId]['file'] = '/build/agent/tests/Unit/DeletedTest.php';
return $graph;
});
$project->git()->switchTo('feature-x', new: true);
$result = $project->pest('--tia', '--filtered');
expect($result->output)->toContain('Some cached tests due a re-run could not be located on disk.')
->and($result->output)->toContain('Running the full suite with replay instead of a filtered run.')
->and($result->output)->not->toContain('No affected tests found');
})->skipOnWindows();
test('filtered mode finds nothing to do on the default branch itself', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--filtered');
$delta = $project->delta();
expect($result->output)->toContain('No affected tests found')
->and($result->exitCode)->toBe(0, $result->describe())
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a detached HEAD replays without minting a branch key', function (): void {
$project = Project::make('master');
$project->seed('master');
@@ -99,13 +115,27 @@ test('a detached HEAD replays without minting a branch key', function (): void {
$result = $project->pest('--tia');
// A detached HEAD has no branch of its own. The default branch is the
// honest key, and no phantom one appears beside it.
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe())
->and($project->branchKeys())->toBe(['master']);
})->skipOnWindows();
test('a detached HEAD does not write into the default branch baseline', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$project->pest(...$arguments);
$delta = $project->delta();
expect($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($project->branchKeys())->toBe(['master']);
})->with([
'sequential' => [['--filter=adds two numbers']],
'parallel' => [['--parallel', '--processes=2', '--filter=adds two numbers']],
])->skipOnWindows();
test('the branch that ran gets its own key and the default branch keeps its baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
+87
View File
@@ -0,0 +1,87 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
test('re-runs a cached failure on a clean tree', function (): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
$result = $project->pest('--tia', '--filtered');
$delta = $project->delta();
expect($result->output)->toContain('from 1 previously unsuccessful test')
->and($result->affected())->toBe(2, $result->describe())
->and($result->tally())->toContain('2 passed')
->and($delta->writtenCount())->toBe(2, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('an explicit path turns filtered mode off', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--filtered', 'tests/Unit');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($result->output)->not->toContain('No affected tests found')
->and($result->tally())->toContain('4 passed')
->and($delta->writtenCount())->toBe(4, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('filtered mode runs the whole suite when there is no baseline', function (): void {
$project = Project::make('master');
$result = $project->pest('--tia', '--filtered');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($result->output)->not->toContain('No affected tests found');
})->skipOnWindows();
test('filtered mode finds nothing to do in parallel either', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--filtered', '--parallel', '--processes=2');
$delta = $project->delta();
expect($result->output)->toContain('No affected tests found')
->and($result->exitCode)->toBe(0, $result->describe())
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a corrupt graph is reported and does not crash the run', function (): void {
$project = Project::make('master');
$project->seed('master');
$graph = $project->graphDir().'/graph.json';
file_put_contents($graph, '{not json');
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($result->output)->toContain('The dependency graph could not be read')
->and(is_file($graph) ? file_get_contents($graph) : null)->not->toBe('{not json');
})->skipOnWindows();
test('--parallel --retry is refused and leaves the graph alone', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--parallel', '--retry');
$delta = $project->delta();
expect($result->exitCode)->not->toBe(0)
->and($result->output)->toContain('--retry')
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->skipOnWindows();
+226
View File
@@ -0,0 +1,226 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
/*
* Invariant 7 — a hostile state dir cannot break a run. Whatever is in
* graph.json, the suite still runs and exits on the tests' merit.
*/
test('a graph mangled beyond use still lets the suite run', function (string $contents): void {
$project = Project::make('master');
$project->seed('master');
file_put_contents($project->graphDir().'/graph.json', $contents);
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->with([
'empty' => '',
'truncated' => '{"schema":1,"files":["app/Calculator.php"],"edg',
'not json' => '{not json',
'json scalar' => '"just a string"',
'json list' => '[1,2,3]',
'json null' => 'null',
'empty object' => '{}',
'nul bytes' => "\0\0\0\0",
])->skipOnWindows();
test('a graph whose shape is wrong everywhere is repaired rather than trusted', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id] = 'nope';
$graph['baselines']['master']['results'][7] = ['status' => 0, 'message' => '', 'time' => 0.1];
$graph['baselines']['master']['tree'] = 'nope';
$graph['baselines']['master']['sha'] = 42;
$graph['baselines'][''] = ['sha' => null, 'tree' => [], 'results' => []];
$graph['baselines']['broken'] = 'nope';
$graph['edges']['tests/Unit/GreeterTest.php'] = 'nope';
$graph['edges'][''] = [0];
$graph['files'][] = ['nested'];
return $graph;
});
$result = $project->pest('--tia', ...$arguments);
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->output)->not->toContain('TypeError')
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($project->branchKeys())->toBe(['master']);
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a cached status this build cannot interpret is re-run, not replayed', function (int $status): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph) use ($status): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = $status;
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($result->uncached())->toBe(1, $result->describe());
})->with([
'below the range' => -1,
'one past the range' => 9,
'far past the range' => 99,
'huge' => PHP_INT_MAX,
])->skipOnWindows();
test('a cached status with no replay of its own does not fail the run', function (int $status): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph) use ($status): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = $status;
$graph['baselines']['master']['results'][$id]['message'] = 'cached detail';
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->with(['notice' => 3, 'deprecation' => 4, 'warning' => 6])->skipOnWindows();
test('a cached skip or todo replays with its message intact', function (int $status, string $tally): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph) use ($status): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = $status;
$graph['baselines']['master']['results'][$id]['message'] = 'a recorded reason';
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain($tally)
->and($result->output)->toContain('a recorded reason')
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
})->with([
'skipped' => [1, '1 skipped'],
'incomplete' => [2, '1 incomplete'],
])->skipOnWindows();
test('a cached failure with a multi-line message re-runs rather than replaying the text', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = 7;
$graph['baselines']['master']['results'][$id]['message'] = "line one\nline two\nline three";
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->uncached())->toBe(1, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->skipOnWindows();
test('a result pointing outside the project is not addressable and widens the run', function (): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
$project->mutateGraph(function (array $graph): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['file'] = '/build/agent/tests/Unit/CalculatorTest.php';
return $graph;
});
$result = $project->pest('--tia', '--filtered');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->output)->toContain('could not be located on disk')
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->skipOnWindows();
test('an edge pointing at a file id that does not exist is ignored', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph): array {
$graph['edges']['tests/Unit/CalculatorTest.php'] = [0, 999, -5];
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
})->skipOnWindows();
test('a graph from a schema this build does not know is rebuilt, not read', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph): array {
$graph['schema'] = 2;
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($result->replayed())->toBe(0, $result->describe());
})->skipOnWindows();
test('graph.json being a directory does not stop the run', function (): void {
$project = Project::make('master');
$project->seed('master');
unlink($project->graphDir().'/graph.json');
mkdir($project->graphDir().'/graph.json');
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->skipOnWindows();
test('a state dir it cannot write to still replays', function (): void {
$project = Project::make('master');
$project->seed('master');
chmod($project->graphDir(), 0500);
try {
$result = $project->pest('--tia');
} finally {
chmod($project->graphDir(), 0700);
}
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe());
})->skipOnWindows();
+143
View File
@@ -0,0 +1,143 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
/*
* A test that triggers a notice, deprecation or warning is reported by PHPUnit
* as passed, and emits Passed. Recording it as a plain success made the cache
* hide the issue: a later run under --fail-on-* came back green where a fresh
* run failed. Invariant 3 — replay is faithful — at its most dangerous.
*/
function tiaTriggering(string $call): string
{
return <<<PHP
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
test('adds two numbers', function (): void {
{$call}
expect((new Calculator)->add(1, 2))->toBe(3);
});
test('subtracts two numbers', function (): void {
expect((new Calculator)->subtract(3, 1))->toBe(2);
});
PHP;
}
test('a triggered issue is recorded as itself, not as a pass', function (array $arguments, string $call, int $status): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering($call));
$project->git()->commit('trigger an issue');
$project->pest('--tia', ...$arguments);
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe($status);
})->with(Project::SEQUENTIAL_AND_PARALLEL)->with([
'deprecation' => ["trigger_error('legacy adder', E_USER_DEPRECATED);", 4],
'notice' => ["trigger_error('a notice', E_USER_NOTICE);", 3],
'warning' => ["trigger_error('a warning', E_USER_WARNING);", 6],
])->skipOnWindows();
test('a cached deprecation still fails the run that asked to fail on one', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"));
$project->git()->commit('trigger a deprecation');
$project->pest('--tia', ...$arguments);
$result = $project->pest('--tia', '--fail-on-deprecation', ...$arguments);
expect($result->exitCode)->toBe(1, $result->describe())
->and($result->tally())->toContain('1 deprecated')
->and($result->uncached())->toBe(1, $result->describe());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('replaying a cached issue does not downgrade it to a pass', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"));
$project->git()->commit('trigger a deprecation');
$project->pest('--tia', ...$arguments);
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
foreach (range(1, 3) as $ignored) {
$project->pest('--tia', ...$arguments);
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(4);
}
$result = $project->pest('--tia', '--fail-on-deprecation', ...$arguments);
expect($result->exitCode)->toBe(1, $result->describe());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a failure outranks an issue triggered on the way to it', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', str_replace(
'toBe(3)',
'toBe(999)',
tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"),
));
$project->git()->commit('an issue then a failure');
$result = $project->pest('--tia');
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
expect($result->exitCode)->not->toBe(0)
->and($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(7);
})->skipOnWindows();
test('a skip outranks an issue triggered on the way to it', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', str_replace(
'expect((new Calculator)->add(1, 2))->toBe(3);',
"\$this->markTestSkipped('not today');",
tiaTriggering("trigger_error('legacy adder', E_USER_DEPRECATED);"),
));
$project->git()->commit('an issue then a skip');
$project->pest('--tia');
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(1);
})->skipOnWindows();
test('a suppressed issue is not recorded', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', tiaTriggering("@trigger_error('quiet', E_USER_DEPRECATED);"));
$project->git()->commit('a suppressed deprecation');
$project->pest('--tia');
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
expect($project->graph()['baselines']['master']['results'][$id]['status'])->toBe(0);
})->skipOnWindows();
+170
View File
@@ -0,0 +1,170 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
test('a filtered run rewrites only the test that ran', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--filter=adds two numbers');
$delta = $project->delta();
expect($result->tally())->toContain('1 passed')
->and($result->output)->not->toContain('TIA does not apply to partial runs')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a filtered run under --tia announces that tia does not apply', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a test suffix narrows the tier even though every test runs', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--test-suffix=Test.php');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a dirty run narrows to the uncommitted test edit', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/GreeterTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Greeter;
test('greets a person', function (): void {
expect((new Greeter)->greet('Nuno'))->toBe('Hello, Nuno!');
});
test('greets the world', function (): void {
expect((new Greeter)->greet('world'))->toBe('Hello, world!');
});
test('greets again', function (): void {
expect((new Greeter)->greet('again'))->toBe('Hello, again!');
});
PHP);
$result = $project->pest('--tia', '--dirty');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($result->tally())->toContain('3 passed')
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->structureMoved())->toBeFalse($delta->summary());
})->skipOnWindows();
test('filtered mode yields to an explicit filter', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--filtered', '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($result->tally())->toContain('1 passed')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('an env flag narrows exactly like the option it mirrors', function (string $variable): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pestWithEnvironment($project->path(), [
$variable => '1',
], '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->with(['PEST_TIA', 'PEST_TIA_FILTERED'])->skipOnWindows();
test('a partial run does not purge the graph even with --fresh', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--fresh', '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('--no-tia does not stop a partial run from refreshing its own entry', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--no-tia', '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->not->toContain('TIA does not apply to partial runs')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('two partial runs each keep the other entry', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->pest('--filter=adds two numbers');
$project->pest('--filter=greets a person');
$delta = $project->delta();
expect($delta->writtenCount())->toBe(2, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a shard is a partial run', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--shard=1/2');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
test('a parallel partial run records the test that ran, like a sequential one', function (): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--parallel', '--processes=2', '--filter=adds two numbers');
$delta = $project->delta();
expect($result->output)->toContain('TIA does not apply to partial runs')
->and($result->tally())->toContain('1 passed')
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
+345
View File
@@ -0,0 +1,345 @@
<?php
declare(strict_types=1);
use Tests\Fixtures\Tia\Project;
afterEach(function (): void {
Project::destroyAll();
});
test('a detached HEAD does not purge the graph on structural drift', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$project->write('composer.lock', (string) json_encode([
'content-hash' => 'drifted',
'packages' => [],
'packages-dev' => [],
]));
$result = $project->pest('--tia', ...$arguments);
$delta = $project->delta();
expect($result->exitCode)->toBe(0, $result->describe())
->and($project->graphExists())->toBeTrue('the detached run deleted graph.json')
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a detached HEAD does not purge the graph with --fresh either', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$result = $project->pest('--tia', '--fresh', ...$arguments);
$delta = $project->delta();
expect($result->exitCode)->toBe(0, $result->describe())
->and($project->graphExists())->toBeTrue('the detached --fresh run deleted graph.json')
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a detached HEAD leaves an unreadable graph for a checkout that can rebuild it', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
file_put_contents($project->graphDir().'/graph.json', '{not json');
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and(file_get_contents($project->graphDir().'/graph.json'))->toBe('{not json');
})->skipOnWindows();
test('a cached failure whose test file was deleted stops widening later runs', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
unlink($project->path('tests/Unit/CalculatorTest.php'));
$project->git()->commit('drop CalculatorTest');
$project->pest('--tia', '--filtered', ...$arguments);
$project->snapshot();
$second = $project->pest('--tia', '--filtered', ...$arguments);
$delta = $project->delta();
expect($second->output)->not->toContain('could not be located on disk')
->and($second->output)->toContain('No affected tests found')
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a complete run reclaims the entry and the edge of a deleted test file', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
unlink($project->path('tests/Unit/CalculatorTest.php'));
$project->git()->commit('drop CalculatorTest');
$project->pest('--tia', ...$arguments);
$graph = $project->graph();
expect($graph['edges'] ?? [])->not->toHaveKey('tests/Unit/CalculatorTest.php')
->and(array_column($graph['baselines']['master']['results'] ?? [], 'file'))
->not->toContain('tests/Unit/CalculatorTest.php');
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a pruned result does not come back from the fallback', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
$project->git()->switchTo('feature-x', new: true);
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
test('adds two numbers, renamed', function (): void {
expect((new Calculator)->add(1, 2))->toBe(3);
});
test('subtracts two numbers', function (): void {
expect((new Calculator)->subtract(3, 1))->toBe(2);
});
PHP);
$project->git()->commit('rename the test on the branch');
$project->pest('--tia', '--filtered', ...$arguments);
$second = $project->pest('--tia', '--filtered', ...$arguments);
expect($second->output)->not->toContain('previously unsuccessful')
->and($second->output)->toContain('No affected tests found');
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('the fallback still reaches a branch that has never run a test file', function (): void {
$project = Project::make('master');
$project->seed('master', failing: ['adds two numbers']);
$project->git()->switchTo('feature-x', new: true);
// A narrowed run mints the branch key holding only the Greeter entries, so
// the layering must still serve master's cached failure for the Calculator.
$project->pest('--filter=greets a person');
$result = $project->pest('--tia', '--filtered');
expect($result->output)->toContain('previously unsuccessful')
->and($result->affected())->toBe(2, $result->describe());
})->skipOnWindows();
test('a branch that git no longer knows loses its baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia');
expect($project->branchKeys())->toBe(['master', 'feature-x']);
$project->git()->switchTo('master');
$project->git()->run(['branch', '-D', 'feature-x']);
$project->pest('--tia');
expect($project->branchKeys())->toBe(['master']);
})->skipOnWindows();
test('an unknown cached status is re-run rather than replayed as a failure', function (int $status): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph) use ($status): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = $status;
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($result->uncached())->toBe(1, $result->describe());
})->with(['unknown' => -1, 'future' => 9, 'garbage' => 99])->skipOnWindows();
test('a cached notice, deprecation or warning does not replay as a failure', function (int $status): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph) use ($status): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id]['status'] = $status;
$graph['baselines']['master']['results'][$id]['message'] = 'cached detail';
return $graph;
});
$result = $project->pest('--tia');
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->with(['notice' => 3, 'deprecation' => 4, 'warning' => 6])->skipOnWindows();
test('a malformed baseline entry cannot break the run', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->mutateGraph(function (array $graph): array {
$id = Project::testId('tests/Unit/CalculatorTest.php', 'adds two numbers');
$graph['baselines']['master']['results'][$id] = 'nope';
$graph['baselines']['master']['tree'] = 'nope';
$graph['edges']['tests/Unit/GreeterTest.php'] = 'nope';
return $graph;
});
$result = $project->pest('--tia', ...$arguments);
expect($result->exitCode)->toBe(0, $result->describe())
->and($result->output)->not->toContain('TypeError')
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed');
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
/*
* Invariant 1 — sequential and parallel must agree — under a process that is
* torn down in the middle of a test file. A worker that flushed what it got to
* before dying has not seen enough of that file to license pruning the
* siblings it never reached.
*/
test('a run torn down mid-file does not prune the tests it never reached', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
test('adds two numbers', function (): void {
expect((new Calculator)->add(1, 2))->toBe(3);
});
test('subtracts two numbers', function (): void {
exit(0);
});
PHP);
$project->git()->commit('a test that kills its own process');
$project->pest('--tia', ...$arguments);
$delta = $project->delta();
expect($delta->removed())->toBe(0, $delta->summary())
->and($delta->shaMoved())->toBeFalse($delta->summary())
->and($delta->structureMoved())->toBeFalse($delta->summary())
->and(array_keys($project->graph()['baselines']['master']['results']))
->toContain(Project::testId('tests/Unit/CalculatorTest.php', 'subtracts two numbers'));
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a fatal error mid-file is a test error, not a truncation', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->write('tests/Unit/CalculatorTest.php', <<<'PHP'
<?php
declare(strict_types=1);
use Fixture\App\Calculator;
test('adds two numbers', function (): void {
expect((new Calculator)->add(1, 2))->toBe(3);
});
test('subtracts two numbers', function (): void {
undefined_function_here();
});
PHP);
$project->git()->commit('a test that fatals');
$result = $project->pest('--tia', ...$arguments);
$delta = $project->delta();
expect($result->exitCode)->not->toBe(0)
->and($result->tally())->toContain('1 failed')
->and($delta->removed())->toBe(0, $delta->summary())
->and($delta->structureMoved())->toBeFalse($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
/*
* Invariant 2 — every command lands in exactly one tier and stays inside it —
* for the combinations that were never exercised.
*/
test('a green complete run leaves the graph exactly as it found it', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', ...$arguments);
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->with([
'bail' => [['--bail']],
'stop-on-failure' => [['--stop-on-failure']],
'compact' => [['--compact']],
'parallel bail' => [['--parallel', '--processes=2', '--bail']],
'parallel one process' => [['--parallel', '--processes=1']],
'parallel more processes than files' => [['--parallel', '--processes=8']],
])->skipOnWindows();
test('--tia --no-tia is a plain run that still refreshes what it executed', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--no-tia', ...$arguments);
$delta = $project->delta();
expect($result->replayed())->toBe(0, $result->describe())
->and($result->tally())->toContain(Project::TOTAL_TESTS.' passed')
->and($delta->writtenCount())->toBe(Project::TOTAL_TESTS, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('--fresh on a partial run neither purges nor prunes', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$result = $project->pest('--tia', '--fresh', '--filter=adds two numbers', ...$arguments);
$delta = $project->delta();
expect($result->exitCode)->toBe(0, $result->describe())
->and($project->graphExists())->toBeTrue()
->and($delta->writtenCount())->toBe(1, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
test('a second green run on a feature branch writes nothing at all', function (array $arguments): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--tia', ...$arguments);
$project->snapshot();
$result = $project->pest('--tia', ...$arguments);
$delta = $project->delta();
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($delta->isHardSuppressed())->toBeTrue($delta->summary());
})->with(Project::SEQUENTIAL_AND_PARALLEL)->skipOnWindows();
+17
View File
@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Runs TiaReplayHooks.php as a suite of its own, so `tests/Features/Tia.php` can
replay it without narrowing the run to a path. TIA declines to replay a
narrowed run, and a single-file invocation is exactly that.
-->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
bootstrap="../../../vendor/autoload.php"
colors="true"
>
<testsuites>
<testsuite name="default">
<file>TiaReplayHooks.php</file>
</testsuite>
</testsuites>
</phpunit>
-34
View File
@@ -4,19 +4,10 @@ declare(strict_types=1);
namespace Tests\Fixtures\Tia;
use Pest\Plugins\Tia\Storage;
use RuntimeException;
use Symfony\Component\Process\Process;
/**
* A git repository the scenario tests drive.
*
* Hermetic by construction: every invocation neutralises the machine's global
* and system config and carries its own committer identity. Without that, an
* ambient `init.defaultBranch = main` would answer questions the scenario means
* to leave unanswered, and rows like "resolves to nothing, degrades safely"
* would pass by accident.
*
* @internal
*/
final readonly class GitRepo
@@ -27,9 +18,6 @@ final readonly class GitRepo
public const array ENV = [
'GIT_CONFIG_GLOBAL' => '/dev/null',
'GIT_CONFIG_SYSTEM' => '/dev/null',
// `GIT_CONFIG_SYSTEM` does not cover every system-level file git reads:
// Apple's git also loads one from inside Xcode, and it sets
// `init.defaultBranch`. Only this suppresses all of them.
'GIT_CONFIG_NOSYSTEM' => '1',
'GIT_AUTHOR_NAME' => 'Pest Fixture',
'GIT_AUTHOR_EMAIL' => 'fixture@pestphp.io',
@@ -39,9 +27,6 @@ final readonly class GitRepo
public function __construct(public string $path) {}
/**
* Initialises the repository on `$branch` and commits everything in it.
*/
public function init(string $branch = 'master'): void
{
$this->run(['init', '--quiet']);
@@ -70,12 +55,6 @@ final readonly class GitRepo
$this->run(['checkout', '--quiet', '--detach']);
}
/**
* Registers an `origin`, which also decides the graph's storage key:
* {@see Storage::projectKey()} prefers the remote's
* identity over the path, so two checkouts of one repository — a worktree,
* say — share a single graph.
*/
public function addOrigin(string $url = 'git@github.com:pestphp/tia-fixture.git'): void
{
$this->run(['remote', 'add', 'origin', $url]);
@@ -86,22 +65,12 @@ final readonly class GitRepo
$this->run(['remote', 'remove', 'origin']);
}
/**
* Points `refs/remotes/origin/HEAD` at a local branch — what
* `git remote set-head` would write, without a remote to talk to.
*/
public function setOriginHead(string $branch): void
{
$this->run(['update-ref', 'refs/remotes/origin/'.$branch, 'HEAD']);
$this->run(['symbolic-ref', 'refs/remotes/origin/HEAD', 'refs/remotes/origin/'.$branch]);
}
/**
* Drops `refs/remotes/origin/HEAD` while keeping the remote-tracking branch
* — what a CI checkout looks like. `actions/checkout` builds its working
* copy with `git init` plus a single-ref `fetch` rather than a `clone`, and
* only a `clone` writes that symbolic ref.
*/
public function unsetOriginHead(): void
{
$this->run(['symbolic-ref', '--delete', 'refs/remotes/origin/HEAD']);
@@ -112,9 +81,6 @@ final readonly class GitRepo
$this->run(['config', '--local', $key, $value]);
}
/**
* Adds a worktree for a new branch and returns its path.
*/
public function worktree(string $path, string $branch): string
{
$this->run(['worktree', 'add', '--quiet', '-b', $branch, $path]);
+3 -49
View File
@@ -5,20 +5,6 @@ declare(strict_types=1);
namespace Tests\Fixtures\Tia;
/**
* What one run did to the graph.
*
* The three tiers a run may respect, from the conformance matrix:
*
* - COMPLETE — may change everything.
* - RESULTS-ONLY — may change only `baselines[<branch>].results` for tests that
* actually ran. It may never remove an entry, nor move `sha`, `tree`,
* `edges`, `files` or `fingerprint`.
* - HARD-SUPPRESSED — may change nothing at all.
*
* {@see self::writtenCount()} is the load-bearing measurement, and the reason
* {@see Project::sentinel()} exists: without falsified cached values there is no
* way to tell "wrote the same values back" from "wrote nothing".
*
* @internal
*/
final readonly class GraphDelta
@@ -42,9 +28,6 @@ final readonly class GraphDelta
return $this->before !== null && $this->after === null;
}
/**
* Result entries whose stored values actually moved.
*/
public function writtenCount(): int
{
$written = 0;
@@ -71,9 +54,6 @@ final readonly class GraphDelta
return $written;
}
/**
* Result entries that appeared.
*/
public function added(): int
{
$added = 0;
@@ -88,9 +68,6 @@ final readonly class GraphDelta
return $added;
}
/**
* Result entries that were pruned.
*/
public function removed(): int
{
$removed = 0;
@@ -106,14 +83,11 @@ final readonly class GraphDelta
}
/**
* The baseline keys after the run — the headline signal for the
* default-branch rows, where a phantom key is the defect.
*
* @return array<int, string>
*/
public function branchKeys(): array
{
return array_keys($this->baselines($this->after));
return array_map(strval(...), array_keys($this->baselines($this->after)));
}
/**
@@ -121,7 +95,7 @@ final readonly class GraphDelta
*/
public function branchKeysBefore(): array
{
return array_keys($this->baselines($this->before));
return array_map(strval(...), array_keys($this->baselines($this->before)));
}
public function branchKeysMoved(): bool
@@ -129,10 +103,6 @@ final readonly class GraphDelta
return $this->branchKeysBefore() !== $this->branchKeys();
}
/**
* Whether the named branch's baseline is byte-identical — how a row proves
* the fallback is read-only.
*/
public function baselineUntouched(string $branch): bool
{
return ($this->baselines($this->before)[$branch] ?? null)
@@ -149,11 +119,6 @@ final readonly class GraphDelta
return array_any($this->branchKeys(), fn (string $branch) => $this->baselineField($branch, 'tree', $this->before) !== $this->baselineField($branch, 'tree', $this->after));
}
/**
* Compares edges by the file paths they resolve to, not by file id: ids are
* an implementation detail that shifts whenever `files` is rebuilt in a
* different order.
*/
public function edgesMoved(): bool
{
return $this->edgeSets($this->before) !== $this->edgeSets($this->after);
@@ -184,17 +149,11 @@ final readonly class GraphDelta
return $this->branchKeysMoved();
}
/**
* Nothing moved at all.
*/
public function isHardSuppressed(): bool
{
return $this->before === $this->after;
}
/**
* Results may have moved for tests that ran; nothing structural did.
*/
public function isResultsOnly(): bool
{
return ! $this->graphWasCreated()
@@ -206,9 +165,6 @@ final readonly class GraphDelta
&& $this->added() === 0;
}
/**
* A one-line verdict, for failure messages.
*/
public function summary(): string
{
if ($this->graphWasCreated()) {
@@ -298,9 +254,7 @@ final readonly class GraphDelta
$baselines = [];
foreach ($this->section($graph, 'baselines') as $branch => $baseline) {
if (is_string($branch)) {
$baselines[$branch] = $baseline;
}
$baselines[(string) $branch] = $baseline;
}
return $baselines;
+2 -29
View File
@@ -5,15 +5,10 @@ declare(strict_types=1);
namespace Tests\Fixtures\Tia;
/**
* The outcome of one `pest` invocation against a fixture project.
*
* @internal
*/
final readonly class PestResult
{
/**
* The run's output, with the terminal's escape sequences taken back out.
*/
public string $output;
/**
@@ -25,39 +20,26 @@ final readonly class PestResult
public int $exitCode,
) {
$this->output = (string) preg_replace([
'#\x1b[[][^A-Za-z]*[A-Za-z]#', // colours, cursor moves
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#', // hyperlinks
'#\x1b[[][^A-Za-z]*[A-Za-z]#',
'#\x1b\]8;[^\x1b\x07]*(?:\x1b\\\\|\x07)#',
], '', $output);
}
/**
* Tests whose cached result was replayed instead of executed.
*/
public function replayed(): int
{
return $this->recapFragment('replayed');
}
/**
* Tests that ran because the graph held nothing for them — the count that
* betrays a fallback which never resolved.
*/
public function uncached(): int
{
return $this->recapFragment('uncached');
}
/**
* Tests that ran because a file they depend on changed.
*/
public function affected(): int
{
return $this->recapFragment('affected');
}
/**
* The `Tests:` summary line, without its label or leading whitespace.
*/
public function tally(): string
{
if (preg_match('/^\s*Tests:\s+(.+)$/m', $this->output, $matches) !== 1) {
@@ -72,10 +54,6 @@ final readonly class PestResult
return str_contains($this->output, $needle);
}
/**
* A description of the run, for failure messages that would otherwise say
* only that 0 !== 6.
*/
public function describe(): string
{
return sprintf(
@@ -86,11 +64,6 @@ final readonly class PestResult
);
}
/**
* Read off the `Tests:` line rather than the whole output: the TIA headline
* counts affected *files*, and matching that instead would be a quietly
* wrong number.
*/
private function recapFragment(string $label): int
{
if (preg_match('/(\d+) '.preg_quote($label, '/').'/', $this->tally(), $matches) !== 1) {
+47 -167
View File
@@ -5,7 +5,6 @@ declare(strict_types=1);
namespace Tests\Fixtures\Tia;
use FilesystemIterator;
use Pest\Factories\TestCaseFactory;
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\ChangedFiles;
use Pest\Plugins\Tia\FileState;
@@ -19,28 +18,11 @@ use RuntimeException;
use Symfony\Component\Process\Process;
/**
* A throwaway Pest project the TIA scenario tests drive.
*
* Two facts about Pest shape everything here:
*
* 1. `bin/pest` derives the project root from **the autoloader it finds**, not
* from the working directory — `dirname($autoloadPath, 2)`. So the project
* owns a real `vendor/autoload.php` and a real copy of `bin/pest` at the path
* a composer install would have put them. A symlinked `vendor` would resolve
* `__DIR__` straight back to the Pest repository, and every scenario would
* silently measure the wrong project.
* 2. TIA cannot *record* without pcov or Xdebug, and CI has neither. So a
* scenario never records: {@see self::seed()} writes the graph a recording
* run would have written, and the run under test exercises the read path.
*
* @internal
*/
final class Project
{
/**
* Test file → the source files a recording run would have linked it to. The
* self-edge every test file gets is added on top of these.
*
* @var array<string, array<int, string>>
*/
public const array EDGES = [
@@ -50,8 +32,6 @@ final class Project
];
/**
* Test file → the descriptions it declares, in declaration order.
*
* @var array<string, array<int, string>>
*/
public const array TESTS = [
@@ -60,15 +40,20 @@ final class Project
'tests/Feature/CoversCalculatorTest.php' => ['adds within a feature test', 'subtracts within a feature test'],
];
/**
* Every test in the fixture suite.
*/
public const int TOTAL_TESTS = 6;
/**
* Every project scaffolded so far, so a row cannot leak one by failing
* before its own cleanup.
* A dataset for the rule that TIA must reach the same outcome sequentially
* and in parallel: the same command, run both ways, must leave the same graph.
*
* @var array<string, array<int, array<int, string>>>
*/
public const array SEQUENTIAL_AND_PARALLEL = [
'sequential' => [[]],
'parallel' => [['--parallel', '--processes=2']],
];
/**
* @var array<int, self>
*/
private static array $created = [];
@@ -85,13 +70,6 @@ final class Project
*/
private array $extraPaths = [];
/**
* The root whose graph this project reads and writes.
*
* Its own, except where a row seeds a worktree: a worktree's `.git` is a
* file rather than a directory, so {@see Storage} cannot read the remote
* from it and resolves a storage key of its own.
*/
private string $graphRoot;
private function __construct(public readonly string $path)
@@ -99,14 +77,6 @@ final class Project
$this->graphRoot = $path;
}
/**
* Scaffolds a project whose default branch is `$branch`, and hands it back
* checked out there.
*
* The repository is realistic on purpose: it has an `origin`, and an
* `origin/HEAD` naming `$branch`, which is what a checkout of a real project
* looks like and what the default branch is autodetected from.
*/
public static function make(string $branch = 'master', ?string $overlay = null): self
{
$project = self::scaffold($overlay);
@@ -118,9 +88,6 @@ final class Project
return $project;
}
/**
* Destroys every project scaffolded so far. Belongs in an `afterEach`.
*/
public static function destroyAll(): void
{
while (self::$created !== []) {
@@ -128,14 +95,6 @@ final class Project
}
}
/**
* A project that is not a git repository at all — for the rows that assert
* TIA still demands git, and that a plain run does not care.
*
* Lives in the system temp directory, so it is outside any enclosing
* repository, and owns a real `vendor` rather than a symlinked one, so its
* baseline key cannot collide with another fixture's.
*/
public static function withoutGit(?string $overlay = null): self
{
return self::scaffold($overlay);
@@ -151,12 +110,6 @@ final class Project
return $relative === '' ? $this->path : $this->path.DIRECTORY_SEPARATOR.$relative;
}
/**
* Overwrites a file in the project.
*
* Edits must be semantic: TIA hashes PHP at the AST level, so a
* comment-only change is not a change at all.
*/
public function write(string $relative, string $contents): void
{
$path = $this->path($relative);
@@ -171,14 +124,6 @@ final class Project
}
}
/**
* Adds a worktree for a new branch, scaffolded so `pest` can run in it, and
* returns its path.
*
* The graph is shared with the main checkout, which is the whole point: both
* resolve the same storage key, because {@see Storage} prefers the `origin`
* identity over the path.
*/
public function worktree(string $branch): string
{
$path = $this->path.'-worktree-'.preg_replace('/[^a-z0-9]+/i', '-', $branch);
@@ -190,18 +135,11 @@ final class Project
return $path;
}
/**
* Runs `pest` in the project and returns what happened.
*/
public function pest(string ...$arguments): PestResult
{
return $this->pestIn($this->path, ...$arguments);
}
/**
* Runs `pest` in `$directory` — a worktree, say — against this project's
* graph.
*/
public function pestIn(string $directory, string ...$arguments): PestResult
{
return $this->pestWithEnvironment($directory, [], ...$arguments);
@@ -222,11 +160,6 @@ final class Project
'PARATEST' => '0',
'PAO_DISABLE' => '1',
'HOME' => $this->home(),
// Blanked for the same reason `GitRepo::ENV` blanks git's own
// config: the default branch a CI provider reports is the one
// *its* build is for. Pest's suite runs on GitHub Actions, so
// without this every scenario would autodetect Pest's default
// branch instead of the fixture's.
'GITHUB_EVENT_PATH' => '',
'CI_DEFAULT_BRANCH' => '',
...$environment,
@@ -244,17 +177,6 @@ final class Project
}
/**
* Writes the graph a clean, green recording run on `$branch` would have
* written, and remembers it as the snapshot the next {@see self::delta()}
* compares against.
*
* Sentinelled by default: a row almost always wants to know which entries
* were written, and only a real recording run's values can answer that.
*
* `$failing` names descriptions to record as failures — a clean green run
* can never cache one, so a row that needs a cached failure to re-run has to
* be handed it.
*
* @param array<int, string> $failing
*/
public function seed(string $branch, bool $sentinel = true, array $failing = []): void
@@ -263,9 +185,6 @@ final class Project
}
/**
* Seeds the graph belonging to `$root` — a worktree, say, which resolves a
* storage key of its own.
*
* @param array<int, string> $failing
*/
public function seedFor(string $root, string $branch, bool $sentinel = true, array $failing = []): void
@@ -279,8 +198,6 @@ final class Project
$graph->setFingerprint(Fingerprint::compute($root));
$graph->setRecordedAtSha($branch, $sha);
// Hashes the tree as it stands, so the run under test sees nothing as
// changed — the same call the recording path makes.
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
$graph->markKnownTestFiles(array_keys(self::EDGES));
@@ -322,13 +239,6 @@ final class Project
$sentinel ? $this->sentinel() : $this->snapshot();
}
/**
* The id PHPUnit reports for a test in the fixture suite.
*
* Mirrors how Pest names generated test classes
* ({@see TestCaseFactory}): a wrong id here shows up as
* `0 replayed`, which every scenario asserts against.
*/
public static function testId(string $testFile, string $description): string
{
$basename = basename($testFile, '.php');
@@ -343,41 +253,53 @@ final class Project
return 'P\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relative).'::'.Str::evaluable($description);
}
/**
* Falsifies every cached value, so the next {@see self::delta()} can tell
* "wrote the same values back" from "wrote nothing".
*
* `assertions` is only ever falsified where it is already non-zero: risky,
* skipped and incomplete statuses are *derived* from "performed no
* assertions", so patching those would rewrite the status on replay and
* destroy the very discriminator this exists to provide.
*/
public function sentinel(): void
{
$this->mutateGraph(function (array $graph): array {
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
foreach (array_keys($baseline['results'] ?? []) as $testId) {
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
}
}
}
return $graph;
});
}
/**
* Adds a second, empty baseline key, so a lone recorded baseline can no
* longer stand in for the default branch.
*/
public function addBaseline(string $branch): void
{
$this->mutateGraph(function (array $graph) use ($branch): array {
$graph['baselines'][$branch] = ['sha' => null, 'tree' => [], 'results' => []];
return $graph;
});
}
/**
* @param callable(array<string, mixed>): array<string, mixed> $callback
*/
public function mutateGraph(callable $callback): void
{
$graph = $this->graph();
if ($graph === null) {
throw new RuntimeException('There is no graph to sentinel.');
throw new RuntimeException('There is no graph to mutate.');
}
foreach ($graph['baselines'] ?? [] as $branch => $baseline) {
foreach (array_keys($baseline['results'] ?? []) as $testId) {
$graph['baselines'][$branch]['results'][$testId]['time'] = 9.999;
if ((int) ($baseline['results'][$testId]['assertions'] ?? 0) > 0) {
$graph['baselines'][$branch]['results'][$testId]['assertions'] = 42;
}
}
}
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($graph, JSON_UNESCAPED_SLASHES));
$this->state()->write(Tia::KEY_GRAPH, (string) json_encode($callback($graph), JSON_UNESCAPED_SLASHES));
$this->snapshot();
}
/**
* The decoded graph, or `null` when there is none.
*
* @return array<string, mixed>|null
*/
public function graph(): ?array
@@ -400,7 +322,8 @@ final class Project
{
$baselines = $this->graph()['baselines'] ?? [];
return is_array($baselines) ? array_keys($baselines) : [];
// A branch named `12345` comes back from json_decode as an integer key.
return is_array($baselines) ? array_map(strval(...), array_keys($baselines)) : [];
}
public function graphDir(): string
@@ -413,43 +336,21 @@ final class Project
return is_file($this->graphDir().DIRECTORY_SEPARATOR.Tia::KEY_GRAPH);
}
/**
* Remembers the graph as it stands now.
*/
public function snapshot(): void
{
$this->snapshot = $this->graph();
}
/**
* What has happened to the graph since the last snapshot.
*/
public function delta(): GraphDelta
{
return new GraphDelta($this->snapshot, $this->graph());
}
/**
* Writes the `vendor` a composer install would have produced.
*
* Pest is mirrored in at `vendor/pestphp/pest` rather than pointed at,
* because Pest locates things from where its own files sit:
* `bin/pest` finds the project root by walking up from the autoloader it
* loads, and a parallel run picks its worker binary — and with it the
* worker's project root — from the directory the runner class was loaded
* from. Deferring to the repository's copy would resolve both back to the
* Pest repository, and every scenario would quietly measure that instead.
*
* Hardlinked where the filesystem allows it, so the mirror costs almost
* nothing and can never drift from the working tree.
*/
public function scaffoldVendor(string $directory): void
{
$pestRoot = dirname(__DIR__, 3);
$pest = $directory.'/vendor/pestphp/pest';
// `overrides`, `resources` and `stubs` come along because Pest loads
// them relative to `src` — the same list `BootExcludeList` walks.
foreach (['src', 'overrides', 'resources', 'stubs'] as $tree) {
self::mirror($pestRoot.'/'.$tree, $pest.'/'.$tree);
}
@@ -460,21 +361,11 @@ final class Project
self::mirror($pestRoot.'/composer.json', $pest.'/composer.json');
// Pest's own autoloader, with the mirrored copy taking precedence: the
// repository's `vendor` supplies PHPUnit, Symfony and the plugin
// packages, none of which care where they are loaded from.
file_put_contents($directory.'/vendor/autoload.php', sprintf(
"<?php\n\n\$loader = require %s;\n\$loader->addPsr4('Pest\\\\', __DIR__.'/pestphp/pest/src', true);\n\nreturn \$loader;\n",
var_export($pestRoot.'/vendor/autoload.php', true),
));
// Invoking the binary directly skips composer's bin proxy, which is
// what would otherwise define `$GLOBALS['_composer_bin_dir']`. Without
// it `Pest\Plugin\Loader` looks for `vendor/bin/../pest-plugins.json`
// relative to the working directory — so that is where the plugin list
// goes, and mirroring the repository's keeps it in step with
// composer.json. `vendor/bin` has to exist for the `..` in that path to
// resolve, empty though it is.
if (! is_dir($directory.'/vendor/bin') && ! @mkdir($directory.'/vendor/bin', 0755, true)) {
throw new RuntimeException(sprintf('Unable to create [%s].', $directory.'/vendor/bin'));
}
@@ -500,9 +391,6 @@ final class Project
}
/**
* `Storage` reads `HOME` from the environment, so the graph lands inside the
* throwaway project instead of the developer's real `~/.pest`.
*
* @template TReturn
*
* @param callable(): TReturn $callback
@@ -529,8 +417,6 @@ final class Project
throw new RuntimeException(sprintf('Unable to create [%s].', $path));
}
// Realpathed because Pest realpaths its own project root, and macOS
// hands out a symlinked temp directory.
$real = realpath($path);
$project = new self($real === false ? $path : $real);
@@ -543,8 +429,6 @@ final class Project
self::copy(__DIR__.'/overlays/'.$overlay, $project->path);
}
// `ChangedFiles` asks git what changed, so anything the scaffold writes
// but the project does not own has to be invisible to it.
$project->write('.gitignore', implode("\n", ['/vendor/', '/.home/', '/.phpunit.cache/', '']));
$project->scaffoldVendor($project->path);
@@ -553,10 +437,6 @@ final class Project
return $project;
}
/**
* Mirrors a file or directory, hardlinking where the filesystem allows it
* and copying where it does not.
*/
private static function mirror(string $from, string $to): void
{
if (is_dir($from)) {
@@ -4,8 +4,6 @@ declare(strict_types=1);
use Fixture\App\Calculator;
// A second test file over the same source file, so a `Calculator` edit narrows
// to two of the three test files rather than to one.
test('adds within a feature test', function (): void {
expect((new Calculator)->add(10, 5))->toBe(15);
});
-3
View File
@@ -2,8 +2,5 @@
declare(strict_types=1);
// The fixture project has no composer autoloader of its own — its `vendor/`
// holds nothing but a shim pointing back at Pest's. Requiring the two classes
// here is enough for every test file in the suite.
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
@@ -4,9 +4,6 @@ declare(strict_types=1);
use Fixture\App\Calculator;
// `test()` rather than `it()`: the harness seeds results by test id, and `it()`
// would prefix every description with `it `, leaving Project::TESTS a step away
// from what is written here.
test('adds two numbers', function (): void {
expect((new Calculator)->add(1, 2))->toBe(3);
});
@@ -0,0 +1,8 @@
<?php
declare(strict_types=1);
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
pest()->tia()->always();
@@ -5,7 +5,4 @@ declare(strict_types=1);
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
// Declared default branch. Wins over whatever the repository autodetects — the
// escape hatch for a checkout with no `origin/HEAD` and a misleading
// `init.defaultBranch`.
pest()->tia()->defaultBranch('master');
@@ -5,6 +5,4 @@ declare(strict_types=1);
require_once __DIR__.'/../app/Calculator.php';
require_once __DIR__.'/../app/Greeter.php';
// A branch the repository does not have. Accepted as configured — a name that
// resolves to no baseline degrades to a full run, which is safe.
pest()->tia()->defaultBranch('nope');
-2
View File
@@ -19,8 +19,6 @@ pest()->in('PHPUnit/GlobPatternTests/SubFolder2/*AsPattern.php')->use(CustomTest
pest()->in('Visual')->group('integration');
// Every row scaffolds a throwaway project and runs `pest` in it as a
// subprocess, which is far too slow for the unit suite.
pest()->in('Features/Tia')->group('integration');
// NOTE: global test value container to be mutated and checked across files, as needed
-2
View File
@@ -67,8 +67,6 @@ describe('applyMigrationChanges()', function (): void {
describe('rerun tracking', function (): void {
beforeEach(function (): void {
// `hasUnlocatedTestsToRerun()` stats each recorded file to tell a
// deleted test apart from a live one, so the files have to exist.
$this->projectRoot = sys_get_temp_dir().'/pest-tia-rerun-'.bin2hex(random_bytes(4));
mkdir($this->projectRoot.'/tests/Feature', 0755, true);
+2
View File
@@ -9,6 +9,8 @@ $run = function (): ?string {
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
);
$process->setTimeout(300.0);
$process->run();
return removeAnsiEscapeSequences($process->getOutput());
+2
View File
@@ -17,6 +17,8 @@ test('visual snapshot of test suite on success', function (): void {
['EXCLUDE' => 'integration', '--exclude-group' => 'integration', 'REBUILD_SNAPSHOTS' => false, 'PARATEST' => 0, 'COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
));
$process->setTimeout(300.0);
$process->run();
return preg_replace([