31 KiB
TIA write-tier conformance — phase two
Your task
Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app at
/Users/nunomaduro/Work/projects/playground/laravel. Every row, including the ones already marked
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
re-measured against the current code. The Phase 1 column is prior evidence and a hint at what to
watch, never a reason to skip a row.
These are not the pestphp/pest repo's own tests (composer test) — do not run those. Each row is
a pest invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do not stop at reading this file or summarising it; the deliverable is executed results.
Work in this order: Part 1 (environment traps) → Part 1b (build the fixtures — the playground has none of them) → Part 2 (the matrix) → Part 3 (priorities). The matrix is too large for one context; take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are the load-bearing ones — do those first if you run short.
Phase one implemented PLAN.md Part 1 items 1–4 plus three §5 items. This file turns the matrix into
a conformance check rather than a bug list.
Code under test: pestphp/pest at /Users/nunomaduro/Work/projects/pestphp/pest, branch
fix/tia-filtered, commit bfd5b756 or later. Verify with
grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php → at least 3 hits. Pre-fix
baseline for every contrast is db70017c.
Part 0 — What phase one changed
| # | Change | Files |
|---|---|---|
| 1 | hasUnlocatedTestsToRerun() stats the file, so a deleted test file is "unlocated" |
src/Plugins/Tia/Graph.php |
| 2 | enterReplayMode() uses activateLinkTracking() under piggyback coverage |
src/Plugins/Tia.php |
| 3 | enterReplayMode() stamps TIA_PIGGYBACK_COVERAGE for workers |
src/Plugins/Tia.php |
| 4 | replaceEdges(…, keepExisting:) — piggyback edges seed empty sets, never overwrite populated ones |
Graph.php, Tia.php |
| 5 | renderFreshGraph() stops claiming "fresh graph" when the graph is kept; reason reworded to recording a coverage baseline |
Tia.php |
| 6 | COVERAGE_REPORT_FLAGS + coverageReportActive() union over originalArguments; new pestCoverageActive() keeps the coverage-cache marker/hijack on Pest's own --coverage |
Tia.php |
| 7 | Tia::recordsEdgesInWorkers() + WrapperRunner::handleTia() inject -d pcov.directory=<root> into worker argv |
Tia.php, src/Plugins/Parallel/Paratest/WrapperRunner.php |
| 8 | Sequential record runs announce structural drift via renderFreshGraph() |
Tia.php |
| 9 | Graph::getTime() + cachedTimeByTestId + resultTime() preserve replayed durations; edge-less write guard is now $recordsEdges = $complete && ($markKnownTestFiles || $this->recordingActive) |
Graph.php, Tia.php |
Deliberately not done: PLAN.md §5 SIGINT propagation, §5 warning/deprecation status=0
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
Target-outcome changes this forces
Two rows in the original matrix asserted the old, buggy behaviour. Their targets are updated below — do not report them as regressions:
- B10 was "
timediffers; statuses stable". Change 9 means replayed entries now keep their recordedtime. New target:timediffers only for tests that actually executed. - B12 gains an edge-preservation assertion it never had (see K1).
One known-failing repo test
tests/Unit/Plugins/Tia/Graph.php:69-76 asserts hasUnlocatedTestsToRerun('main') is false for
tests/Feature/FooTest.php under new Graph(sys_get_temp_dir()) — a path that does not exist. That
assertion encodes the I9 bug and will fail under change 1. It needs re-pointing at a root/file
that exists (e.g. dirname(__DIR__, 4) + 'tests/Unit/Plugins/Tia/Graph.php'). Left untouched by
request; it is a repo-test matter, not a playground one.
Part 1 — Environment, and the traps in it
Playground: /Users/nunomaduro/Work/projects/playground/laravel
cd /Users/nunomaduro/Work/projects/playground/laravel
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
-
Every pest invocation needs
PAO_DISABLE=1— the app haslaravel/pao, which emits JSON when it detects an agent. -
Never run
git checkout .orgit checkout -- composer.lockin the playground. It has four pre-existing user-modified files —AGENTS.md,CLAUDE.md,composer.json,composer.lock— that a blanket reset would destroy.PLAN.md's "reset ritual" is unsafe as written. Scope resets to what you touched:git checkout -- tests app/git clean -fd tests app, and forcomposer.lock(needed for the drift rows) copy it aside and copy it back, verifying withshasum. -
vendor/pestphp/pestis a dist copy, not a symlink. Composer installeddev-fix/tia-filtered as 5.2.0, so edits in the pest repo do not reach the playground.Tell Nuno whenever a sync is needed to move forward — do not sync silently. Say what is stale and what the sync would be, then wait. This includes temporarily swapping in
db70017cfiles for a before/after contrast. When you report any playground result, state which commit produced it. The sync itself, once he agrees:PEST=/Users/nunomaduro/Work/projects/pestphp/pest V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \ src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; doneVerify with
grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"→1. Always restore the current version before continuing after a pre-fix contrast. -
Coverage driver: pcov only, no xdebug.
ini_get('pcov.directory')is''by default — that emptiness is the entire mechanism behind G12. -
Suite shape as found: 7 tests in 5 files —
tests/Unit/{CalculatorTest,ExampleTest,GreeterTest, TrioTest}.php,tests/Feature/ExampleTest.php. A healthy sequential graph isfiles=22, 5 edge keys, self-edge on every test (PLAN.md'sfiles=25is stale). Pre-fix parallel givesfiles=4with zero self-edges. These numbers shift the moment you add the Part 1b fixtures — re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants that do not shift: sequential and parallel must agree, and every test must have a self-edge. -
Sentinel patching is the only reliable discriminator between "wrote identical values" and "wrote nothing": rewrite every cached result to
time=9.999 assertions=42, snapshot, run the case, diff. A canary test absent fromedgesis unreliable under--filterbecause it never matches the filter and so never runs. -
Seeding a cached failure needs
--fresh(or an env-driven flaky fixture):--tiaon a clean green tree replays rather than executes, so it can never cache a failure. Working recipe — break an assertion,pest --tia --fresh, then restore the source. -
Comment-only edits to a test file are not changes (AST-level hashing). Use semantic edits.
-
The shell is zsh — build commands with arrays or
eval; unquoted$argsdoes not word-split.
Graph summariser
Write this to a scratch path and use it for every diff.
<?php // summarise.php <graph.json> [label]
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
ksort($edges);
foreach ($edges as $test => $ids) {
$self = 'no';
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
}
foreach ($g['baselines'] ?? [] as $branch => $b) {
$r = $b['results'] ?? [];
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
foreach ($r as $id => $x) {
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
}
}
Normalised edge-set equality (for G12 / I5 / K1):
<?php // edgediff.php <a.json> <b.json>
function edges(string $p): array {
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
ksort($out); return $out;
}
$a = edges($argv[1]); $b = edges($argv[2]);
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
foreach ($a as $t => $s) {
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
}
Per-case loop
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
Part 1b — Fixtures you must build first
The playground has none of the fixtures the matrix depends on. Verified inventory: the only test
files are tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php and
tests/Feature/ExampleTest.php — 7 plain tests, zero occurrences of ->group(), ->only(),
->skip(), ->todo(), ->note(), ->flaky(), ->covers(), ->uses(), ->issue(), ->pr(),
->ticket(), ->assignee(), datasets, or any FLAKY_OK-style env hook. phpunit.xml defines only
the Unit and Feature testsuites.
So roughly 30 rows below cannot run as written until you create the fixtures. Build them all up
front, in one commit, then re-derive the baseline graph shape once and use those numbers for the
whole sweep — every fixture you add changes files, the edges key count and n, so adding them
piecemeal invalidates earlier rows.
| Fixture to create | Rows that need it |
|---|---|
a test with ->group('smoke') |
C7, C8, C9, C10 |
an env-driven flaky test (passes iff FLAKY_OK=1) |
E1, E2, E3, E4 |
a ->skip()ed test |
E9, D8 |
a ->todo() test |
E10, C23, C39 |
a risky test (no assertions; pair with --disallow-test-output) |
E11, D7 |
| a test that triggers a PHPUnit warning | D6 |
a test calling markTestIncomplete() |
D9 |
| a test that triggers a deprecation | D10 |
| a dataset test with ≥3 rows | E13, E14 |
->only() — added and removed per case, not left in |
C31, C32, G7 |
->covers(App\Services\Calculator::class) |
C18 |
->uses(...) / UsesClass annotation |
C19 |
->note(...) |
C24 |
->flaky() annotation (distinct from the env-driven flaky test above) |
C25 |
->issue(123), ->pr(1), ->ticket('X'), ->assignee('X') |
C26, C27, C28, C37, C38 |
This is PLAN.md §6's "test-harness gaps to close before re-running", now itemised: without these,
the listed rows' graph-invariant assertions match zero tests and prove nothing — they pass
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous only because its fixture is
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
Rows needing an action rather than a fixture — breaking trio one for the D rows, an uncommitted
test edit for --dirty, branch renames and a non-git dir for H5–H10, --mutate against
app/Services for the F rows — are fine as written; pestphp/pest-plugin-mutate is installed.
Part 2 — Full case matrix
Tiers: COMPLETE may change everything · RESULTS-ONLY (RO) may change only
baselines[<branch>].results for tests that ran, and must never remove an entry, add a result for a
test file absent from edges, or alter sha/tree/edges/files/fingerprint ·
HARD-SUPPRESSED may change nothing.
Phase-1 column: VERIFIED = re-run against the fixed code in the phase-one session, pre-fix
contrast captured · PASS (sweep) = passed in the original db70017 sweep and not re-checked
since the changes — these are the bulk of phase two's work · SKIP = not runnable as written.
A — Setup & sanity
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| A1 | composer show pestphp/pest |
version dev-fix/tia-filtered |
PASS (sweep) |
| A2 | pest --baseline |
prints an existing dir; exit 0 | VERIFIED |
| A3 | delete graph, pest --tia |
graph.json created; one edges key per test file; one result per test |
VERIFIED |
| A4 | git status after reset ritual |
clean except the four user-modified files (see trap 2) | VERIFIED |
| A5 | extension_loaded("pcov") |
true |
VERIFIED |
| A6 | delete graph, plain pest |
no graph created | PASS (sweep) |
| A7 | delete graph, pest --filter=adds |
no graph created | PASS (sweep) |
| A8 | two consecutive pest --tia |
second replays everything; sha unchanged |
VERIFIED |
B — COMPLETE runs still write
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| B1 | pest --tia (clean) |
replays; graph written; sha = git rev-parse HEAD |
VERIFIED |
| B2 | pest --no-tia |
results refreshed; prune applied; sha/tree/edges unchanged |
PASS (sweep) — recheck under change 9 |
| B3 | pest (plain) |
same as B2 | PASS (sweep) — recheck under change 9 |
| B4 | pest --tia --fresh |
graph purged and rebuilt; edges rebuilt; files repopulated |
VERIFIED |
| B5 | pest --bail, all green |
COMPLETE — graph written, prune applied | PASS (sweep) |
| B6 | edit Calculator.php, pest --tia |
CalculatorTest re-runs, others replay; tree updated |
PASS (sweep) |
| B7 | delete trio three, pest --tia |
its result pruned; trio one/two remain |
PASS (sweep) |
| B8 | delete GreeterTest.php, pest --tia |
result and edges survive (missing-file prune is --fresh-only) |
PASS (sweep) |
| B9 | B8 then pest --tia --fresh |
entry gone; files drops Greeter.php too |
PASS (sweep) |
| B10 | pest --tia twice |
UPDATED TARGET: time differs only for tests that executed; replayed entries keep their recorded time; statuses stable |
VERIFIED |
| B11 | add a test file, pest --tia |
new edges key and new result appear in the same run |
VERIFIED (regression guard for change 9) |
| B12 | pest --tia --coverage |
completes; graph written; coverage report prints | VERIFIED |
C — Selection narrowing → RESULTS-ONLY
All rows: RO invariants hold. "Notice" = TIA does not apply to partial runs — running the selected tests directly.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| C1 | pest --filter="adds numbers" |
RO; no notice | PASS (sweep) |
| C2 | pest --tia --filter="adds numbers" |
RO; notice | PASS (sweep) |
| C3 | pest --filter="trio one" |
RO; trio two/three byte-identical |
PASS (sweep) |
| C4 | pest --filter="trio" |
RO; all three update; nothing else does | PASS (sweep) |
| C5 | pest --exclude-filter="Feature" |
RO; no notice; no prune | PASS (sweep) |
| C6 | pest --tia --exclude-filter="Feature" |
RO; notice | PASS (sweep) |
| C7 | pest --group=smoke |
RO; no notice | PASS (sweep) |
| C8 | pest --tia --group=smoke |
RO; notice | PASS (sweep) |
| C9 | pest --exclude-group=smoke |
RO; no notice; no prune | PASS (sweep) |
| C10 | pest --tia --exclude-group=smoke |
RO; notice | PASS (sweep) |
| C11 | pest tests/Unit |
RO; no notice; Feature entry not pruned | PASS (sweep) |
| C12 | pest --tia tests/Unit |
RO; notice | PASS (sweep) |
| C13 | pest tests/Unit/TrioTest.php |
RO; no notice | PASS (sweep) |
| C14 | pest --tia tests/Unit/TrioTest.php |
RO; notice | PASS (sweep) |
| C15 | pest --testsuite=Unit |
RO; no notice | PASS (sweep) |
| C16 | pest --tia --testsuite=Unit |
RO; notice | PASS (sweep) |
| C17 | pest --tia --exclude-testsuite=Feature |
RO; notice | PASS (sweep) |
| C18 | pest --tia --covers='App\Services\Calculator' |
RO; notice | PASS (sweep) |
| C19 | pest --tia --uses='App\Services\Calculator' |
RO; notice | PASS (vacuous — needs fixture) |
| C20 | pest --tia --test-suffix=Test.php |
RO even though all tests run; edges/files/n unchanged |
PASS (sweep) |
| C21 | pest --dirty with an uncommitted test edit |
RO; no notice | PASS (sweep) |
| C22 | pest --tia --dirty |
RO; notice | PASS (sweep) |
| C23 | pest --tia --todos |
RO; notice | PASS (vacuous) |
| C24 | pest --tia --notes |
RO; notice | PASS (vacuous) |
| C25 | pest --tia --flaky |
RO; notice | PASS (vacuous) |
| C26 | pest --tia --issue=123 |
RO; notice | PASS (vacuous) |
| C27 | pest --tia --pr=1 |
RO; notice | PASS (vacuous) |
| C28 | pest --tia --pull-request=1 |
RO; notice | PASS (vacuous) |
| C29 | pest --tia --shard=1/2 |
RO; notice; no prune | PASS (sweep) |
| C30 | pest --tia --shard=2/2 |
RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
| C31 | ->only() on trio one, pest --tia |
RO; no notice; siblings untouched | PASS (sweep) |
| C32 | ->only() on trio one, plain pest |
RO; no notice | PASS (sweep) |
| C33 | pest --tia --filtered --filter=adds |
RO; notice; filtered yields to narrowing | PASS (sweep) |
| C34 | PEST_TIA=1 pest --filter=adds |
RO; notice | PASS (sweep) |
| C35 | PEST_TIA_FILTERED=1 pest --filter=adds |
RO; notice (regression guard) | PASS (sweep) |
| C36 | pest --filtered --filter=adds |
RO; notice | PASS (sweep) |
| C37 | pest --tia --ticket=X |
RO; notice. --ticket is a real option (src/Plugins/Snapshot.php:83) — it narrows legitimately |
PASS (vacuous) |
| C38 | pest --tia --assignee=X |
RO; notice (src/Plugins/Snapshot.php:81) |
PASS (vacuous) |
| C39 | pest --tia --todo |
RO; notice; matches nothing | PASS (vacuous) |
D — Truncation → RESULTS-ONLY
D1–D5, D11–D13, D16 precondition: trio one broken. D6–D10 run against a green suite so the --stop-on-* trigger is the only narrowing.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| D1 | pest --bail |
RO; trio one = status=7; siblings survive unchanged |
PASS (sweep) |
| D2 | pest --retry |
RO; siblings survive | PASS (sweep) |
| D3 | pest --stop-on-failure |
RO | PASS (sweep) |
| D4 | pest --stop-on-defect |
RO | PASS (sweep) |
| D5 | pest --stop-on-error (no error occurs) |
COMPLETE — nothing stopped it | PASS (sweep) |
| D6 | pest --stop-on-warning |
RO when it fires. Warning stored as status=0, not 6 |
PASS (sweep) |
| D7 | pest --stop-on-risky --disallow-test-output |
RO; status=5 |
PASS (sweep) |
| D8 | pest --stop-on-skipped |
RO; status=1 |
PASS (sweep) |
| D9 | pest --stop-on-incomplete |
RO; status=2 |
PASS (sweep) |
| D10 | pest --stop-on-deprecation |
RO. Deprecation stored as status=0, not 4 |
PASS (sweep) |
| D11 | pest --tia --bail |
RO | PASS (sweep) |
| D12 | pest --bail --filter=trio |
RO (narrowed and truncated) | PASS (sweep) |
| D13 | stopOnFailure="true" in phpunit.xml, plain pest |
RO — caught via stoppedEarly(), not flag matching |
PASS (sweep) |
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
| D15 | pest --bail, last test in run order fails |
RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
| D16 | pest --tia --filtered --bail, broken affected test |
RO | PASS (sweep) |
E — Result merge semantics
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| E1 | cached flaky=7, FLAKY_OK=1 pest --filter=flaky |
flips to status=0; all other results byte-identical |
PASS (sweep) |
| E2 | cached flaky=0, pest --filter=flaky (env unset) |
flips to status=7 |
PASS (sweep) |
| E3 | after E1, pest --tia --filtered clean |
No affected tests found; zero graph delta |
PASS (sweep) |
| E4 | after E2, pest --tia --filtered |
re-runs, from 1 previously unsuccessful test |
PASS (sweep) |
| E5 | any partial run | assertions and time update for the test that ran (it executed, so change 9 does not apply) |
PASS (sweep) |
| E6 | any partial run | message of untouched tests unchanged |
PASS (sweep) |
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
| E8 | pest --filter="trio one" |
siblings keep exact status/time/assertions/message |
PASS (sweep) |
| E9 | partial run of a ->skip()ed test |
status=1 |
PASS (sweep) |
| E10 | partial run of a ->todo() test |
status=1, assertions=0, message="__TODO__" |
PASS (sweep) |
| E11 | partial run of a risky test | status=5 |
PASS (sweep) |
| E12 | any partial run | fingerprint byte-identical |
PASS (sweep) |
| E13 | dataset test, --filter matching one row |
that row updates; other rows survive (the old prune bug) | PASS (sweep) |
| E14 | dataset test, full --tia after deleting a row |
the deleted row is pruned | PASS (sweep) |
F — Guard rails
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| F1 | test absent from edges, pest --filter="brand new" |
no result recorded | VERIFIED |
| F2 | complete pest --tia first, then the same filter |
result is recorded | VERIFIED |
| F3 | pest --mutate --path=app/Services --covered-only |
statuses/edges/sha/tree/files identical; only time may change |
PASS (sweep) |
| F4 | pest --mutate --parallel --path=… --covered-only |
same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
| F5 | weakened test so a mutant survives, pest --mutate |
no status change in the baseline | PASS (sweep) |
| F6 | pest --mutate with the graph deleted |
no graph created | PASS (sweep) |
| F7 | mutation subprocess env | carries PEST_MUTATION_TESTING; --mutate stripped from argv |
PASS (sweep) |
| F8 | grep baseline after --mutate |
no mutation-flavoured messages | PASS (sweep) |
G — Parallel
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| G1 | pest --tia --parallel (clean) |
COMPLETE — graph written; all 7 results present | VERIFIED |
| G2 | pest --tia --parallel --filter=adds |
RO | PASS (sweep) |
| G3 | pest --parallel --bail, trio one broken |
RO; siblings survive | PASS (sweep) |
| G4 | pest --tia --parallel --bail |
RO | PASS (sweep) |
| G5 | pest --tia --filtered --parallel after a source edit |
narrows to affected; writes | VERIFIED (see I5) |
| G6 | pest --tia --parallel |
worker results reach the parent baseline | VERIFIED |
| G7 | ->only() + pest --tia --parallel |
RO | PASS (sweep) |
| G8 | pest --tia --parallel --shard=1/2 |
RO | PASS (sweep) |
| G9 | broken test in one worker, pest --parallel --bail |
whole run is RO, not just that worker's slice | PASS (sweep) |
| G10 | pest --parallel --retry |
InvalidOption; graph untouched |
PASS (sweep) |
| G11 | pest --tia --parallel --fresh |
graph rebuilt | VERIFIED |
| G12 | edges after pest --tia vs pest --tia --parallel --fresh |
equivalent edge sets — files=22, self-edge on all 5 tests, both |
VERIFIED (fixed) — pre-fix: files=4, 0 self-edges |
H — Baseline key / branch resolution
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| H1 | pest --tia on master |
only a master key |
VERIFIED |
| H2 | plain pest on master |
writes master, not main |
PASS (sweep) |
| H3 | pest --bail truncated on master |
no main key minted |
PASS (sweep) |
| H4 | pest --filter=adds on master |
no main key minted |
PASS (sweep) |
| H5 | git branch -m main, full pest --tia |
single main key |
PASS (sweep) |
| H6 | git checkout -b feature/x, pest --tia |
feature/x key appears; reads fall back to the existing baseline |
PASS (sweep) |
| H7 | detached HEAD, pest --tia |
falls back to the main key for both reads and writes; no HEAD key |
PASS (sweep) |
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
| H9 | non-git dir, pest --tia |
MissingDependency — The feature "Tia mode" requires "git". |
PASS (sweep) |
| H10 | non-git dir, plain pest |
runs normally; no baseline dir created | PASS (sweep) |
I — Filtered mode
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| I1 | clean + green, pest --tia --filtered |
No affected tests found; zero graph delta |
PASS (sweep) |
| I2 | edit Calculator.php |
only CalculatorTest runs |
VERIFIED |
| I3 | cached failure, clean tree | re-runs it, from 1 previously unsuccessful test |
PASS (sweep) |
| I4 | I2 with sentinels | tree updated; exactly one entry rewritten; all others retained |
PASS (sweep) |
| I5 | pest --tia --filtered --parallel |
same narrowing as I2 and edges unchanged |
VERIFIED (fixed) — 1 affected test, edges identical |
| I6 | PEST_TIA_FILTERED=1 pest --tia |
identical delta to I4 | PASS (sweep) |
| I7 | pest --tia --filtered tests/Unit |
explicit path wins; filtered off; RO; notice | PASS (sweep) |
| I8 | pest --tia --filtered --coverage-text |
filtered mode disabled by an active coverage report → full suite runs | VERIFIED (fixed) — pre-fix: No affected tests found |
| I9 | cached failure whose test file was deleted | WARN Some cached tests due a re-run could not be located on disk + Running the full suite with replay instead of a filtered run |
VERIFIED (fixed) — pre-fix: No tests found, exit 0 green |
| I10 | pest --tia --filtered with no baseline yet |
records a baseline instead of filtering | PASS (sweep) |
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
| I12 | edit composer.lock |
fingerprint drift → full rebuild, and the reason is printed sequentially: fresh graph (composer.lock changed) |
VERIFIED (fixed) — pre-fix: bare Running in TIA mode. |
J — Interactions & regressions
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| J1 | pest --tia --fresh --filter=adds |
graph not purged; RO | PASS (sweep) |
| J2 | pest --tia --refetch --filter=adds |
refetch not performed; RO | PASS (sweep) |
| J3 | pest --no-tia --filter=adds |
RO — narrowing wins over --no-tia; no notice |
PASS (sweep) |
| J4 | pest --tia --no-tia |
TIA disabled; COMPLETE | PASS (sweep) |
| J5 | <groups><exclude><group>integration</group></exclude></groups>, pest --tia |
COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
| J6 | pest --tia --compact |
COMPLETE | PASS (sweep) |
| J7 | pest --tia -v |
COMPLETE | PASS (sweep) |
| J8 | pest --tia --profile |
COMPLETE | PASS (sweep) |
| J9 | pest --tia --order-by=random |
COMPLETE | PASS (sweep) |
| J10 | pest --tia --random-order-seed=1234 |
COMPLETE | PASS (sweep) |
| J11 | pest --tia --repeat=2 |
COMPLETE — repetition is not narrowing | SKIP — --repeat is not a Pest option |
| J12 | pest --filter=adds twice |
only the executed test's time differs; rest byte-identical |
PASS (sweep) |
| J13 | pest --tia --min=50 |
COMPLETE (silent no-op without --coverage) |
PASS (sweep) |
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
| J15 | delete the graph, pest --tia --filtered |
records a baseline rather than erroring | PASS (sweep) |
| J16 | corrupt graph.json, pest --tia |
recovers by rebuilding; no crash | PASS (sweep) |
K — New rows for the phase-one fixes
These assert behaviour no original row covered. All were verified in phase one; re-run them as regression guards.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| K1 | healthy pest --tia graph, then pest --tia --coverage |
edges byte-identical — piggyback data may seed empty sets, never narrow populated ones; files=22 stays 22 |
VERIFIED (fixed) — pre-fix: ExampleTest 16→2 edges, self-edges lost |
| K2 | same run's headline | Experimental TIA mode enabled / recording a coverage baseline. — no false fresh graph |
VERIFIED (fixed) |
| K3 | pest --tia ×3 on a clean tree |
replayed entries keep their recorded time across all three |
VERIFIED (fixed) — pre-fix: 0.053 → 0.001 → 0.001 → 0.001 |
| K4 | add a test file, one pest --tia (graph exists, fingerprint matches → replay+refresh) |
result and edges appear in that same run; n grows by 1 |
VERIFIED — regression guard for change 9 |
| K5 | add a test file, plain pest --no-tia |
no result written for it; n unchanged; no edges |
VERIFIED (fixed) — pre-fix: edge-less result inflated n |
| K6 | pest --tia --parallel worker argv |
carries -d pcov.directory=<projectRoot>; PEST_TIA unset; not re-exec'd |
VERIFIED — probe bin/worker.php |
| K7 | second pest --tia --coverage (cache primed → replay) |
recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — new work |
| K8 | pest --tia --parallel --coverage |
workers read TIA_PIGGYBACK_COVERAGE; no widened pcov scope; report intact |
Not yet measured — new work |
| K9 | pest --tia --filtered --coverage-html=<dir> / --coverage-clover=<file> |
filtered mode off, same as I8, for every flag in COVERAGE_REPORT_FLAGS |
Not yet measured — new work |
Part 3 — Priorities for phase two
- Part 1b fixtures — nothing in C, D6–D10, or E9–E14 means anything until they exist.
- K7, K8, K9 — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
reasoned about but not observed; K9 covers the seven
COVERAGE_REPORT_FLAGSbeyond--coverage-text. - B2, B3, F4 and all of D and E — change 9 touched the shared result-write path, and
these are the rows that exercise it hardest.
$recordsEdgesis the thing to falsify: it must be false for every partial and every non-recording run. - F3–F8 — change 7 injects a
-dinto worker argv;--mutate --parallelis the one place that both spawns workers and must write nothing. - H1–H10 — untouched by these changes; cheapest bulk confirmation.
- Raw PHPUnit coverage flags print no report at all in Pest, with or without TIA
(
--no-tia --coverage-textis equally silent). Pre-existing, unrelated to change 6 — do not chase it as a regression, but it means I8/K9 can only assert the selection half.
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
failure the pre-fix contrast (write git show db70017c:<file> into vendor, re-run, restore) so a
regression is told apart from a pre-existing defect.