Compare commits

...

424 Commits

Author SHA1 Message Date
nuno maduro 7bfb2a6185 chore: upgrade snapshots 2026-08-06 22:24:26 +01:00
nuno maduro ca56ca5216 dw 2026-08-06 22:19:47 +01:00
nuno maduro b49ba062d5 wq 2026-08-06 22:05:15 +01:00
nuno maduro 09699847a2 wqd 2026-08-06 20:53:09 +01:00
nuno maduro 545d0c2784 wip 2026-08-06 20:48:22 +01:00
nuno maduro b795af3b1b wip 2026-08-06 17:52:04 +01:00
nuno maduro a40cc6bc0e more tests 2026-08-06 16:35:03 +01:00
nuno maduro 4d3d0105b7 chore: style 2026-08-06 16:02:36 +01:00
nuno maduro e90e4f70fc wip 2026-08-05 20:20:35 +01:00
nuno maduro d112582857 wip 2026-08-05 19:58:57 +01:00
nuno maduro 9f3c4e1e82 fix 2026-08-05 18:51:49 +01:00
nuno maduro 71f39366c2 wip 2026-08-05 18:21:34 +01:00
nuno maduro bfd5b75677 fix 2026-08-05 16:49:35 +01:00
nuno maduro 411b9954b5 wip 2026-08-05 16:18:03 +01:00
nuno maduro db70017cb2 wip 2026-08-05 03:30:50 +01:00
nuno maduro 86adaedbbd fix: tia filtered 2026-08-04 23:30:03 +01:00
nuno maduro 92c7677c6e fix: don't allow invalid class names 2026-08-04 20:46:21 +01:00
nuno maduro 872f0a50c2 fix(tia): replayable scenarios running afterEach 2026-08-04 20:19:12 +01:00
nuno maduro 668809bc75 chore: adds missing test 2026-08-04 19:28:58 +01:00
nuno maduro 8d8f45c843 chore: bump deps 2026-08-04 19:28:52 +01:00
nuno maduro 086b3e9107 chore: ai instructions 2026-08-04 19:01:16 +01:00
nuno maduro 43fe26f324 chore: bumps deps 2026-08-04 19:01:08 +01:00
nuno maduro 865c1e5113 ci: snapshots 2026-08-04 18:51:04 +01:00
stephanbouman 19eed8d581 Fixed typos (#1819)
* Fixed typo in message

* Fixed typo in message
2026-08-03 11:29:03 -04:00
Danny de Wit ad1850b110 fix: keep TableExtractor table names as strings (#1794)
PHP coerces numeric-string array keys to ints, so collecting table names in
$tables[$name] and returning array_keys() can yield ints. That breaks the
declared list<string> and throws a TypeError in Recorder::linkTable(string).

Standard SQL like substring(x FROM 1 FOR 10) is enough to trigger it, since
the numeric operand matches the FROM pattern.

Closes #1793

Co-authored-by: Danny de Wit <d.dewit@fenceweb.com>
2026-08-02 22:50:47 -04:00
nuno maduro 585de259a2 chore: style 2026-08-02 22:26:34 -04:00
nuno maduro 953664dce0 release: v5.0.3 2026-08-02 22:24:32 -04:00
nuno maduro 1dd959848c chore: updates snapshots 2026-08-02 22:24:24 -04:00
Jeffrey van Hees 047753c836 fix: key Tia results per dataset row (#1799)
The result cache was keyed `Class::method` on both sides, so every row of a
`->with()` test shared one entry and the last writer won. On replay that single
status was handed to every row.

`TestMethod::id()` already appends `#dataSetName`, and `TestCase` reaches the
same value object through `valueObjectForEvents()`, so both sides can use it.
2026-08-02 21:43:15 -04:00
nuno maduro b63626a94d docs: adjusts readme
Signed-off-by: nuno maduro <enunomaduro@gmail.com>
2026-07-30 11:53:41 -04:00
nuno maduro ae080becd6 chore: style 2026-07-29 15:27:11 -04:00
nuno maduro aa86304794 release: v5.0.2 2026-07-29 14:59:13 -04:00
nuno maduro 75963ea973 chore: bumps deps 2026-07-29 14:59:03 -04:00
ohnotnow 5d0ceb1a5d Check for pcov being enabled for tia (#1779)
* Check for pcov being enabled for tia

* Delete tests/Features/TiaDriverGate.php

Signed-off-by: nuno maduro <enunomaduro@gmail.com>

---------

Signed-off-by: nuno maduro <enunomaduro@gmail.com>
Co-authored-by: Ohffs <ohffsnotnow@gmail.com>
Co-authored-by: nuno maduro <enunomaduro@gmail.com>
2026-07-29 14:28:38 -04:00
talaridisTh 6f0c0536a4 feat: document the tia options in pest --help (#1780) 2026-07-29 14:11:17 -04:00
Pushpak Chhajed 6b2cd358e8 Add Laravel Boost skill for Pest 5 testing (#1775) 2026-07-28 13:28:46 -04:00
Than Tibbetts e09e1be76d Update Pest v5 announcement link in README (#1773)
Signed-off-by: Than Tibbetts <than@me.com>
2026-07-28 12:21:33 -04:00
dependabot[bot] 759ac63139 build(deps): bump actions/checkout in the github-actions group (#1768)
Bumps the github-actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0...3d3c42e5aac5ba805825da76410c181273ba90b1)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-24 01:48:21 +01:00
nuno maduro 333f469efb adjustments
Signed-off-by: nuno maduro <enunomaduro@gmail.com>
2026-07-23 00:25:58 +01:00
nuno maduro c4ec925aa5 Revise Pest description in README.md
Updated the description of Pest to include AI agents.

Signed-off-by: nuno maduro <enunomaduro@gmail.com>
2026-07-22 22:15:30 +01:00
nuno maduro 5c8cfed483 docs: sync sponsors with pestphp.com
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 21:58:54 +01:00
nuno maduro 962510b7e3 chore: adjusts snapshots 2026-07-21 01:14:59 +01:00
Punyapal Shah e9c268a584 feat: validation expectations (#1762)
* feat: add validation expectations for IP, MAC, hostname, domain, base64, and hexadecimal

* improvements

* improvements
2026-07-20 11:21:57 +01:00
nuno maduro fa91fcf744 fix: coverage merger 2026-07-19 16:45:06 +01:00
nuno maduro c6073511a2 chore: fixes test printer 2026-07-19 13:59:31 +01:00
nuno maduro b5e1b83bb4 fix: cache results on parallel 2026-07-19 12:29:33 +01:00
nuno maduro 1c11595504 chore: moves to coding stlye 2026-07-19 03:41:57 +01:00
nuno maduro cd0e921158 chore: style 2026-07-19 00:42:29 +01:00
nuno maduro f8c4c061f8 fixes snapshots 2026-07-18 03:12:47 +01:00
nuno maduro 81eacd79fd fix: tia engine correctness issues
- keep table/Inertia/database link-tracking alive in piggyback-coverage
  recording, and merge link-only edges into the piggybacked edge set
- route migration changes through the watch-pattern fallback when the
  graph has no recorded table usage
- re-run cached failures whose test file cannot be located, downgrading
  filtered runs to full replay instead of silently skipping them
- align non-filtered replay with the failOn*/displayDetailsOn* rerun policy
- resolve anonymous index Blade components (<x-card> from card/index.blade.php)
  and stop swallowing components whose static usage walk selected no tests
- parse CTE/REPLACE queries and schema-qualified identifiers in TableExtractor
- scope-filter the xdebug recording path like pcov
- ignore in-flight .tmp files in FileState::keysWithPrefix and delete
  unreadable worker partials
- vite deps helper: resolve rolldown through rolldown-vite installs and
  degrade gracefully, honour vite.config resolve.alias (incl. the
  laravel-vite-plugin '@' default), resolve .vue/.svelte/.mts/.cts
  extensions, and stop memoizing cycle-tainted transitive sets

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 02:50:20 +01:00
nuno maduro 1ef680c75d fix: tia 2026-07-18 02:28:29 +01:00
nuno maduro 820fa08313 fix: package lock fingerprint 2026-07-18 01:10:01 +01:00
nuno maduro 9e79e491e2 Merge branch '4.x' into 5.x 2026-07-17 16:38:01 +01:00
nuno maduro 1b83b8fcda release: v4.7.6 2026-07-17 16:33:29 +01:00
nuno maduro ff4aba2ffa chore: updates snapshots 2026-07-17 14:32:33 +01:00
nuno maduro feaac1ab7e chore: fixes rector ignore 2026-07-17 14:32:27 +01:00
nuno maduro bbc6ada21e chore: bumps deps 2026-07-17 14:32:17 +01:00
Sonali dudhia 3927dbfcdf fix: report dataset provider errors as failing tests (#1749)
Errors raised while resolving a test's dataset (missing named dataset,
throwing dataset closure) previously crashed the whole run. Now the data
provider catches them, wraps them in a DatasetProviderError, and the test
method rethrows the original throwable so the affected test fails cleanly
with a non-zero exit code while other tests keep running.
2026-07-17 14:13:01 +01:00
nuno maduro b52bbb4cca chore: snapshots 2026-07-14 15:11:20 +01:00
nuno maduro 52819501eb chore: fixes tia when not running on root 2026-07-14 14:34:00 +01:00
nuno maduro 7b18287374 feat: evals configuration 2026-07-07 19:10:35 +01:00
nuno maduro 33e3bf2888 chore: moves to phpunit 13.2 2026-07-07 13:40:02 +01:00
nuno maduro 72ab506d7b Merge branch '4.x' into 5.x 2026-07-06 18:47:25 +01:00
nuno maduro 5dc49a71d6 release: v4.7.5 2026-07-06 18:06:29 +01:00
nuno maduro 1adb484d3f chore: bumps deps 2026-07-06 18:05:35 +01:00
nuno maduro aadf7e92a6 fix: escape generated test case filename (#1746)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:18:26 +01:00
nuno maduro df42b9ed7b chore: style 2026-07-06 13:17:11 +01:00
dependabot[bot] 632ae79c8b build(deps): bump actions/cache in the github-actions group (#1745)
Bumps the github-actions group with 1 update: [actions/cache](https://github.com/actions/cache).


Updates `actions/cache` from 5.0.5 to 6.1.0
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/27d5ce7f107fe9357f9df03efb73ab90386fccae...55cc8345863c7cc4c66a329aec7e433d2d1c52a9)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 12:57:08 +01:00
nuno maduro b5fed82883 chore: style 2026-06-25 20:24:28 +01:00
nuno maduro 145e20db75 chore: merges 4.x 2026-06-25 20:20:20 +01:00
nuno maduro ee2e97e932 release: v4.7.4 2026-06-25 20:09:05 +01:00
nuno maduro b561236325 chore: bumps phpunit 2026-06-25 20:09:05 +01:00
dependabot[bot] e685179b7e Bump actions/checkout from 6.0.3 to 7.0.0 in the github-actions group (#1737)
Bumps the github-actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 6.0.3 to 7.0.0
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/df4cb1c069e1874edd31b4311f1884172cec0e10...9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-25 20:03:39 +01:00
Conor Murphy ac8feafdcc fix: guard Tia argv scanning against non-string args (#1736)
The Tia plugin scans the raw argv with str_starts_with() in
argumentPresent() and hasExplicitPathArgument(). In the parallel worker
path (bin/worker.php) the unserialized argv can contain an integer
--random-order-seed value as a separate element, which made
str_starts_with() throw:

    TypeError: str_starts_with(): Argument #1 ($haystack) must be of
    type string, int given

This is the same class of bug as #1206, which was only fixed in
HandleArguments; it resurfaced in the newer Tia plugin. Cast each argv
element to string before scanning, mirroring the #1206 fix.

Adds a regression test, and updates the parallel count assertion and the
success snapshot to account for it.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 20:03:06 +01:00
Sonali dudhia b49cd150c9 feat: add toBeEmail expectation (#1735)
Adds `toBeEmail()` backed by `filter_var(FILTER_VALIDATE_EMAIL)`.
Mirrors the existing `toBeUrl()` pattern — no new dependencies, pure PHP.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 15:51:35 +01:00
Norby Baruani 5cfb4133bf [5.x] Make time-based sharding namespace-agnostic and forward --test-directory (#1677)
* refactor(shard): extract parseListTestsOutput for testability

* test(shard): characterize parseListTestsOutput current behavior

* fix(shard): parse any PHP FQCN namespace from --list-tests

* fix(shard): forward --test-directory to list-tests subprocess

* chore: lint + snapshot fixups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shard): make extracted helpers private, test via reflection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* revert visual_snapshot_of_help_command_output

* revert visual_snapshot_of_help_command_output

* keep function removeParallelArguments

* strip --processes argument when building list-tests command

The removeParallelArguments method was not filtering --processes flags, causing the list-tests subprocess to fail when parallel execution was enabled. This prevented time-based sharding from working correctly with the --parallel option.

Now both --parallel/-p and --processes arguments are removed from the command used to enumerate tests, ensuring the subprocess runs successfully.

* test: re-add namespace-agnostic sharding tests

- 5.x merge kept describe()-style test file, dropped PR #1677 tests for parseListTestsOutput + buildListTestsCommand. Re-add them in matching style.
- Also fix removeParallelArguments test broken by merge: source array_values() + strips --processes, so expects ['bin/pest','tests/'].

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-14 13:09:30 +01:00
nuno maduro 3c8bae5f05 chore: bumps dependencies 2026-06-12 20:30:22 +01:00
nuno maduro b2998bc69e chore: updates snapshots 2026-06-12 20:30:16 +01:00
nuno maduro 3876093cd2 fix: missing array values 2026-06-12 20:22:47 +01:00
Moshe Brodsky 932f8bcc07 consistent sharding logic when no shards file (#1710) 2026-06-12 20:19:30 +01:00
oddvalue d393799d2a Optimize buildFilterArgument in Shard plugin for compact regex generation and add comprehensive tests (#1675) 2026-06-12 20:06:50 +01:00
nuno maduro 0d7814ca16 chore: update snapshots 2026-06-12 18:02:06 +01:00
flap152 8467c64c22 fix: popArgument drops duplicate arguments breaking --parallel --exclude-gropup= (#1674)
* fix: popArgument drops duplicate arguments breaking --parallel multi-exclude-group

Fixes #1437

* fix: ensure popArgument handles duplicate arguments

* fix: update expected test results and snapshots after rebase

---------

Signed-off-by: nuno maduro <enunomaduro@gmail.com>
Co-authored-by: nuno maduro <enunomaduro@gmail.com>
2026-06-12 17:58:37 +01:00
oddvalue 97714a7088 Enforce filter length validation and add tests for Shard plugin (#1673) 2026-06-12 17:56:32 +01:00
nuno maduro afb582616d chore: points to 5.x 2026-06-12 07:29:53 +01:00
nuno maduro 15e9b6a507 chore: snapshots 2026-06-12 07:22:13 +01:00
nuno maduro 520ce29376 Merge branch '4.x' into 5.x 2026-06-12 07:22:01 +01:00
nuno maduro 87882a8561 fix: exit code 2026-06-12 06:57:27 +01:00
nuno maduro 77ef7e0df1 ci: bumps setup-php pinned commit 2026-06-12 06:51:58 +01:00
dependabot[bot] 12100dd901 Bump actions/checkout from 6.0.2 to 6.0.3 in the github-actions group (#1730)
Bumps the github-actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 6.0.2 to 6.0.3
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/de0fac2e4500dabe0009e67214ff5f5447ce83dd...df4cb1c069e1874edd31b4311f1884172cec0e10)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-12 06:51:20 +01:00
nuno maduro 20c12d006e docs: fixes alt 2026-06-12 06:32:20 +01:00
nuno maduro 3aab02d2eb ci: fixes dd test 2026-06-11 10:50:24 +01:00
nuno maduro 553aac65e6 release: v4.7.3 2026-06-11 10:40:18 +01:00
Punyapal Shah be49a3ce18 Fix: dd (#1692)
* fix: update dd method to return never type

Co-authored-by: Copilot <copilot@github.com>

* fix: enhance var_dump calls to accept additional arguments

* fix: update dd method to handle paratest and collision printer environments

* test: add dd method tests for ExpectationFailedException in parallel mode

Co-authored-by: Copilot <copilot@github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
2026-06-11 10:16:07 +01:00
Sonali dudhia 774a340400 feat: add toBeUlid assertion and isUlid validation method (#1726) 2026-06-11 10:14:00 +01:00
Chetan 04c9d41895 fix: update gitlab urls for issues and prs to match new format (#1728)
* fix: update gitlab urls for issues and prs to match new format

Added an optional host parameter to the gitlab method, defaulting to 'gitlab.com'. Updated the issues and prs URLs to include '/-/' in the path, which is the new format for GitLab URLs. This change ensures that the URLs generated for GitLab projects are correct and reflect the new structure of GitLab's URLs for issues and merge requests.

Host parameter allows users to specify a custom GitLab instance if they are using a self-hosted version of GitLab, while still maintaining the default behavior for users who are using gitlab.com.

* fix: update gitlab method to use hostname parameter correctly
2026-06-11 10:13:26 +01:00
nuno maduro dfb7b870af fix: retry output expected string 2026-06-11 10:12:41 +01:00
nuno maduro 37821e1108 chore: fixes output snapshots on team city 2026-06-11 10:12:32 +01:00
nuno maduro cd711a25d5 chore: bumps phpunit 2026-06-11 10:08:36 +01:00
nuno maduro eee60c9e11 fix: duplicated team city output 2026-06-11 10:08:27 +01:00
jp c40c8dbc24 [4.x] Fix Checks (#1709)
* fix(composer): set root version for feature-branch CI

pest-plugin-browser ^4.3.1 requires pestphp/pest ^4.4.5 on the root package.
Without an explicit version, Composer infers dev-<branch> on PR branches,
which fails composer update before tests run.

* set root version of composer

* fix indent
2026-06-06 01:23:08 +01:00
dependabot[bot] 2d93c9c373 Bump shivammathur/setup-php in the github-actions group (#1707)
Bumps the github-actions group with 1 update: [shivammathur/setup-php](https://github.com/shivammathur/setup-php).


Updates `shivammathur/setup-php` from 2.37.0 to 2.37.1
- [Release notes](https://github.com/shivammathur/setup-php/releases)
- [Commits](https://github.com/shivammathur/setup-php/compare/accd6127cb78bee3e8082180cb391013d204ef9f...7c071dfe9dc99bdf297fa79cb49ea005b9fcadbc)

---
updated-dependencies:
- dependency-name: shivammathur/setup-php
  dependency-version: 2.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-04 17:55:17 +01:00
dependabot[bot] 3d5bba93f8 Bump shivammathur/setup-php in the github-actions group (#1706)
Bumps the github-actions group with 1 update: [shivammathur/setup-php](https://github.com/shivammathur/setup-php).


Updates `shivammathur/setup-php` from 2.37.0 to 2.37.1
- [Release notes](https://github.com/shivammathur/setup-php/releases)
- [Commits](https://github.com/shivammathur/setup-php/compare/accd6127cb78bee3e8082180cb391013d204ef9f...7c071dfe9dc99bdf297fa79cb49ea005b9fcadbc)

---
updated-dependencies:
- dependency-name: shivammathur/setup-php
  dependency-version: 2.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-04 17:55:13 +01:00
nuno maduro e5ab3af05b ci: update dependabot config (add cooldown, single update entry) 2026-06-04 17:54:45 +01:00
nuno maduro 79bc7a8257 Merge branch '4.x' into 5.x 2026-06-01 07:09:57 +01:00
nuno maduro 40b88b62ef release: v4.7.2 2026-06-01 07:08:59 +01:00
nuno maduro e3361bc321 fix: don't ob_start with pao 2026-06-01 07:08:50 +01:00
nuno maduro fc48c1bd1e Merge branch '4.x' into 5.x 2026-06-01 06:33:35 +01:00
nuno maduro 92e76eb5ab ci: runs ci only against stable 2026-06-01 06:32:29 +01:00
nuno maduro da726beffc chore: merges 4.x 2026-06-01 06:28:44 +01:00
nuno maduro 4ef12b9aac Merge branch '4.x' into 5.x 2026-06-01 06:25:56 +01:00
nuno maduro bd22f478b8 chore: fixes issues with contracts and symfony 8.1 2026-06-01 06:24:42 +01:00
nuno maduro eeaac34cf6 release: v4.7.1 2026-06-01 05:42:12 +01:00
nuno maduro b9b07d8983 chore: bump dependencies 2026-06-01 05:42:03 +01:00
nuno maduro 6aa7d2f891 fix: better fatal exceptions reporting 2026-06-01 05:41:58 +01:00
nuno maduro 4d550cecfd Merge branch '4.x' into 5.x 2026-05-13 12:20:46 +01:00
nuno maduro 1c21a7647a chore: fixes types 2026-05-13 12:20:00 +01:00
nuno maduro d649de1988 chore: add security policy 2026-05-12 02:48:25 +01:00
nuno maduro 783ca4bcd6 chore(deps): limit dependabot to maintained branches (4.x + 5.x) 2026-05-12 02:34:08 +01:00
nuno maduro ba07497219 chore: enable Dependabot version updates for GitHub Actions (#1700) 2026-05-11 22:12:07 -03:00
nuno maduro 34695843b3 chore: pin GitHub Actions to commit SHAs (#1699)
* chore: pin GitHub Actions to commit SHAs

* chore: pin GitHub Actions to commit SHAs
2026-05-11 22:12:04 -03:00
nuno maduro 1ca021dea6 chore: pin GitHub Actions to commit SHAs (#1695)
* chore: pin GitHub Actions to commit SHAs

* chore: pin GitHub Actions to commit SHAs
2026-05-12 02:08:47 +01:00
nuno maduro d17be9decd types 2026-05-04 08:02:09 -03:00
nuno maduro b828ddcec7 chore: style 2026-05-04 07:38:50 -03:00
nuno maduro f859bb179d Merge branch '4.x' into 5.x 2026-05-04 07:38:40 -03:00
nuno maduro 2fc75cfcf0 chore: updates snapshots 2026-05-03 13:09:32 -03:00
nuno maduro 6cc48f63f8 chore: style 2026-05-03 13:06:24 -03:00
nuno maduro e0419d1328 release: v4.7.0 2026-05-03 12:46:24 -03:00
nuno maduro faa6988801 Merge pull request #1682 from pestphp/feat/tia
[4.x] TIA Engine
2026-05-03 16:27:58 +01:00
nuno maduro c12247fafd Revert "wip"
This reverts commit 1b168aba1c.
2026-05-03 11:45:39 -03:00
nuno maduro 29b4452443 wip 2026-05-03 11:33:30 -03:00
nuno maduro 1b168aba1c wip 2026-05-03 11:30:13 -03:00
nuno maduro 6aabd977cd wip 2026-05-03 11:01:52 -03:00
nuno maduro a882543c53 wip 2026-05-03 10:57:28 -03:00
nuno maduro c250b9da4f wip 2026-05-03 10:37:17 -03:00
nuno maduro 46bc3dc628 wip 2026-05-03 10:34:44 -03:00
nuno maduro d3ce498b8a wip 2026-05-03 10:31:47 -03:00
nuno maduro e1a4b98b71 wip 2026-05-03 10:16:10 -03:00
nuno maduro 9afbcd5c18 wuip 2026-05-03 09:54:02 -03:00
nuno maduro 75593b6454 wip 2026-05-03 13:37:27 +01:00
nuno maduro 89590d6120 wip 2026-05-03 13:35:01 +01:00
nuno maduro fb0978c9bf wip 2026-05-03 13:26:48 +01:00
nuno maduro a3796daa42 wip 2026-05-03 13:24:10 +01:00
nuno maduro e3004db666 wip 2026-05-03 12:43:38 +01:00
nuno maduro 99cc4e0146 wip 2026-05-02 19:33:09 +01:00
nuno maduro a47e6f8fef wip 2026-05-02 19:30:14 +01:00
nuno maduro 536d79f765 wip 2026-05-02 19:20:55 +01:00
nuno maduro 65c0fbc528 wip 2026-05-02 19:07:41 +01:00
nuno maduro 9e4cf4b665 wip 2026-05-02 18:58:42 +01:00
nuno maduro 7bea819978 wip 2026-05-02 18:47:26 +01:00
nuno maduro 4280233b40 wip 2026-05-02 18:37:24 +01:00
nuno maduro d6db3a8a20 wip 2026-05-02 18:32:05 +01:00
nuno maduro 51c8ce4df6 wip 2026-05-02 18:31:32 +01:00
nuno maduro 5b8393b925 wip 2026-05-02 18:25:41 +01:00
nuno maduro e4d9b61fdf wip 2026-05-02 18:25:27 +01:00
nuno maduro e2d940cd53 wip 2026-05-02 18:25:21 +01:00
nuno maduro 380ccd30b4 wip 2026-05-02 18:03:25 +01:00
nuno maduro 31c200716d wip 2026-05-02 18:03:14 +01:00
nuno maduro 6add4da543 wip 2026-05-02 18:02:20 +01:00
nuno maduro 8ddcd3e853 wip 2026-05-02 18:02:13 +01:00
nuno maduro e3e178fd94 wip 2026-05-02 17:59:21 +01:00
nuno maduro 7b1ec9f003 wip 2026-05-02 17:59:13 +01:00
nuno maduro 1e48c5d473 wip 2026-05-02 17:59:00 +01:00
nuno maduro d00ec95dd9 wip 2026-05-02 17:58:55 +01:00
nuno maduro 89f3d6cb39 wip 2026-05-02 17:45:54 +01:00
nuno maduro a07a2e512a wip 2026-05-02 17:39:15 +01:00
nuno maduro 57eecb2b3d wip 2026-05-02 17:38:12 +01:00
nuno maduro 9f804dc954 wip 2026-05-02 17:38:08 +01:00
nuno maduro 7cbad4c589 wip 2026-05-02 17:38:01 +01:00
nuno maduro 5cae93b059 wip 2026-05-02 17:37:56 +01:00
nuno maduro df829ad19d wip 2026-05-02 17:37:47 +01:00
nuno maduro 635460653c wip 2026-05-02 17:37:34 +01:00
nuno maduro 1aa80dc398 wip 2026-05-02 17:18:35 +01:00
nuno maduro 8a14056111 wip 2026-05-02 17:15:46 +01:00
nuno maduro f247dd8e7b wip 2026-05-02 17:11:49 +01:00
nuno maduro 1c7c9754fd wip 2026-05-02 17:07:08 +01:00
nuno maduro 5f37939fda wip 2026-05-02 17:02:11 +01:00
nuno maduro 28305fcb7a wip 2026-05-02 16:35:52 +01:00
nuno maduro 5242803694 wip 2026-05-02 15:54:00 +01:00
nuno maduro 925935a7e8 wip 2026-05-02 15:33:38 +01:00
nuno maduro 460401c379 wip 2026-05-02 15:26:58 +01:00
nuno maduro 348b439172 wip 2026-05-02 15:15:53 +01:00
nuno maduro a4e77766c5 wip 2026-05-02 15:07:51 +01:00
nuno maduro 4a8c2d7d78 wip 2026-05-02 15:03:44 +01:00
nuno maduro 7d51601120 wip 2026-05-02 14:15:37 +01:00
nuno maduro 631bbe318b wip 2026-05-02 13:43:32 +01:00
nuno maduro 9b7c15d5b6 wip 2026-05-02 12:03:35 +01:00
nuno maduro 872796bd9b wip 2026-05-02 12:00:47 +01:00
nuno maduro c38d32ae86 wip 2026-05-02 09:49:33 +01:00
nuno maduro 6407c4f78f wip 2026-05-02 01:58:39 +01:00
nuno maduro 6e1bf63f6a wip 2026-05-02 01:40:35 +01:00
nuno maduro 1d3e8bb5dd wip 2026-05-02 01:03:06 +01:00
nuno maduro 3cc9b169e3 wip 2026-05-02 00:52:57 +01:00
nuno maduro c4911d046b wip 2026-05-02 00:06:04 +01:00
nuno maduro d0295f6168 wip 2026-05-01 23:59:25 +01:00
nuno maduro 21efbc3107 wip 2026-05-01 22:55:38 +01:00
nuno maduro e59b99cd73 wip 2026-05-01 22:51:55 +01:00
nuno maduro bf48e20880 wip 2026-05-01 22:36:15 +01:00
nuno maduro 53db68e005 wip 2026-05-01 22:31:00 +01:00
nuno maduro 34f1e9a7f2 fix 2026-05-01 21:51:09 +01:00
nuno maduro 57fd5ce042 wip 2026-05-01 21:50:56 +01:00
nuno maduro 3bcabfb63b fix 2026-05-01 21:50:52 +01:00
nuno maduro aa3a7c303a wip 2026-05-01 21:32:59 +01:00
nuno maduro 5c08a135f7 wip 2026-05-01 21:30:44 +01:00
nuno maduro 6e0e030d71 wip 2026-05-01 21:22:33 +01:00
nuno maduro b2c07561e7 wip 2026-05-01 20:54:24 +01:00
nuno maduro 97600b6f0b wip 2026-05-01 20:53:40 +01:00
nuno maduro 8a51f15d65 wip 2026-05-01 20:45:51 +01:00
nuno maduro a349f53964 wip 2026-05-01 20:42:14 +01:00
nuno maduro a725e774c0 wip 2026-05-01 20:28:39 +01:00
nuno maduro bed5e5b54a wip 2026-05-01 20:02:46 +01:00
nuno maduro 45b1d4ce20 wip 2026-05-01 19:50:54 +01:00
nuno maduro d106b70766 wip 2026-05-01 17:24:22 +01:00
nuno maduro 6ac6c1518e wip 2026-05-01 17:17:33 +01:00
nuno maduro fda515a17f wip 2026-05-01 16:42:01 +01:00
nuno maduro 0a97d3a288 asd 2026-05-01 15:33:06 +01:00
nuno maduro 3802fa80e6 asd 2026-05-01 15:19:19 +01:00
nuno maduro 5c3cbc14d2 wip 2026-05-01 15:07:10 +01:00
nuno maduro 6b9c768172 wip 2026-05-01 14:39:23 +01:00
nuno maduro 4a2fc179ae asd 2026-05-01 13:54:25 +01:00
nuno maduro b5bb2139dc wqdqwd 2026-05-01 12:57:12 +01:00
nuno maduro 07416a3c61 wip 2026-05-01 03:30:28 +01:00
nuno maduro 30b94e3034 qdw 2026-05-01 02:10:08 +01:00
nuno maduro be34eecb2f wip 2026-05-01 01:55:18 +01:00
nuno maduro 5d9f95f8d4 qwdqwd 2026-05-01 01:44:08 +01:00
nuno maduro 48b70a03d5 wip 2026-05-01 01:32:48 +01:00
nuno maduro 4b8642b972 wip 2026-05-01 00:48:31 +01:00
nuno maduro 8711d51eac fix 2026-05-01 00:19:44 +01:00
nuno maduro 58dfb6da64 wip 2026-04-30 22:12:53 +01:00
nuno maduro d7735d1faa wip 2026-04-30 22:00:56 +01:00
nuno maduro 6b59166f3c wip 2026-04-30 21:08:00 +01:00
nuno maduro 3a26028d17 wip 2026-04-30 20:58:06 +01:00
nuno maduro 3c91bf4ad2 wip 2026-04-30 20:51:57 +01:00
nuno maduro 6a434be0f6 wip 2026-04-30 20:45:36 +01:00
nuno maduro f355b99bbf wip 2026-04-29 22:59:56 +01:00
nuno maduro 95a00341e9 wip for now 2026-04-29 02:22:37 +01:00
nuno maduro 466259646d wip 2026-04-28 22:12:42 +01:00
nuno maduro 00f8d56083 wip 2026-04-28 21:41:20 +01:00
nuno maduro ca2dca592d wup 2026-04-28 21:34:40 +01:00
nuno maduro 405d8d4406 wip 2026-04-28 21:28:46 +01:00
nuno maduro b944ee5841 wip 2026-04-27 19:15:42 +01:00
nuno maduro f4e22dcafe wip 2026-04-27 18:57:41 +01:00
nuno maduro 339c1e8cac wip 2026-04-27 18:14:10 +01:00
nuno maduro d4c7362132 wip 2026-04-27 16:56:27 +01:00
nuno maduro 81bfdbf8fe wip 2026-04-27 13:16:05 +01:00
nuno maduro f45cbf43c5 wip 2026-04-27 13:11:48 +01:00
nuno maduro b9088d23fb wip 2026-04-27 13:03:07 +01:00
nuno maduro 7250185423 wip 2026-04-27 12:22:05 +01:00
nuno maduro e457eb0e9c wip 2026-04-27 11:15:59 +01:00
nuno maduro 48357c6f30 wip 2026-04-27 10:30:08 +01:00
nuno maduro b46f051550 wip 2026-04-23 17:32:27 -07:00
nuno maduro 3d3c5d41ac wip 2026-04-23 12:29:24 -07:00
nuno maduro caabebf2a1 wip 2026-04-23 10:56:17 -07:00
nuno maduro 470a5833d4 wip 2026-04-23 10:30:44 -07:00
nuno maduro c1feefbb9e wip 2026-04-23 09:44:12 -07:00
nuno maduro e876dba8ba wip 2026-04-23 09:29:56 -07:00
nuno maduro d9c18f9c02 wip 2026-04-22 09:03:10 -07:00
nuno maduro 660b57b365 wip 2026-04-22 08:42:32 -07:00
nuno maduro 68527c996f wip 2026-04-22 08:25:38 -07:00
nuno maduro c6a42a2b28 wip 2026-04-22 08:07:52 -07:00
nuno maduro fcf5c27914 chore: adds YouTube channel badge
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:56:04 -07:00
nuno maduro 856a370032 style 2026-04-21 09:44:26 -07:00
nuno maduro e24882c486 wip 2026-04-21 09:41:19 -07:00
nuno maduro 51fc380789 wip 2026-04-21 09:40:01 -07:00
nuno maduro f6609f4039 wip 2026-04-21 08:36:41 -07:00
nuno maduro 2941f9821f wip 2026-04-21 08:15:24 -07:00
nuno maduro ed399af43e wip 2026-04-21 07:41:50 -07:00
nuno maduro 0d66dc4322 chore: removes https 2026-04-21 07:26:19 -07:00
nuno maduro 7e4280bf83 chore: improves feedback 2026-04-21 07:13:08 -07:00
nuno maduro a5915b16ab wip 2026-04-20 20:58:38 -07:00
nuno maduro 1476b529a1 wip 2026-04-20 19:56:23 -07:00
nuno maduro 2892341c28 wip 2026-04-20 14:28:18 -07:00
nuno maduro 59e781e77b wip 2026-04-20 13:48:05 -07:00
nuno maduro 55a3394f8c wip 2026-04-20 13:31:43 -07:00
nuno maduro 0d99c33c4e wip 2026-04-20 13:16:59 -07:00
nuno maduro adc5aae6f8 wip 2026-04-20 13:00:41 -07:00
nuno maduro 980667e845 wip 2026-04-20 12:25:39 -07:00
nuno maduro 8c849c5f40 fix: missing warning 2026-04-20 12:18:32 -07:00
nuno maduro 47f1fc2d94 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro 9c8033d60c feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro 42d1092a9e feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro c7e32f5d33 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro d379128cc4 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro f09d6f2064 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro 494cc6e2a4 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro f52a455773 fix 2026-04-20 10:09:07 -07:00
nuno maduro 184f5d2742 feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro 1d81069a2a feat(tia): continues to work on poc 2026-04-20 10:09:07 -07:00
nuno maduro 4b9bb77b54 feat(tia): adds poc 2026-04-20 10:09:07 -07:00
nuno maduro c440031e28 feat(tia): adds poc 2026-04-20 10:09:07 -07:00
nuno maduro 18bbca748f Merge branch '4.x' into 5.x 2026-04-18 07:03:46 -07:00
nuno maduro bff44562a9 release: v4.6.3 2026-04-18 06:51:25 -07:00
nuno maduro 9ebb990f96 chore: bumps phpunit 2026-04-18 06:51:17 -07:00
nuno maduro f142aad8ad Merge branch '4.x' into 5.x 2026-04-17 19:35:53 -07:00
nuno maduro cabff738f7 release: v4.6.2 2026-04-17 19:32:23 -07:00
nuno maduro 0746173a32 chore: bumps phpunit 2026-04-17 19:32:18 -07:00
nuno maduro 74a28d4f5e fix: wrapper runner 2026-04-17 07:29:03 -07:00
nuno maduro 6053e15d00 Merge branch '4.x' into 5.x 2026-04-17 06:07:14 -07:00
nuno maduro 87db0b4847 release: v4.6.1 2026-04-15 09:03:09 -07:00
nuno maduro 6ba373a772 chore: bumps phpunit 2026-04-15 08:49:34 -07:00
nuno maduro 945d476409 fix: allow to update individual screenshots 2026-04-15 08:34:06 -07:00
nuno maduro a8cf0fe2cb chore: improves CI 2026-04-15 08:20:50 -07:00
nuno maduro 2ae072bb95 feat: makes boot time much faster 2026-04-15 07:47:38 -07:00
nuno maduro 59d066950c chore: missing header 2026-04-15 07:47:22 -07:00
nuno maduro 0dd1aa72ef fix: updating snapshots in --parallel 2026-04-15 07:22:10 -07:00
nuno maduro 4e03cd3edb release: v4.6.0 2026-04-14 10:23:26 -07:00
nuno maduro eeab24e2bb Merge pull request #1671 from pestphp/feat/time-based-sharding
[4.x] Time based sharding
2026-04-14 18:18:09 +01:00
nuno maduro 9b64d5425a removes time balanced 2026-04-14 10:12:57 -07:00
nuno maduro 0acab1cbb4 wip 2026-04-14 09:53:57 -07:00
nuno maduro e616eab9fb wip 2026-04-14 09:36:38 -07:00
nuno maduro 7cbb1fcdb2 wip 2026-04-14 09:29:41 -07:00
nuno maduro cb5f6e1bd2 chore: style 2026-04-14 09:17:18 -07:00
nuno maduro 985dadd934 update 2026-04-14 09:16:32 -07:00
nuno maduro 10aee6045c feat(time-based-sharding): updates exception name 2026-04-14 09:08:52 -07:00
nuno maduro 4ac14b2528 feat(time-based-sharding): updates plugin 2026-04-14 08:34:41 -07:00
nuno maduro 2d649d765f chore: adjusts tests 2026-04-11 01:54:13 +01:00
nuno maduro 4fb4908570 Merge branch '4.x' into 5.x 2026-04-10 22:37:24 +01:00
nuno maduro 13c322bab3 ci: fixes incorrectCasing test 2026-04-10 20:51:40 +01:00
nuno maduro 3855249ce9 feat: adds --flaky cli option 2026-04-10 20:03:50 +01:00
nuno maduro f528bd8427 feat: adds flaky 2026-04-10 19:52:31 +01:00
nuno maduro acd8aafa63 fix: printer with --colors 2026-04-10 19:21:49 +01:00
nuno maduro e8d630e774 fix: printer with --colors 2026-04-10 19:21:41 +01:00
nuno maduro b6385dc865 fix: namespaces on toBeCasedCorrectly 2026-04-10 19:21:31 +01:00
nuno maduro 02dc8d7bcc chore: bumps deps 2026-04-10 19:21:18 +01:00
nuno maduro 729f18a152 fix: stacktrace with nested with calls 2026-04-10 17:25:05 +01:00
nuno maduro bdf60cea91 Merge pull request #1565 from louisbels/fix-dataset-method-chaining
fix: dataset inheritance with method chaining (beforeEach()->with(), describe()->with())
2026-04-10 17:05:25 +01:00
nuno maduro 3a8ee8291c Merge pull request #1628 from DevDavido/patch-1
fix: Parameter closure this type annotations in Functions
2026-04-10 16:58:39 +01:00
nuno maduro 654cb726c9 Merge branch '4.x' into patch-1 2026-04-10 16:58:26 +01:00
nuno maduro bce26aeaad Merge pull request #1634 from dbpolito/dataset_named_params
Dataset Named Parameters
2026-04-10 16:54:57 +01:00
nuno maduro 5948bcd71e chore: type improvements 2026-04-10 16:50:10 +01:00
nuno maduro 89006d83a9 chore: env 2026-04-10 16:27:44 +01:00
nuno maduro a8e974d64a chore: updates snapshots 2026-04-10 14:42:34 +01:00
nuno maduro 617b074049 Merge pull request #1626 from SimonBroekaert/feat/add_only-covered_option
feat: add '--only-covered' option to '--coverage'
2026-04-10 12:53:17 +01:00
nuno maduro 2eea71a664 Merge pull request #1624 from Vmadmax/fix/unicode-filename-filter
fix: preserve Unicode characters in filenames for --filter matching
2026-04-10 12:29:40 +01:00
nuno maduro 4b5374d507 Merge branch '4.x' into fix/unicode-filename-filter 2026-04-10 12:29:30 +01:00
nuno maduro 9085561ece chore: runs at 9am 2026-04-10 12:24:30 +01:00
nuno maduro b71bfc513a chore: guards 2026-04-10 12:23:49 +01:00
nuno maduro 75938ac9eb ci: updates deps 2026-04-10 12:18:28 +01:00
nuno maduro e766825f5b chore: fixes test:unit 2026-04-10 12:15:00 +01:00
nuno maduro 8a83a1a1a9 Merge pull request #1655 from stsepelin/fix/parallel-empty-suite-reporting
fix: nested dataset discovery and parallel invalid-dataset reporting
2026-04-10 11:59:48 +01:00
nuno maduro e63a886f98 Merge pull request #1661 from Avnsh1111/fix/opposite-expectation-truncated-message
fix: preserve full error message in not() expectation failures
2026-04-10 11:48:24 +01:00
nuno maduro 109bb22c5e Merge pull request #1615 from smirok/parallel-teamcity-concurrency-fix
fix: enhance support for --parallel and --teamcity arguments by restoring --teamcity for ParaTest and fixing teamcity output concurrency
2026-04-10 11:42:45 +01:00
nuno maduro 89dd212d84 Merge pull request #1580 from bibrokhim/patch-1
Add Rules to Laravel preset
2026-04-10 11:42:13 +01:00
nuno maduro cd07c6d966 Merge pull request #1569 from treyssatvincent/patch-1
add missing classes before toExtend on laravel preset
2026-04-10 11:41:53 +01:00
nuno maduro 8dddb47ad5 Merge branch '4.x' into fix-dataset-method-chaining 2026-04-10 11:41:13 +01:00
nuno maduro 3a6c2fab37 Merge pull request #1515 from yondifon/fix-trait-inheritance-detection
BugFix: Fix toUseTrait to detect inherited and nested traits
2026-04-10 11:39:56 +01:00
nuno maduro 281dbf6cf4 Merge pull request #1455 from SimonBroekaert/feat/to_be_cased_correctly_arch_test_assertion
feat: add toBeCasedCorrectly arch test assertion
2026-04-10 11:38:46 +01:00
nuno maduro 40c8429058 Merge pull request #1653 from orphanedrecord/fix/pest-php-stub-typo
Fix typo in Pest.php stubs
2026-04-10 11:35:41 +01:00
nuno maduro 8dd650fd05 Merge branch '4.x' into 5.x 2026-04-09 21:39:15 +01:00
nuno maduro d9d46c73f8 chore: stores statically the result 2026-04-09 21:36:49 +01:00
nuno maduro fbca346d7c fix: types 2026-04-07 14:40:55 +01:00
nuno maduro 3f13bca0f7 just in case 2026-04-07 14:37:13 +01:00
nuno maduro d3acb1c56a fix: coverage 2026-04-07 14:33:41 +01:00
avinash e601e6df31 fix: preserve full error message in not() expectation failures
When using not() expectations with custom error messages, the message
was truncated because throwExpectationFailedException() passed all
arguments through shortenedExport() which limits strings to ~40 chars.

Uses the full export() method for arguments instead of shortenedExport()
so custom error messages are displayed in their entirety.

Fixes #1533
2026-04-07 18:12:54 +05:30
nuno maduro 6fdbca1226 fix: parallel testing 2026-04-06 23:37:49 +01:00
nuno maduro 54359b895f Merge branch '4.x' into 5.x 2026-04-06 21:57:41 +01:00
nuno maduro e44c554a0b chore: bumps dependencies 2026-04-06 21:57:05 +01:00
nuno maduro 44c04bfce1 chore: bumps paratest 2026-04-06 14:41:38 +01:00
nuno maduro 271c680d3c Merge branch '4.x' into 5.x 2026-04-06 11:24:05 +01:00
nuno maduro 9797a71dbc feat(ai): allow temporary namesapce 2026-04-03 14:43:28 +01:00
nuno maduro c1a54df233 feat: --ai work in progress 2026-04-03 14:04:20 +01:00
nuno maduro 4a1d8d27b8 chore: bumps dependencies 2026-04-03 12:12:27 +01:00
nuno maduro 0f6924984c Merge branch '4.x' into 5.x 2026-04-03 12:02:36 +01:00
nuno maduro ce05ee9aad release: v4.4.4 2026-04-03 12:00:04 +01:00
nuno maduro 3d2ebdb273 bump: dependencies 2026-04-03 11:59:54 +01:00
nuno maduro 668ca9f5de feat: adds pao 2026-04-02 15:45:13 +01:00
nuno maduro f47b74445b Merge pull request #1657 from morpheus7CS/pint-formatting-rules-applied-to-stubs 2026-04-02 13:04:00 +01:00
Gal Jakic (We Wow Web) 6c42e7f4ea Laravel Pint default formatting applied to Pest-php.stub 2026-04-01 16:48:14 +02:00
dbpolito be3ff37517 Merge branch '4.x' of https://github.com/pestphp/pest into dataset_named_params 2026-03-26 18:08:26 -03:00
nuno maduro a087555383 bump: dependencies 2026-03-26 14:30:03 +00:00
Aleksandr Štšepelin 4b50cb486d Restore success snapshot coverage with lower memory limit 2026-03-25 23:59:29 +02:00
Aleksandr Štšepelin f7175ecfd7 Fix parallel dataset reporting and nested fixtures 2026-03-25 23:59:29 +02:00
Aleksandr Štšepelin 07737bc0b2 Fix parallel file selection and empty-suite reporting 2026-03-25 23:59:28 +02:00
nuno maduro f659a45311 Merge branch '4.x' into 5.x 2026-03-21 13:20:25 +00:00
nuno maduro e6ab897594 release: v4.4.3 2026-03-21 13:14:39 +00:00
nuno maduro a753b41409 chore: bumps phpunit 2026-03-21 13:14:35 +00:00
brianseymour 1a4c06bd6e Fix Pest comment typo while still honoring the Otwellian Waterfall 2026-03-18 20:51:45 -07:00
nuno maduro 12c1da29ee Merge branch '4.x' into 5.x 2026-03-10 21:21:24 +00:00
nuno maduro 5d42e8fe3a release: v4.4.2 2026-03-10 21:09:12 +00:00
nuno maduro 9d17b872dd chore: update stubs 2026-03-10 21:09:02 +00:00
nuno maduro 2a80101f42 chore: update styling 2026-03-10 21:06:28 +00:00
Nuno Maduro f7015fe59c chore: adjusts sponsors 2026-02-24 10:44:48 +00:00
nuno maduro 7281e0ded7 Add SerpApi to the list of sponsors
Add SerpApi as a sponsor in the README.
2026-02-20 01:18:43 +00:00
SimonBroekaert 1675dd1d41 chore: add tests for toBeCasedCorrectly() arch test 2026-02-17 19:03:46 +01:00
Nuno Maduro fa27c8daef chore: version 2026-02-17 17:52:40 +00:00
Nuno Maduro f0a08f0503 chore: missing types 2026-02-17 17:52:00 +00:00
Nuno Maduro 2c040c5b1f chore: style 2026-02-17 17:45:50 +00:00
Nuno Maduro a9ce1fd739 chore: code refactor 2026-02-17 17:45:34 +00:00
Nuno Maduro 3533356262 chore: updates snapshots 2026-02-17 17:44:56 +00:00
Nuno Maduro 4aa41d0b14 chore: bumps dependencies 2026-02-17 17:41:38 +00:00
Nuno Maduro e4ed60085c chore: bumps dependencies 2026-02-17 17:18:45 +00:00
Nuno Maduro e2b119655d chore: point pestphp dependencies to ^5.0.0 2026-02-17 17:13:36 +00:00
Nuno Maduro fcf5baf0a9 chore: start preparing for pest 5.x 2026-02-17 16:55:03 +00:00
SimonBroekaert df7b6c8454 feat: add toBeCasedCorrectly arch test assertion 2026-02-17 17:31:02 +01:00
Nuno Maduro 5de8693e3b chore: style 2026-02-17 16:15:58 +00:00
Nuno Maduro 7d80f1d20e chore: removes non used files 2026-02-17 15:34:32 +00:00
nuno maduro b3119cc120 Merge pull request #1562 from michaelw85/patch-1
[Fix] Pass test dir to worker
2026-02-17 15:33:16 +00:00
nuno maduro 4e294edf76 Merge pull request #1639 from imliam/patch-1
[Laravel Preset] Allow App\Http to be used in providers
2026-02-17 15:31:01 +00:00
Nuno Maduro f96a1b2786 release: v4.4.1 2026-02-17 15:27:18 +00:00
Nuno Maduro a49cf7edc5 ci: speed up ci 2026-02-17 15:21:20 +00:00
Nuno Maduro b0f6a74cb6 ci: makes jobs faster 2026-02-17 15:18:33 +00:00
Nuno Maduro aaa226f6a6 chore: tests against symfony 8 2026-02-17 15:14:45 +00:00
Nuno Maduro 69cb752d02 chore: bumps dependencies 2026-02-17 15:01:37 +00:00
Nuno Maduro cf00e58b7d chore: bumps dependencies 2026-02-17 11:22:04 +00:00
Liam Hammett 1f39b28e2c Allow App\Http to be used in providers 2026-02-16 00:25:47 +01:00
nuno maduro 9fcbca69d4 Update README.md 2026-02-13 10:41:22 +00:00
dbpolito b081584ab6 Improvements 2026-02-11 18:09:09 -03:00
dbpolito 6966802afc Cleanup 2026-02-11 18:02:21 -03:00
dbpolito c61dcad42b Dataset Named Parameters 2026-02-11 17:57:07 -03:00
David B. ec3e0b2d33 fix: Parameter closure this type annotations in Functions.php 2026-02-09 20:48:56 +01:00
SimonBroekaert c3620840b4 feat: add '--only-covered' option to '--coverage' 2026-02-06 11:57:21 +01:00
vmadmax 10a19f16ba refactor: simplify regex to use Unicode properties \p{L} and \p{N 2026-02-02 09:41:54 +01:00
vmadmax a956de5446 fix: preserve Unicode characters in class names for --filter matching 2026-02-02 09:01:24 +01:00
Nuno Maduro 3a4329ddc7 release: 4.3.2 2026-01-28 01:01:19 +00:00
Ilia Smirnov e6f511302b fix: enhance support for --parallel and --teamcity arguments by restoring --teamcity for ParaTest and fixing teamcity output concurrency 2026-01-27 16:47:24 +01:00
nuno maduro dd01229d7b Merge pull request #1606 from smirok/teamcity-fix-for-describe-tests-with-dataset
fix: replace `substr` with `mb_substr` in Str::beforeLast to ensure multibyte string compatibility and correct TeamCity test names for datasets in "describe" blocks
2026-01-15 01:52:01 +00:00
Ilia Smirnov c7e4efcea4 fix: replace substr with mb_substr in Str::beforeLast to ensure multibyte string compatibility and correct TeamCity test names for datasets in "describe" blocks 2026-01-14 00:55:35 +01:00
nuno maduro df3205e814 Merge pull request #1554 from pindab0ter/feature/extend-closure-this
Specify closure this for extend
2026-01-13 01:19:47 +00:00
Bakhromjon 0e7c2abe8b Add Rules to Laravel preset 2025-11-25 15:32:36 +05:00
Nino bd5fed9e12 add missing classes before toExtend on laravel preset
Add missing `->classes()` before `->toExtend()` on the laravel preset for 2 namespaces.

Otherwise you can't use interfaces on theses namespace.

For example, if you create an interface `YourSuperRequestContract` on the `app/Http/Requests` namespace you will get this error:

```
Expecting 'app/Http/Requests/YourSuperRequestContract.php' to extend 'Illuminate\Foundation\Http\FormRequest'.
```
2025-11-10 15:26:56 +00:00
Louis 26345fd9f4 fix: dataset inheritance with method chaining (beforeEach()->with(), describe()->with())
Fixes issue where datasets were not applied when using method chaining patterns
like beforeEach()->with([...]) or describe()->with([...]) inside nested describe blocks.

Root cause: Multiple functions were using Backtrace::file() which returns the
immediate caller's filename. This breaks when called through method chaining
because the backtrace returns internal Pest files instead of the test file.

Solution: Use Backtrace::testFile() which walks the entire backtrace to find
the actual test file being executed. This matches the pattern already used by
test() and describe() functions.

Changes in src/Functions.php:
- beforeEach(): Use testFile() to fix beforeEach()->with() pattern
- afterEach(): Use testFile() for consistency with beforeEach()
- beforeAll(): Use testFile() for better error messages
- afterAll(): Use testFile() for better error messages
- pest(): Use testFile() to fix pest()->beforeEach() pattern
- uses(): Use testFile() for consistency with pest()
- covers(): Use testFile() for correct test file context
- mutates(): Use testFile() for correct test file context

Changes in src/PendingCalls/DescribeCall.php:
- __destruct(): Force BeforeEachCall destructor before test creation
- __call(): Use $this->filename instead of Backtrace, more efficient
- __call(): Properly merge describing context for nested describe blocks

Fixes patterns:
- beforeEach()->with([...])
- describe()->with([...])
- pest()->beforeEach()->with([...]

Tests passing:
- tests/Features/Describe.php (all dataset tests)
- tests/Hooks/BeforeEachTest.php (global hook execution)
- tests/Features/Expect/toMatchSnapshot.php (28 tests)
2025-11-05 17:46:52 +01:00
Michael Withagen ae1da79ac1 Pass test dir to worker
#1444 
Test directory argument is lost when spawning workers, add it again.
2025-11-05 17:15:51 +01:00
Hans van Luttikhuizen-Ross b7b16096db Specify closure this for extend 2025-10-29 11:20:08 +01:00
Malico dc9a1e8ace BugFix: Fix toUseTrait to detect inherited and nested traits 2025-09-20 19:06:23 +01:00
482 changed files with 24932 additions and 2427 deletions
+13
View File
@@ -0,0 +1,13 @@
# Security Policy
**PLEASE DON'T DISCLOSE SECURITY-RELATED ISSUES PUBLICLY, [SEE BELOW](#reporting-a-vulnerability).**
## Reporting a Vulnerability
If you discover a security vulnerability in Pest, please report it privately using one of the following channels:
1. **GitHub Private Vulnerability Reporting** (preferred) — go to the repository's **Security** tab and click **"Report a vulnerability"**. This creates a private advisory visible only to maintainers and provides a structured workflow for triage, fix coordination, and CVE assignment.
2. **Email** — send the details to Nuno Maduro at **enunomaduro@gmail.com**.
All security vulnerabilities will be promptly addressed.
+12
View File
@@ -0,0 +1,12 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
cooldown:
default-days: 5
groups:
github-actions:
patterns:
- "*"
+31 -9
View File
@@ -2,9 +2,17 @@ name: Static Analysis
on:
push:
branches: [5.x]
pull_request:
schedule:
- cron: '0 0 * * *'
- cron: '0 9 * * *'
concurrency:
group: static-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
static:
@@ -12,25 +20,42 @@ jobs:
name: Static Tests
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: true
matrix:
dependency-version: [prefer-lowest, prefer-stable]
dependency-version: [prefer-stable]
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup PHP
uses: shivammathur/setup-php@v2
uses: shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240 # v2
with:
php-version: 8.3
php-version: 8.4
tools: composer:v2
coverage: none
extensions: sockets
- name: Get Composer cache directory
id: composer-cache
shell: bash
run: echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT
- name: Cache Composer dependencies
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: static-php-8.4-${{ matrix.dependency-version }}-composer-${{ hashFiles('**/composer.json', '**/composer.lock') }}
restore-keys: |
static-php-8.4-${{ matrix.dependency-version }}-composer-
static-php-8.4-composer-
- name: Install Dependencies
run: composer update --prefer-stable --no-interaction --no-progress --ansi
env:
COMPOSER_ROOT_VERSION: 5.x-dev
run: composer update --${{ matrix.dependency-version }} --no-interaction --no-progress --ansi
- name: Profanity Check
run: composer test:profanity
@@ -41,8 +66,5 @@ jobs:
- name: Type Coverage
run: composer test:type:coverage
- name: Refacto
run: composer test:refacto
- name: Style
run: composer test:lint
+31 -4
View File
@@ -2,35 +2,60 @@ name: Tests
on:
push:
branches: [5.x]
pull_request:
schedule:
- cron: '0 9 * * *'
concurrency:
group: tests-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
tests:
if: github.event_name != 'schedule' || github.repository == 'pestphp/pest'
runs-on: ${{ matrix.os }}
timeout-minutes: 15
strategy:
fail-fast: true
matrix:
os: [ubuntu-latest, macos-latest] # windows-latest
symfony: ['7.3']
php: ['8.3', '8.4', '8.5']
symfony: ['8.0']
php: ['8.4', '8.5']
dependency_version: [prefer-stable]
name: PHP ${{ matrix.php }} - Symfony ^${{ matrix.symfony }} - ${{ matrix.os }} - ${{ matrix.dependency_version }}
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup PHP
uses: shivammathur/setup-php@v2
uses: shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240 # v2
with:
php-version: ${{ matrix.php }}
tools: composer:v2
coverage: none
extensions: sockets
- name: Get Composer cache directory
id: composer-cache
shell: bash
run: echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT
- name: Cache Composer dependencies
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ matrix.os }}-php-${{ matrix.php }}-symfony-${{ matrix.symfony }}-composer-${{ hashFiles('**/composer.json', '**/composer.lock') }}
restore-keys: |
${{ matrix.os }}-php-${{ matrix.php }}-symfony-${{ matrix.symfony }}-composer-
${{ matrix.os }}-php-${{ matrix.php }}-composer-
- name: Setup Problem Matches
run: |
echo "::add-matcher::${{ runner.tool_cache }}/php.json"
@@ -38,6 +63,8 @@ jobs:
- name: Install PHP dependencies
shell: bash
env:
COMPOSER_ROOT_VERSION: 5.x-dev
run: composer update --${{ matrix.dependency_version }} --no-interaction --no-progress --ansi --with="symfony/console:^${{ matrix.symfony }}"
- name: Unit Tests
+1 -1
View File
@@ -1,7 +1,7 @@
.idea/*
.idea/codeStyleSettings.xml
.temp/*
composer.lock
/composer.lock
/vendor/
coverage.xml
.phpunit.result.cache
+34
View File
@@ -0,0 +1,34 @@
# CLAUDE.md
**Do not edit this file.** Agents must never add, remove, or reword anything here. If a change seems needed, say so and let a human do it.
## Ask before testing
When asked to review code or build a feature, do not run the test suite and do not write new tests. Make the change, report it, then ask the user whether tests should be added — describing the tests you have in mind — and wait for the user to confirm.
Two reasons this matters here: the suite takes minutes, and `tests/.snapshots/success.txt` plus the tally in `tests/Visual/Parallel.php` encode the whole suite's result, so a single added test breaks both.
Once the user confirms:
```bash
composer test:unit # fast, excludes the visual group
composer test:integration # visual and snapshot tests
composer test # everything CI runs, in CI's order
composer update:snapshots # only when a test was added or removed
```
## TIA scenario tests
`tests/Features/Tia/*` scaffold a throwaway git project, run a real `pest` subprocess against it, and diff the TIA graph it wrote. They exist because TIA's contract is about what a run *writes* — replay, branch keys, and the COMPLETE / RESULTS-ONLY / HARD-SUPPRESSED tiers are invisible to ordinary assertions, and every case used to be measured by hand against a playground app.
Add one whenever a change touches branch resolution, replay, filtered mode, or the write tiers. How:
- `Project::make('master')` scaffolds; `seed('master')` writes a graph and sentinels every cached result (`time=9.999`, `assertions=42`) so any rewrite shows up.
- `$project->pest('--tia', …)` runs it; `$project->delta()` compares against that snapshot. `writtenCount()` is the discriminator — `0` means "replayed", not "wrote the same values". `mutateGraph()` bends one entry; overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`.
- Keep expectations driver-independent: a cold recording run needs pcov/xdebug and behaves differently without one. Seed a graph instead of recording one.
Run them by file (a directory argument finds nothing) or by `--filter`:
```bash
php bin/pest tests/Features/Tia/PartialRunWriteTier.php
```
-14
View File
@@ -1,14 +0,0 @@
# Well documented Makefiles
DEFAULT_GOAL := help
help:
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z0-9_-]+:.*?##/ { printf " \033[36m%-40s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
build: ## Build all docker images. Specify the command e.g. via make build ARGS="--build-arg PHP=8.2"
docker compose build $(ARGS)
##@ [Application]
install: ## Install the composer dependencies
docker compose run --rm composer install
test: ## Run the tests
docker compose run --rm composer test
+342
View File
@@ -0,0 +1,342 @@
# TIA write-tier conformance — plan
Baseline sweep: **147 cases, 144 PASS / 2 FAIL / 1 SKIP** against `dev-fix/tia-filtered` (`db70017`)
on `playground/laravel` with pcov enabled.
This file records (1) what is left to fix, and (2) the full case matrix with the target
outcome for every row, so the sweep can be re-run as a conformance check after the fixes land.
---
## Part 1 — What's left
### 1. `--parallel` silently under-records `edges` (G12, I5) — **high**
Any `--tia --parallel` complete run writes `files=4` where sequential writes `files=25`. Only
`app/**` and blade views survive; every `config/`, `routes/`, `bootstrap/` edge and every test
self-edge is lost. A `routes/web.php` edit that sequential TIA catches, parallel TIA reports as
`No affected tests found`.
Root cause — `src/Restarters/PcovRestarter.php:33`:
```php
if (! Tia::isEnabledForRun($arguments)) {
return;
}
```
`Tia::isEnabledForRun()` (`src/Plugins/Tia.php:275-288`) returns true only for `--tia` in argv or
the `PEST_TIA` env flag. A paratest worker's argv carries neither, so workers never re-exec with
`-d pcov.directory=<projectRoot>` and record under pcov's empty default scope.
Proven both directions:
- `PEST_PCOV_RESTARTER_RESTARTED=1 pest --tia` (sequential, restart suppressed) reproduces
`files=4` with edge sets identical to the parallel graph.
- `PEST_TIA=1 pest --parallel` restores the full `files=25` graph. **Working workaround today.**
Coverage config is irrelevant: widening `phpunit.xml` `<include>` to `app|config|routes|bootstrap|tests`
still yields `files=4`.
Sticky, not just per-run: `pest --tia --filtered --parallel` re-records only the affected test
through a worker and strips that entry's edges from an otherwise-healthy sequential graph, so one
parallel run degrades a good graph incrementally.
**Fix direction:** propagate the TIA-enabled decision into workers (e.g. set `PEST_TIA` in the
worker env, or have the parent stamp a recording global the restarter also consults) so
`PcovRestarter` restarts them. Then assert `edges` equivalence between sequential and parallel.
### 2. Deleted test file with a cached failure wedges `--filtered` forever (I9) — **high**
Expected a WARN and a full-suite fallback. Actual: `--filtered` selects the phantom file, runs
nothing, exits **0 green**, on every subsequent invocation.
Root cause — `src/Plugins/Tia/Graph.php:1523-1525`:
```php
if ($real === false) {
$real = $path;
}
```
`relative()` falls back to the raw path when `realpath()` fails, so a deleted-but-in-project test
file is never "unlocated", `Graph::hasUnlocatedTestsToRerun()` never fires, and the WARN branch at
`src/Plugins/Tia.php:982-986` is unreachable for the deletion case. `pruneMissingTests()` only runs
under `--fresh`, so nothing clears it.
**Fix direction:** distinguish "path could not be resolved" from "path resolved outside the root" in
`relative()`, or have `hasUnlocatedTestsToRerun()` stat the file directly.
### 3. `--coverage-text` does not disable filtered mode (I8) — **low**
`coverageReportActive()` (`src/Plugins/Tia.php:1705-1710`) reads only Pest's own `--coverage`, so
raw PHPUnit coverage flags (`--coverage-text`, `--coverage-html`, `--coverage-clover`, …) leave
filtered mode on. `Tia::COVERAGE_FLAGS` at `:113-115` already enumerates them — that list is not
consulted here.
### 4. `--coverage` narrows edges like parallel does — **medium, needs triage**
`pest --tia --coverage` prints `fresh graph (recording coverage baseline)` even when a graph
exists, and writes `Feature/ExampleTest` with 2 files instead of 16, dropping self-edges. Same
observable shape as G12, likely a different mechanism (piggyback collector scoped by
`phpunit.xml <include>` rather than the pcov-restarted recorder). Theme: **edges silently narrow to
`app/**` whenever recording does not go through the pcov-restarted TIA recorder.** Confirm whether
these are one fix or two.
### 5. Smaller items
| Item | Where | Note |
|---|---|---|
| Warning/deprecation recorded as `status=0` | write path | Codes `6` and `4` appear unreachable; risky/skipped/incomplete record faithfully |
| Complete non-TIA runs write edge-less results | `src/Plugins/Tia.php:1645`, `:638` | Guard is `! $complete && ! knowsTest()`; complete bypasses it and `markKnownTestFiles` stays false. Inert — replay guards on the same predicate at `:360` — but inflates `n` |
| SIGINT never reaches the suite | `PcovRestarter` re-exec | Parent ignores it; the child holds PHPUnit's handler. CI `timeout`/Ctrl-C will not stop a run |
| Structural drift never announced sequentially | `src/Plugins/Tia.php:1178-1181` | `enterRecordMode()` prints a bare `Running in TIA mode.`; `renderFreshGraph()` (`fresh graph (composer.lock changed)`) only runs under `--parallel` or coverage piggyback |
| ~~Replay clobbers cached `time`~~ | write path | **Struck in phase four — does not reproduce**, sequentially or in parallel. Both write paths route through `resultTime()`; a replay with every cached `time` sentinelled writes nothing |
| ~~`pest --parallel` without `--tia` writes nothing~~ | G3, G6 | **Closed in phase four.** Workers flush their results through `requestWorkerResults()`, so a parallel run refreshes and prunes exactly like the sequential run of the same command |
| ~~`--tia --parallel --filter`/`--shard` record nothing~~ | G2, G8 | **Closed in phase four** (partial parity landed first, complete parity with the row above) |
| `--min=50` without `--coverage` is a silent no-op | — | — |
| `pest --repeat=2` is not a Pest option | J11 | Case unrunnable as written; drop it or add the option |
### 6. Test-harness gaps to close before re-running
- The playground has no `UsesClass`, `->note()`, `->flaky()`, `->issue()`, `->pr()`, `->ticket()` or
`->assignee()` annotations, so **C19, C23C28, C37C39 matched zero tests** and their
graph-invariant assertions proved nothing. Add annotated fixtures to make those rows load-bearing.
- Use **sentinel patching** (rewrite every cached entry to `time=9.999 assertions=42`, then see which
entries get overwritten) as the discriminator. It is the only way to tell "wrote identical values"
from "wrote nothing". A canary test absent from `edges` is unreliable under `--filter` because it
never matches the filter and so never runs.
- `--tia` on a clean green tree can never cache a failure (unchanged tests replay rather than
execute); seeding one requires `--fresh` or an env-driven flaky fixture.
- Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
---
## Part 2 — Full case matrix (target outcomes)
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Status column: `PASS` = conforming today · `FIX` = must pass once the item above lands ·
`SKIP` = case not runnable as written.
### A — Setup & sanity
| # | Case | Target outcome | Status |
|---|---|---|---|
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS |
| A2 | `pest --baseline` | prints an existing dir; exit 0 | PASS |
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | PASS |
| A4 | `git status` after reset ritual | clean | PASS |
| A5 | `extension_loaded("pcov")` | `true` | PASS |
| A6 | delete graph, plain `pest` | no graph created | PASS |
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS |
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | PASS |
### B — COMPLETE runs still write
| # | Case | Target outcome | Status |
|---|---|---|---|
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | PASS |
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS |
| B3 | `pest` (plain) | same as B2 | PASS |
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | PASS |
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS |
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS |
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS |
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS |
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS |
| B10 | `pest --tia` twice | `time` differs; statuses stable | PASS |
| B11 | add a test file, `pest --tia` | new `edges` key + new result appear | PASS |
| B12 | `pest --tia --coverage` | completes; graph written | PASS |
### C — Selection narrowing → RESULTS-ONLY
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
| # | Case | Target outcome | Status |
|---|---|---|---|
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS |
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS |
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS |
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS |
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS |
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS |
| C7 | `pest --group=smoke` | RO; no notice | PASS |
| C8 | `pest --tia --group=smoke` | RO; notice | PASS |
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS |
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS |
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS |
| C12 | `pest --tia tests/Unit` | RO; notice | PASS |
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS |
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS |
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS |
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS |
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS |
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS |
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS |
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS |
| C22 | `pest --tia --dirty` | RO; notice | PASS |
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS |
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS |
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS |
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS |
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS |
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS |
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS |
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS |
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
### D — Truncation → RESULTS-ONLY
D1D5, D11D13, D16 precondition: `trio one` broken. D6D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
| # | Case | Target outcome | Status |
|---|---|---|---|
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS |
| D2 | `pest --retry` | RO; siblings survive | PASS |
| D3 | `pest --stop-on-failure` | RO | PASS |
| D4 | `pest --stop-on-defect` | RO | PASS |
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS |
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS |
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS |
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS |
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS |
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS |
| D11 | `pest --tia --bail` | RO | PASS |
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS |
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS |
| D14 | D13 config, green suite | COMPLETE | PASS |
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS |
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS |
### E — Result merge semantics
| # | Case | Target outcome | Status |
|---|---|---|---|
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS |
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS |
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS |
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS |
| E5 | any partial run | `assertions` and `time` update for the test that ran | PASS |
| E6 | any partial run | `message` of untouched tests unchanged | PASS |
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS |
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS |
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS |
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS |
| E11 | partial run of a risky test | `status=5` | PASS |
| E12 | any partial run | `fingerprint` byte-identical | PASS |
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS |
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS |
### F — Guard rails
| # | Case | Target outcome | Status |
|---|---|---|---|
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | PASS |
| F2 | complete `pest --tia` first, then the same filter | result is recorded | PASS |
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS |
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS |
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS |
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS |
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS |
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS |
### G — Parallel
| # | Case | Target outcome | Status |
|---|---|---|---|
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written | PASS |
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS |
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS |
| G4 | `pest --tia --parallel --bail` | RO | PASS |
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | PASS (but see G12 — it strips edges) |
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | PASS |
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS |
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS |
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS |
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS |
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | PASS |
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** (`files=25` both) | **FIX (item 1)** |
### H — Baseline key / branch resolution
| # | Case | Target outcome | Status |
|---|---|---|---|
| H1 | `pest --tia` on `master` | only a `master` key | PASS |
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS |
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS |
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS |
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS |
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS |
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS |
| H8 | after every C and D case | no baseline key other than the real branch | PASS |
| H9 | non-git dir, `pest --tia` | `MissingDependency``The feature "Tia mode" requires "git".` | PASS |
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS |
### I — Filtered mode
| # | Case | Target outcome | Status |
|---|---|---|---|
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS |
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | PASS |
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS |
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS |
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **FIX (item 1)** |
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS |
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS |
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report | **FIX (item 3)** |
| I9 | cached failure whose test file was deleted | WARN `could not be located on disk`; falls back to the full suite with replay | **FIX (item 2)** |
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS |
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS |
| I12 | edit `composer.lock` | fingerprint drift → full rebuild (and the drift reason should be printed — item 5) | PASS |
### J — Interactions & regressions
| # | Case | Target outcome | Status |
|---|---|---|---|
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS |
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS |
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS |
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS |
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS |
| J6 | `pest --tia --compact` | COMPLETE | PASS |
| J7 | `pest --tia -v` | COMPLETE | PASS |
| J8 | `pest --tia --profile` | COMPLETE | PASS |
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS |
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS |
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP**`--repeat` is not a Pest option |
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS |
| J13 | `pest --tia --min=50` | COMPLETE | PASS |
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (signal must go to the re-exec'd child — item 5) |
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS |
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS |
---
## Re-run procedure
```bash
cd /Users/nunomaduro/Work/projects/playground/laravel
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json"
# per case
git checkout -q . && git clean -qfd tests app
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
```
Every pest invocation must be prefixed with `PAO_DISABLE=1` (the app has `laravel/pao`, which emits
JSON when it detects an agent). The shell is zsh — build commands with arrays or `eval`; unquoted
`$args` does not word-split.
+312
View File
@@ -0,0 +1,312 @@
# TIA deep audit — phase five
## Your task
Phases one through four fixed the defects that were *reported*. This phase is the opposite shape: go
looking. Read TIA's read/write path adversarially, find what is wrong or fragile, and leave behind a
scenario suite that covers the edges nobody has exercised yet.
Two deliverables, both required:
1. **A ranked findings list.** Every finding needs a *reproduction*, not a reading of the code. A
finding you cannot reproduce is a hypothesis — say so and rank it separately.
2. **New scenario tests** in `tests/Features/Tia/*`, covering the edges you probed. Rows that pass go
in the repo (they are the regression net). Rows that fail stay in your scratchpad until the fix
lands — **never commit a red test.**
Work in passes, and **report between passes** rather than at the very end:
- **Pass A** — reproduce the leads in Part 3 below. Report which are real.
- **Pass B** — your own hunt: the invariants in Part 2, attacked with inputs nobody tried.
- **Pass C** — fixes, smallest first, each with the row that pins it.
Fix what is clearly a defect with an obvious correct answer. **Stop and ask** when the fix is a
behaviour *choice* (what should TIA do when two runs race for one graph?) — those are Nuno's calls,
and a reproduction with a crisp yes/no question is worth more than a guessed fix.
---
## Part 1 — The harness
### 1.1 What TIA is
Test Impact Analysis. `pest --tia` records a dependency graph (test file → source files it touched)
plus a per-branch baseline of results, then on later runs replays the tests whose dependencies did not
change. State lives in a per-project dir under `~/.pest/tia/`; `graph.json` is the whole thing.
Source of truth: `src/Plugins/Tia.php` (the plugin, ~1900 lines) and `src/Plugins/Tia/Graph.php` (the
graph model + read/write API). Supporting: `src/Plugins/Tia/ChangedFiles.php` (git), `Fingerprint.php`
(environment/structure hashing), `State.php` (the state dir).
Read `PLAN.md`, `PLAN_PHASE_TWO.md`, `PLAN_PHASE_THREE.md`, `PLAN_PHASE_FOUR.md` first — in that
order. They carry the history, the tier contract, and the decisions already made. Struck-through rows
are fixed; do not re-report them.
### 1.2 The scenario harness
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the graph it wrote. Everything lives in `tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `seedFor($root, …)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `path($rel)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroy()`, `destroyAll()`, `SEQUENTIAL_AND_PARALLEL` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()` |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`); `Project::EDGES` and
`Project::TESTS` describe the graph `seed()` writes. `Project::testId($file, $description)` builds a
result key. Overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`
that is how you configure `pest()->tia()->…` for a scenario.
**If the fixture cannot express your case, extend the fixture.** A new `GitRepo` verb or a new overlay
is a legitimate part of the deliverable — several findings below need one. Do not water down a
scenario to fit the current helpers.
### 1.3 The sentinel discriminator — read before writing any assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`, `assertions=42`**
(only where assertions are non-zero — risky/skipped/incomplete statuses are *derived* from "performed
no assertions", so falsifying those would rewrite the status on replay), then snapshots. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
- A replayed suite reports inflated assertion counts (6 tests × 42). That is the sentinel, not a bug.
The three write tiers, unchanged since phase two:
- **COMPLETE** — may change everything.
- **RESULTS-ONLY** — may change only `baselines[<branch>].results` for tests that ran; never removes
an entry, never adds a result for a test file absent from `edges`, never touches
`sha`/`tree`/`edges`/`files`/`fingerprint`.
- **HARD-SUPPRESSED** — may change nothing.
### 1.4 Running them
They are in the `integration` group (`tests/Pest.php`), so `composer test:unit` skips them. **A
directory argument finds nothing** — pass files, space-separated, as separate argv entries:
```bash
PAO_DISABLE=1 php84 bin/pest tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/PartialRunWriteTier.php \
tests/Features/Tia/CompleteRunWriteTier.php tests/Features/Tia/FilteredMode.php
PAO_DISABLE=1 php bin/pest <same list>
```
**Both interpreters must be green.** `php84` is 8.4.x with **no pcov** — that is what CI has
(`.github/workflows/tests.yml` sets `coverage: none`). `php` is 8.5.x with pcov — that is the dev
machine. Any assertion that depends on a coverage driver passes locally and fails in CI. Concretely:
- A **cold recording run writes no graph at all** without pcov/xdebug (it prints `Running in TIA mode,
however TIA is skipped as it needs ext-pcov or Xdebug`). **Seed a graph, never record one**, unless
the driver *is* the point of the row.
- A **PHP source file edit** driverless triggers `Detected PHP source changes but no coverage driver
is available` → full suite, `affected=0`. Edit *test* files, not `app/` files.
`PAO_DISABLE=1` is mandatory on every pest invocation and every probe: `laravel/pao` emits JSON under
agents and corrupts the captured output.
Baseline before you touch anything: **72 passed** on both interpreters, at `HEAD` plus the phase-four
working-tree changes. If that number does not reproduce, stop and say so.
### 1.5 Measure before you assert
Never guess an expectation from reading the code. Probe first, in your scratchpad:
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`; `Project::destroyAll()`
at the end. Run every probe on **both** interpreters before believing it.
### 1.6 Ground rules
- **Do not** run `composer test`. It takes minutes and you do not need it.
- **Do not** run `composer update:snapshots`. `tests/.snapshots/success.txt` and the tally in
`tests/Visual/Parallel.php` encode the whole suite's result, so every row you add breaks them.
That is expected — **report that they need regenerating and let Nuno run it.**
- **Do not commit.** Leave the tree dirty.
- **Do not touch the playground's `vendor/`.** If a finding truly needs the playground (25 real tests,
real timings), say so and ask — syncing it is a manual step Nuno owns.
- Run `vendor/bin/phpstan analyse <the files you touched> --memory-limit=-1 --no-progress` and
`vendor/bin/pint <the files you touched>` before reporting. **Scope both to files you changed** —
Nuno edits `src/` live, and a repo-wide fixer will revert his work.
- The fixture projects **hardlink `src/`**. If you edit `src/` while a scenario run is in flight,
those rows go red for no reason. Finish the edit, then run.
- Keep scratch probes out of the repo.
- **Never weaken an existing assertion to make something pass.** If an existing row contradicts your
fix, that is a finding: report the contradiction and ask.
---
## Part 2 — The invariants to attack
These are the properties TIA is supposed to have. Each one is a place to hunt: construct the input
that breaks it.
1. **Parity.** `pest <args>` and `pest --parallel --processes=N <args>` must leave the **same graph**
and reach the same tally. This is a hard rule from Nuno — sequential and parallel must *always*
agree. `Project::SEQUENTIAL_AND_PARALLEL` is the dataset that encodes it; consider making every
new write-path row use it. Vary `--processes` (1, 2, 8 — more processes than test files).
2. **The tiers hold.** Every command lands in exactly one of COMPLETE / RESULTS-ONLY /
HARD-SUPPRESSED, and stays inside it. Combinations are where this frays: `--fresh --parallel
--filter`, `--bail --shard`, `--filtered` plus an explicit path, `--tia --no-tia`, `--retry`.
3. **Replay is faithful.** A replayed test reports the same status, message, time and assertion count
as the recorded run — and replay itself writes nothing. Statuses beyond pass/fail are the soft
spot: skipped, incomplete, risky, notice, deprecation, warning, todo, and a test that failed with a
multi-line message.
4. **Reads never write.** No read path may mint a baseline key, move a `sha`, or create the state dir.
A project that has never run TIA must gain nothing from a plain `pest` run.
5. **A branch never corrupts another branch's baseline.** Writes land on the branch that ran, and
only there. Reads may *layer* the default branch under the current one (phase four, B1) — but that
layering must not leak into a write.
6. **Nothing is unbounded.** Baseline keys, `files`, `edges`, worker partials, state files: something
must eventually reclaim them, or the graph grows forever.
7. **A hostile state dir cannot break a run.** Corrupt, truncated, empty, wrong-schema, read-only,
absent, or *someone else's* `graph.json` — the suite still runs and exits on the tests' merit.
8. **Git shapes are all handled.** Detached HEAD (read-only, per phase four B2), worktrees, no commits
yet, no `origin`, no `origin/HEAD`, submodules, a repo whose root is above the pest project (that
one panics deliberately — `TiaRequiresRepositoryRoot`), renamed branches, deleted branches.
---
## Part 3 — Leads to reproduce first (Pass A)
These came out of reading the phase-four diff. **Each is a hypothesis, not a finding** — several may
turn out to be fine. Reproduce or refute each, in order, and report the measured delta either way.
### L1 — the per-entry fallback may resurrect a pruned or deleted test · **highest value**
Phase four made `Graph::baselineFor()` layer the branch's results **over** the default branch's. Two
consequences worth probing:
- `pruneStaleResults()` unsets an entry from `baselines[branch].results`. The very next read layers the
**fallback's** entry for that same test id back in. So a delete may not stick from a branch's point
of view.
- `hasUnlocatedTestsToRerun()` returns true when a *failing* cached result names a file that no longer
exists on disk — and that forces a **full suite**. If a feature branch deletes a test file that
fails on the default branch, the merged read still carries master's entry pointing at the now-absent
file. Suspected symptom: **that branch runs the full suite forever.**
Probe: seed master with a failing test (`seed('master', failing: [...])`), branch off, delete the test
file that holds it, run `--tia`, and compare `replayed`/`uncached`/`affected` against the same shape
where the failure is on the branch instead. Then the mirror case with a green deleted test.
### L2 — environment drift clears one branch's results and the fallback serves them right back
`reconcileFingerprint()` on environmental drift calls `$graph->clearResults($this->branch)` and warns
`results dropped, edges reused`. On a feature branch that clears only the *branch's* results — the
layered read then re-serves the default branch's results, which were recorded under the **old**
environment. Suspected symptom: the drop is a no-op on any branch that is not the default one.
Probe: `pestWithEnvironment()` to shift whatever `Fingerprint` reads as environmental (check
`Fingerprint::environmentalDrift()` for the exact keys), on the default branch vs a feature branch,
and compare what survives.
### L3 — `sha`/`tree` may be taken from a different commit than the results
`baselineFor()` takes `sha` from the branch when non-null and otherwise from the fallback; `tree`
likewise when the branch's is empty. So a branch can end up computing "what changed since" against the
**default branch's** recorded sha while reading its own results — or vice versa. Is there a shape where
that under-reports changed files (a test replays that should have run)? That is the dangerous
direction: a false replay is a lie about a passing test.
Probe: seed master, commit a test edit on the branch so the shas genuinely differ, and check whether
the edited test is treated as affected.
### L4 — pruning from merged worker partials
Phase four made a complete `--parallel` run write and prune from merged worker results. The stated
safety net is that `pruneStaleResults()` only prunes files it saw results for, and that a truncated
worker sets results-only. Try to defeat it: a worker that reports results for a test file it did not
finish. Candidate shapes — a fatal error mid-file (not an assertion failure), `exit()` inside a test,
an uncaught error in an `afterEach`, a test that kills its own process, `--stop-on-failure` variants,
`--processes` greater than the number of test files.
### L5 — two runs racing for one `graph.json`
`State::write()` has no locking. Two pest processes on one project (a watcher plus a manual run, two CI
jobs sharing a cache dir, `--parallel` where the parent writes while a straggler worker flushes) can
lose an update or interleave. Probe by launching two `pest --tia` subprocesses concurrently against one
project and diffing. **This is likely a design decision, not a bug** — if you reproduce a lost update,
report it as a question (accept last-writer-wins, or lock?), do not invent a locking scheme.
### L6 — a detached HEAD still purges on structural drift
Phase four made a detached HEAD read-only *for writes* (`saveGraph()` refuses). But
`reconcileFingerprint()` deletes the whole graph on structural drift (`Tia.php`, the
`state->delete(KEY_GRAPH)` in the structural branch) before any write happens. So `--tia` from a
detached checkout with a changed `composer.lock` can still wipe the default branch's baseline. Confirm
it, then ask: should the detached-HEAD guard cover the purge too?
### L7 — statuses that may not round-trip
`PLAN.md` §5 claims warnings and deprecations record as `status=0`, and that codes `6` and `4` look
unreachable. `Graph::getResult()` maps 08 to `TestStatus`. Verify each status end to end: record it,
replay it, and check the replayed run reports the same thing — including the message, the exit code,
and whether `shouldRerunStatus()` decides to re-execute it. `failOnRisky` / `failOnSkipped` /
`displayDetailsOn*` change that decision, so an overlay that flips those config flags is part of this.
A status that replays as a pass would be the most serious class of bug in TIA.
### L8 — branch-key hygiene
Nothing appears to reclaim baseline keys. Probe: create and delete 5 branches, rename one
(`GitRepo::rename()`), and check what `branchKeys()` holds afterwards. Also try names that stress the
JSON keying: `feature/x/y` (already covered), a name differing from another only in case (macOS is
case-insensitive — does the key match the ref?), a name with a space or a unicode character, a branch
literally called `HEAD`, and a very long name. Then: is unbounded growth acceptable, or does this need
a cap / GC? Ask rather than build.
### L9 — `soleRecordedBranch()` as a fallback source
When config, CI env and git all fail to name a default branch, resolution falls back to "the only
branch in the graph". If that sole key was minted by a *narrowed* run on a feature branch (which phase
four's B1 made a live possibility), the fallback now names a feature branch, and every other branch
layers **its** results underneath. Probe: `withoutGit()` or `removeOrigin()` + no config, with a graph
whose only key is `feature-x`.
### L10 — the state dir as an adversary
Beyond corrupt JSON (fixed in phase four by deleting it): a valid-JSON graph with `schema: 2`; a graph
whose `files` and `edges` disagree; `results` entries with a `file` pointing outside the project root
or at an absolute path from another machine; a `graph.json` that is a directory; a state dir with no
write permission; `$HOME` unset. Each should degrade to "run the tests", never crash and never write
garbage.
---
## Part 4 — Reporting
**Between passes**, not just at the end. Per finding:
- **Reproduced (yes / no / hypothesis only)**, with the exact command and the measured
`tally` + `delta()->summary()` on **both** interpreters.
- **Severity**, and say why in one line. The scale that matters here: *a test wrongly replayed as
passing* (worst) > *cache silently useless, full suite forever* > *graph grows / stale data* >
*cosmetic*.
- **Fixed / deferred / needs-a-decision**, and the row that pins it.
- For anything needing a decision: **one yes/no question**, no essay.
Close with:
1. The count of `tests/Features/Tia/*` green on **both** `php84` (no pcov) and `php85` (pcov), against
the 72 baseline.
2. Every row you added, and what invariant from Part 2 it defends.
3. Which findings are still open, as yes/no questions.
4. That `tests/.snapshots/success.txt` and the `tests/Visual/Parallel.php` tally need regenerating —
**do not regenerate them.**
5. **What you looked at and found solid.** A list of attacks that did not break anything is a real
result: it tells the next phase where not to spend its time.
+351
View File
@@ -0,0 +1,351 @@
# TIA defect sweep — phase four
## Your task
Five defects in TIA's read/write path, found while building the repo's TIA scenario suite. One is
confirmed and load-bearing (**B1**); four need a decision before a fix (**B2****B5**).
Work **one bug at a time, in order**, and for each:
1. **Reproduce it as a repo test first.** The reproduction is the deliverable even when the fix is
deferred — a red test that pins the exact symptom is worth more than a prose report. Do not commit
a red test to the suite; keep it in a scratch file until the fix lands (see Part 1.4).
2. Confirm the measured numbers in this file still hold. They were taken at commit `4d3d0105` plus
the two uncommitted changes described in Part 2. If a number has moved, say so and stop.
3. Fix, then re-run **the whole `tests/Features/Tia/*` set on two interpreters** (Part 1.3).
4. **HARD STOP after B1.** Report the diff and wait — B1's fix changes `Graph`'s read semantics for
every caller, and Nuno wants to see it before B2B5 pile on top.
Per `CLAUDE.md`: do not run `composer test`, and do not regenerate snapshots unless told. Do not
commit. Do not touch the playground's `vendor/`.
---
## Part 1 — The harness
### 1.1 What exists
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the TIA graph it wrote. Everything lives in
`tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroyAll()` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()` |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`), with `Project::EDGES` and
`Project::TESTS` describing the graph `seed()` writes. `Project::testId($file, $description)` builds
a result key.
### 1.2 The sentinel discriminator — read this before writing an assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`,
`assertions=42`** (non-zero assertion counts only — risky/skipped/incomplete statuses are *derived*
from "performed no assertions", so falsifying those would rewrite the status on replay), and takes a
snapshot. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
Tiers, unchanged since phase two: **COMPLETE** may change everything · **RESULTS-ONLY** may change
only `baselines[<branch>].results` for tests that ran, never removing an entry, never adding a result
for a test file absent from `edges`, never touching `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
### 1.3 Running them
They are in the `integration` group (`tests/Pest.php:22`), so `composer test:unit` skips them. A
**directory argument finds nothing** — pass files:
```bash
F="tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/PartialRunWriteTier.php \
tests/Features/Tia/CompleteRunWriteTier.php tests/Features/Tia/FilteredMode.php"
PAO_DISABLE=1 php84 bin/pest $F # 8.4.23, NO pcov — this is what CI has
PAO_DISABLE=1 php bin/pest $F # 8.5.8, pcov — this is what your machine has
```
**Both must be green.** `.github/workflows/tests.yml` sets `coverage: none`, so any assertion that
depends on a coverage driver fails in CI while passing locally. Two tests were already caught by
this. Concretely: a **cold recording run writes no graph at all** without pcov/xdebug (it prints
`Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug`), and a **PHP source
file edit** triggers `Detected PHP source changes but no coverage driver is available` → full suite,
`affected=0`. Seed a graph instead of recording one, and edit *test* files rather than `app/` files,
unless the point of the row is the driver itself.
### 1.4 Measure before you assert
Do not guess expectations from reading the code — every number in Part 3 came from a scratch probe.
The pattern (put it in your scratchpad, not in the repo):
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`.
---
## Part 2 — What the code looks like right now
Phase three landed default-branch resolution: `ChangedFiles::defaultBranch()`, the
`pest()->tia()->defaultBranch()` config surface, `Graph::setFallbackBranch()` + `?string
$fallbackBranch = null` on the seven read methods, and `Tia::resolveFallbackBranch()`
(`Tia.php:~1776`) resolving **config → CI env (`CiDefaultBranch`) → git (`origin/HEAD`, then
`init.defaultBranch` if the branch exists) → `soleRecordedBranch()`**, failing loudly when nothing
can name it.
On top of that, **two uncommitted changes** you will see in `git diff`:
1. `TIA_RESULTS_ONLY` global — a *partial* parallel run with an existing graph now purges stale
worker partials, sets the global, and workers flush their results through the existing
`flushWorkerReplay()` / `mergeWorkerReplayPartials()` path; the parent writes them with
`complete: false`. Gated on a graph already existing, so a TIA-less project still creates no
baseline dir. This gave `--parallel --filter` parity with sequential — **and, per B1, handed it
the shadowing bug too.**
2. `loadGraph()` emits `WARN The dependency graph could not be read — it will be rebuilt.` once per
parent process when `graph.json` exists but will not decode. Previously silent.
62 scenario tests cover this and pass on both interpreters.
---
## Part 3 — The defects
### B1 — a thin baseline key permanently shadows the default-branch fallback · **confirmed, priority 1**
**Symptom.** Any *narrowed* run on a new branch (`--filter`, `--group`, a path, `--bail`, `--shard`,
and now `--parallel --filter`) writes a baseline key holding only the tests that ran. From then on
`--tia` on that branch reads that thin key instead of falling back to the default branch, so
everything else is uncached — **one full suite per branch, forever**, which is the exact cost
issue [#1823](https://github.com/pestphp/pest/issues/1823) was about, re-entering through a side door.
**Measured** (fixture: 6 tests, graph seeded on `master`):
```
switch -c feature-x; pest --filter="adds two numbers"
→ keys=[master,feature-x] (feature-x holds 1 result)
pest --tia
→ 6 passed (6 assertions, 5 uncached, 1 replayed) ← want: 6 replayed, 0 uncached
same via: pest --parallel --processes=2 --filter="adds two numbers" → identical
```
**Where.** `src/Plugins/Tia/Graph.php::baselineFor()` (~line 814):
```php
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
```
The fallback is all-or-nothing: it fires only when the branch has **no** key at all. The key itself
is minted by `Graph::setResult()``ensureBaseline($branch)` (~599 / ~829), reached from
`Tia::snapshotTestResults()` on partial runs.
**Reproduction to add** (`tests/Features/Tia/DefaultBranchReplay.php`):
```php
test('a narrowed run on a new branch does not cost the fallback', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->switchTo('feature-x', new: true);
$project->pest('--filter=adds two numbers');
$result = $project->pest('--tia');
expect($result->replayed())->toBe(Project::TOTAL_TESTS, $result->describe())
->and($result->uncached())->toBe(0, $result->describe());
})->skipOnWindows();
```
Add the `--parallel --processes=2 --filter=…` variant as a second row (dataset), since the two write
paths are different code.
**Fix direction.** Make the fallback **per entry** rather than per baseline: in `baselineFor()`,
when the branch has its own baseline *and* a distinct fallback baseline exists, return
`results` = the branch's results **layered over** the fallback's (branch wins per test id), and take
`sha`/`tree` from the fallback when the branch's are `null`/empty. `baselineFor()` is the single
funnel for `recordedAtSha()`, `lastRunTree()`, `getResult()`, `getTime()`, `getAssertions()`,
`testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so one change covers every reader. An
alternative — never mint a key from a partial run — is smaller but loses the executed result
entirely, which regresses the parity just gained.
**Done when.** Both reproduction rows are green on both interpreters, and none of these move:
- `the branch that ran gets its own key and the default branch keeps its baseline` — writes stay on
the real branch; the merge must be **read-only** and must not leak into `ensureBaseline()`/`setResult()`.
- `a declared default branch that does not exist degrades to a full run` — with a fallback that names
nothing, a branch's own thin results must still be all you get.
- `a detached HEAD replays without minting a branch key`, `writes nothing on a second run on the same
branch`, `filtered mode finds nothing to do…` (both rows) — a merged read must not make a clean
replay start writing.
- The whole `PartialRunWriteTier.php` / `CompleteRunWriteTier.php` set — tier semantics are unchanged
by this fix.
---
### B2 — a partial run on detached HEAD writes into the default branch's baseline · **needs a decision**
**Symptom.** With `HEAD` detached, `Tia::resolveBranch()` (`Tia.php:~1756`) sets
`$this->branch = $changedFiles->currentBranch() ?? $this->fallbackBranch` — and that branch is used
for **writes**. A `--tia` run in this state happens to be harmless (a clean replay writes nothing),
but any run that *executes* tests writes their results into the default branch's baseline.
**Measured.**
```
seed on master; git checkout --detach; pest --filter="adds two numbers"
→ keys=[master] w=1 struct:ok ← master's baseline rewritten from a detached checkout
seed on master; git checkout --detach; pest --tia
→ keys=[master] w=0 ← read-only, as intended
```
**The decision.** `PLAN_PHASE_THREE.md` §2.3 **D3** recommended detached HEAD be *read-only*. If that
still stands, suppress writes when `currentBranch()` is `null` (a dedicated flag — note
`resultsOnlyWrites` is **not** enough, it still writes results). If Nuno prefers the current
behaviour, add a test pinning it and close this out.
**Reproduction** (`tests/Features/Tia/DefaultBranchWriteTier.php`), written for the read-only answer:
```php
test('a detached HEAD does not write into the default branch baseline', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->git()->detach();
$project->pest('--filter=adds two numbers');
$delta = $project->delta();
expect($delta->baselineUntouched('master'))->toBeTrue($delta->summary())
->and($project->branchKeys())->toBe(['master']);
})->skipOnWindows();
```
---
### B3 — an unreadable graph is never repaired on a machine with no coverage driver · **needs a decision**
**Symptom.** A corrupt `graph.json` is now *reported* (Part 2, change 2) but only *rebuilt* when a
coverage driver is present, because rebuilding means recording. Driverless, the file stays corrupt
run after run and TIA is silently inert until someone deletes it by hand — while the WARN claims
`it will be rebuilt`.
**Measured** (`php84`, no pcov):
```
overwrite graph.json with '{not json'
run 1: exit=0, 6 passed, file still '{not json'
run 2: exit=0, 6 passed, file still '{not json'
headline: "Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug"
```
**Options.** (a) delete the file when it cannot be decoded, so the next drivered run starts clean and
the state dir does not carry a permanent landmine; (b) keep the file but reword the WARN when no
driver is available. (a) is the honest one and costs one `State::delete()`.
**Reproduction** (`tests/Features/Tia/FilteredMode.php`, extending the existing corrupt-graph row):
```php
expect($result->output)->toContain('The dependency graph could not be read')
->and(file_get_contents($project->graphDir().'/graph.json'))->not->toBe('{not json');
```
Must pass on **both** interpreters — that is the whole point of the row.
---
### B4 — a complete `--parallel` run writes nothing and prunes nothing · **needs a decision**
**Symptom.** With a graph present and TIA not flagged, a sequential run refreshes results and applies
the prune; the same run under `--parallel` does neither, because the parent's `ResultCollector` is
empty (results live in the workers) and workers only flush when they were told to record, replay, or
— since Part 2's change — results-only for a *partial* run. So parallel CI contributes nothing to the
cache, and a deleted test's entry survives forever.
**Measured** (graph seeded on `master`, sentinelled):
```
pest → w=6 +0 -0 struct:ok (sequential baseline)
pest --parallel --processes=2 → w=0 +0 -0 struct:ok ← writes nothing
delete a test, then:
pest --parallel --processes=2 → w=0 +0 -0 struct:ok ← and does not prune (sequential gives -1)
pest --tia --parallel --processes=2 → w=0 ← correct: everything replayed
```
**The decision.** Extending the `TIA_RESULTS_ONLY` mechanism to complete parallel runs is
mechanically easy, but a *complete* run also prunes, and pruning from merged worker partials is the
risky half: a worker that dies, or a shard that never ran, would look like "these tests no longer
exist". If it is done, the prune must key off "every worker reported" and fall back to
results-only when it cannot prove that. `PLAN.md` §5 lists this as a known gap, not a regression.
**Reproduction** (`tests/Features/Tia/CompleteRunWriteTier.php`) — mirror the two sequential rows
that already exist (`a complete run prunes a deleted test`, `--no-tia refreshes results…`) with
`--parallel --processes=2` added, and assert the same deltas.
---
### B5 — G4 ("parallel replay clobbers cached `time`") no longer reproduces · **verify, then correct the record**
**Symptom.** `PLAN_PHASE_THREE.md` §4.6 lists as still-present: *"parallel replay clobbers cached
`time` on all non-executed tests — `mergeWorkerReplayPartials()` takes `$result['time']` verbatim,
never routing through `resultTime()`"*. The repo fixture disagrees: after a parallel replay the
sentinelled `time=9.999` / `assertions=42` survive on every non-executed test.
**Measured.** `a parallel run merges worker results into the parent baseline` edits one test file,
then runs `--tia --parallel --processes=2`: `2 affected, 4 replayed, w=2`. If replayed times were
being clobbered, `w` would be `6`.
**Why the record may be stale.** `flushWorkerReplay()` (`Tia.php:~1286`) already applies
`resultTime()` **worker-side** before writing the partial, so the parent's verbatim read is reading
values that were already corrected.
**What to do.** Either find a shape where it still reproduces (the playground has 25 tests and real
timings; the fixture has 6 and may be too small), or confirm it is fixed and strike it from §4.6.
Add a direct row either way:
```php
test('a parallel replay keeps the recorded time of tests that did not run', function (): void {
$project = Project::make('master');
$project->seed('master');
$project->pest('--tia', '--parallel', '--processes=2');
$delta = $project->delta();
expect($delta->writtenCount())->toBe(0, $delta->summary())
->and($delta->isResultsOnly())->toBeTrue($delta->summary());
})->skipOnWindows();
```
---
## Part 4 — Reporting
Per bug: **reproduced (yes/no)** with the measured delta, **fixed (yes/no/deferred)**, and the test
that now pins it. Close with:
1. Whether all `tests/Features/Tia/*` are green on **both** `php84` (no pcov) and `php85` (pcov).
2. Which of B2B5 still need Nuno's decision, phrased as a yes/no question each.
3. Whether `tests/.snapshots/success.txt` and the `tests/Visual/Parallel.php` tally now need
regenerating (they will, if you added rows) — **do not run `composer update:snapshots` unless
asked.**
4. Anything you found that is not in this file.
Leave the tree uncommitted and the scratch probes out of the repo.
+386
View File
@@ -0,0 +1,386 @@
# TIA deep audit — phase six
## Your task
Phase five went looking rather than fixing what was reported, and found nine defects in the
**read/write path and the state dir**. All nine are fixed and pinned by rows in the repo. It stopped
there deliberately: roughly half of TIA was never opened. Phase six is that other half.
Two deliverables, both required:
1. **A ranked findings list.** Every finding needs a *reproduction*, not a reading of the code. A
finding you cannot reproduce is a hypothesis — say so and rank it separately.
2. **New scenario tests** in `tests/Features/Tia/*`. Rows that pass go in the repo (they are the
regression net). Rows that fail stay in your scratchpad until the fix lands — **never commit a red
test.**
Work in passes, and **report between passes** rather than at the very end:
- **Pass A** — reproduce the leads in Part 3. Report which are real.
- **Pass B** — your own hunt: the invariants in Part 2, attacked with inputs nobody tried.
- **Pass C** — fixes, smallest first, each with the row that pins it.
Fix what is clearly a defect with an obvious correct answer. **Stop and ask** when the fix is a
behaviour *choice* — those are Nuno's calls, and a reproduction with a crisp yes/no question is worth
more than a guessed fix.
---
## Part 1 — Where things stand
### 1.1 What TIA is
Test Impact Analysis. `pest --tia` records a dependency graph (test file → source files it touched)
plus a per-branch baseline of results, then on later runs replays the tests whose dependencies did not
change. State lives in a per-project dir under `~/.pest/tia/`; `graph.json` is the whole thing.
Source of truth: `src/Plugins/Tia.php` (the plugin) and `src/Plugins/Tia/Graph.php` (the graph model +
read/write API). Supporting: `ChangedFiles.php` (git), `Fingerprint.php` (environment/structure
hashing), `Storage.php` / `FileState.php` (the state dir), `ResultCollector.php` (what a run observed),
`Recorder.php` + `CoverageCollector.php` (how edges are recorded).
Read `PLAN.md`, then `PLAN_PHASE_TWO.md``THREE``FOUR``FIVE` for the history and the tier
contract. Struck-through rows are fixed; do not re-report them. **Section 1.2 below supersedes any
phase-five row that contradicts it.**
### 1.2 What phase five settled — do not re-report these
Nine defects, all fixed, each with a row that fails if it comes back:
| # | Defect | Fix | Pinned by |
|---|---|---|---|
| 1 | `pruneStaleResults()` unset an entry; the next read layered the default branch's entry for the same test id back in, so a renamed/removed test stayed "previously unsuccessful" on a feature branch forever | `Graph::baselineFor()` layers per *file* once a branch has had a complete run (new `complete` flag on the baseline); a key minted by a narrowed run keeps the per-test-id merge | `StateReclamation`*a pruned result does not come back from the fallback*, *the fallback still reaches a branch that has never run a test file* |
| 2 | A cached failure whose test file was deleted was never reclaimed, so `--filtered` degraded to a full replay on every later run | `hasUnlocatedTestsToRerun()` widens only for a path it cannot *address*; `Graph::pruneResultsForMissingFiles()` + `pruneMissingTests()` run on every complete write | `StateReclamation`*a cached failure whose test file was deleted stops widening later runs*, *a complete run reclaims the entry and the edge of a deleted test file* |
| 3 | A detached HEAD is read-only for writes, but three paths still *deleted*: structural drift, `--fresh` (`Storage::purge`), and the corrupt-graph discard. The checkout that wiped the baseline could never rebuild it | `Tia::deleteState()` no-ops when detached; `Storage::purge` guarded | `StateReclamation` → the three *a detached HEAD does not purge…* rows |
| 4 | A status int outside 08 became `TestStatus::unknown()`, which `ReplayType` folded into `Failure` — a green test went red, exit 1 | `Graph::getResult()` returns `null` for an unknown status (re-run, don't replay); `shouldRerunStatus()` treats unknown as re-run | `HostileState`*a cached status this build cannot interpret is re-run, not replayed* |
| 5 | Notice/deprecation/warning (3/4/6) decode fine but `ReplayType` had no case, so they also folded into `Failure` | Explicit `Pass` cases — those statuses only reach replay when the configured `failOn*` / `displayDetailsOn*` policies say they do not matter | `HostileState`*a cached status with no replay of its own does not fail the run* |
| 6 | `Graph::decode()` took `baselines`, `edges` and `files` verbatim; one malformed entry raised a `TypeError` inside a test | `decodeBaselines()` / `decodeResults()` / `decodeEdges()` / `decodeFiles()` validate every field. Numeric-looking keys are cast, not filtered — a branch named `12345` decodes as an `int` | `HostileState`*a graph whose shape is wrong everywhere is repaired rather than trusted* |
| 7 | Baseline keys grew forever — one full copy of the suite per branch ever created | `ChangedFiles::branchNames()` (local + remote refs) + `Graph::pruneMissingBranches()`, on complete writes only, and only when the fallback branch is visible in the refs | `BranchShapes`*deleting many branches reclaims every one of their baselines* and the four rows around it |
| 8 | A test that triggered a deprecation was recorded as `status=0`, because PHPUnit emits `Passed` for it and TIA had no issue subscribers. A fresh `--fail-on-deprecation` run exited 1; the replayed one exited 0 | Six subscribers (`Notice`/`PhpNotice`/`Deprecation`/`PhpDeprecation`/`Warning`/`PhpWarning`) feed `ResultCollector`; most-important-status-wins; a plain `Passed` does not downgrade a triggered issue; `@`-suppressed issues are ignored | `IssueStatuses`*a triggered issue is recorded as itself, not as a pass*, *a cached deprecation still fails the run that asked to fail on one* |
| 9 | A replay wrote back the status it *looked* like from outside, so a cached deprecation replaying as a pass was persisted as `0` — defect 8's fix eroded after one run | `Tia::replayedAsRecorded()` writes back the cached status and message for replayed tests, in the sequential path and in the worker flush | `IssueStatuses`*replaying a cached issue does not downgrade it to a pass* |
| 10 | A run torn down mid-file (an `exit()` inside a test) still flushed what it had, and the parent read that as licence to prune the siblings it never reached. Sequential and parallel disagreed | `ResultCollector::hasUnfinishedTest()` demotes such a run to results-only, in `terminate()` (the shutdown path) and in `addOutput()` | `StateReclamation`*a run torn down mid-file does not prune the tests it never reached* |
**One existing assertion was changed.** `DefaultBranchWriteTier > filtered mode falls back to a full
replay when a cached failure cannot be located` used `tests/Unit/DeletedTest.php` — a path that
resolves but does not exist, which is precisely the shape behind defect 2. It now points at
`/build/agent/…`, so it still pins the widening safety net for the case where widening can help. If
you disagree with that reading, that is a finding, not a licence to change it back quietly.
### 1.3 Attacked in phase five and found solid — do not spend time here again
- **Parity.** Sequential vs `--processes=1/2/8` across replay-with-edit, `--filtered` with a cached
failure, a first run on a feature branch, `--bail`, `--stop-on-failure`, `--compact`: identical
tally *and* identical graph delta every time.
- **Hostile state dir.** Empty / truncated / not-JSON / JSON scalar / JSON list / `null` / `{}` / NUL
bytes; `graph.json` as a directory; a read-only state dir; dangling and negative edge ids; a result
`file` pointing outside the project; `schema: 2`. All degrade to "run the tests", exit 0.
- **Status int mapping.** `ResultCollector` (`asInt()`), `Graph::getResult()` and `TestStatus::from()`
agree exactly on 08. No off-by-one.
- **`FileState::write`** is tmp + rename, so concurrent runs cannot tear a file. Racing runs are
last-writer-wins **by design** — treat as a decision, not a bug, unless you can show data loss
worse than that.
- **Branch names.** Slashes, dots, unicode, digits-only, 180 characters, case-only differences,
remote-only branches, worktree branches: all keyed and reclaimed correctly.
- **`sha`/`tree` layering.** No shape found that under-reports changed files. The fallback `tree` only
drops a file whose current content hashes identically to what the fallback ran with, which is sound.
### 1.4 Known and deliberately unfixed
| Item | Why it is still open |
|---|---|
| Reclamation is skipped when a complete run executes **zero** tests | Stale edges linger until a run that executes at least one test. Writing the graph from a run that produced nothing costs more than it buys. Revisit if you find a real project stuck there. |
| SIGINT never reaches the suite (`PcovRestarter` re-exec) | Real, but signal handling outside the TIA read/write path. Needs its own pass. |
| Orphaned state dirs when the origin URL changes | `Storage::projectKey()` keys on the origin identity so clones share a graph. Changing the remote silently moves TIA to a fresh dir and nothing reclaims the old one. Unbounded growth in `~/.pest/tia`. |
| `environmental` fingerprint holds only `php_minor` | Nuno declined. The `clearResults()` drift path is therefore latent — it becomes live the moment that bucket grows. **Do not re-flag the `PHP_MAJOR_VERSION` line itself.** |
| PLAN.md §3 (raw coverage flags leave filtered mode on) | **No longer reproduces**`coverageReportActive()` consults `COVERAGE_REPORT_FLAGS` now. Struck. |
---
## Part 2 — The invariants to attack
Same list as phase five; 1, 2, 4, 5, 7 and 8 are now well covered, so weight your effort toward 3 and
6 and toward the *recording* half of the system, which nothing below the plugin has ever probed.
1. **Parity.** `pest <args>` and `pest --parallel --processes=N <args>` must leave the same graph and
reach the same tally. Hard rule. `Project::SEQUENTIAL_AND_PARALLEL` encodes it; use it on every new
write-path row.
2. **The tiers hold.** COMPLETE / RESULTS-ONLY / HARD-SUPPRESSED, exactly one each, no leaking.
3. **Replay is faithful.***weak spot.* A replayed test reports the same status, message, time and
assertion count as the recorded run. Phase five fixed statuses; **edges** are unproven.
4. **Reads never write.**
5. **A branch never corrupts another branch's baseline.**
6. **Nothing is unbounded.***weak spot.* Baseline keys are reclaimed now; `files`, `edges`, worker
partials, coverage caches and orphaned state dirs are not.
7. **A hostile state dir cannot break a run.**
8. **Git shapes are all handled.**
---
## Part 3 — Leads to reproduce first (Pass A)
Ranked by expected value. Everything here is **unexplored**, not merely unfixed — phase five never
opened these files.
### M1 — the recording path with a real coverage driver · **highest value**
Everything phase five did was driver-independent by design (seed a graph, never record one). Nothing
verified that a *recorded* graph is correct. This is the biggest blind spot in the audit.
- Do the recorded edges match what the test actually touched? Record with pcov, then hand-check
`edges` against the source files each fixture test uses.
- `PLAN.md` §4: **`pest --tia --coverage` narrows edges** — `Feature/ExampleTest` recorded with 2
files instead of 16, dropping self-edges. Same observable shape as the parallel bug G12 that phase
four closed, likely a different mechanism (the piggyback collector is scoped by `phpunit.xml
<include>`, the pcov-restarted recorder is not). **Confirm whether these are one fix or two.**
- `Recorder::activateLinkTracking()` (piggyback) vs `activate()` (pcov restart) must produce the same
edge set for the same suite. Compare them directly.
- `keepExisting: $this->piggybackCoverage` in `replaceEdges()` — what happens to a test whose edges
genuinely shrank while piggybacking?
Rows for this **must** be `->skipOnPhpVersionsWithoutCoverage()`-style guarded, or seeded, or CI goes
red on `php84`. That constraint is why phase five skipped it; solve it deliberately rather than by
accident. Adding a coverage-driver guard helper to the fixture is a legitimate deliverable.
### M2 — `BaselineSync` (621 lines, never opened)
The remote-baseline fetch is the only path where a graph arrives from **another machine**, which is
exactly where the hostile-state work matters most and where none of it has been exercised.
- A fetched baseline whose `fingerprint` matches but whose `files`/`edges` describe a different tree.
- A fetched baseline recorded on a branch this checkout does not have.
- `fetchIfAvailable()` under a broken network, a 404, a truncated download, a non-gzip body.
- `KEY_FETCH_COOLDOWN` — does it bound retries, and does a corrupt cooldown file break a run?
- Interaction with defect 7's branch GC: a fetched baseline carries branch keys this clone has never
heard of. **They will be pruned on the next complete write.** Is that right, or must fetched keys be
exempt? This is a real question, answer it.
### M3 — selection paths nobody has probed
`Graph::affected()` is ~600 lines and phase five only exercised the plain PHP-edge path.
- **Migrations** → `TableExtractor``testTables` intersection. What happens with an unparseable
migration, a migration that drops a table, a squashed schema dump?
- **Blade** — `bladeAncestorsFor()` walks `@include`/`@extends`/`<x-*>` transitively. Cycles?
Depth? A component referenced only dynamically?
- **Inertia** — `componentForInertiaPage()`, `jsFileToComponents`, `JsModuleGraph::buildStrict()`.
What if `vite` is missing, or the resolver returns garbage?
- **`usesSiblingHeuristicForUnknownPhp()`** — a hard-coded list of Laravel directories. A changed file
in `app/Providers/` widens to every test whose deps share that directory. Measure how much that
over-selects on a real tree.
- **Arch tests** — `testSourceDeclaresArchGroup()` greps the source with three regexes. False
positives (the string `arch(` in a comment) select the file on *every* PHP source change.
### M4 — git shapes phase five left alone
- A repo with **no commits yet** (`currentSha()` returns null / git fails).
- **Submodules** — a changed file inside one; `git status --porcelain` reports the submodule path.
- A repo whose root is **above** the pest project — `TiaRequiresRepositoryRoot` panics deliberately;
confirm it still panics and writes nothing.
- **Shallow / single-branch CI checkouts.** Defect 7's guard (`fallbackBranch` must be visible in the
refs) was reasoned about, not measured. Build one and check nothing is over-pruned.
- A branch **behind** the recorded sha, so `merge-base --is-ancestor` fails and the graph is declared
unreachable. Does it recover, or thrash?
### M5 — a real concurrent race
Phase five established `FileState::write` is atomic and called it last-writer-wins by design. Nobody
launched two runs. Launch them: a watcher plus a manual run, two `--tia` processes on one project,
`--parallel` where the parent writes while a straggler worker flushes. Look for **loss worse than
last-writer-wins** — a partially-merged baseline, a pruned entry from a run that never saw the file,
worker partials from run A consumed by run B (`KEY_WORKER_*` are not namespaced per run).
That last one is the sharpest: `purgeWorkerPartials()` deletes *all* partials by prefix, so two
concurrent parallel runs will eat each other's. Probe it.
### M6 — the unbounded remainder
Defect 7 reclaimed baseline keys. These still grow:
- `files` and `edges` — a deleted *source* file's entry is never removed (`pruneMissingTests()` only
covers test files). Every rename leaves an orphan id forever.
- Orphaned state dirs under `~/.pest/tia` (see 1.4).
- `KEY_COVERAGE_CACHE` / `KEY_COVERAGE_MARKER` — who deletes them, and when?
- Worker partials when a worker dies before `terminate()`.
Measure the growth on a realistic tree before proposing anything. Then **ask** — a cap or a GC is a
behaviour choice.
---
## Part 4 — The harness
### 4.1 The scenario suite
`tests/Features/Tia/*` scaffold a throwaway git project into a temp dir, run a **real `pest`
subprocess** against it, and diff the graph it wrote. Everything lives in `tests/Fixtures/Tia/`:
| Class | What it gives you |
|---|---|
| `Project` | `make(branch, overlay:)`, `withoutGit()`, `seed(branch, sentinel:, failing:)`, `seedFor($root, …)`, `pest(...$args)`, `pestWithEnvironment($dir, $env, ...$args)`, `pestIn($dir, ...)`, `write($rel, $contents)`, `path($rel)`, `graph()`, `branchKeys()`, `graphDir()`, `graphExists()`, `snapshot()`, `delta()`, `mutateGraph(fn)`, `addBaseline($branch)`, `worktree($branch)`, `destroy()`, `destroyAll()`, `SEQUENTIAL_AND_PARALLEL` |
| `GitRepo` (`$project->git()`) | `switchTo($b, new:)`, `rename($from, $to)`, `detach()`, `commit($msg)`, `config($k, $v)`, `addOrigin()`, `removeOrigin()`, `setOriginHead($b)`, `unsetOriginHead()`, `worktree()`, `sha()`, `branchNames()`, `run([...])` for anything else |
| `GraphDelta` (`$project->delta()`) | `writtenCount()`, `added()`, `removed()`, `branchKeys()`, `baselineUntouched($b)`, `shaMoved()`, `treeMoved()`, `edgesMoved()`, `filesMoved()`, `fingerprintMoved()`, `structureMoved()`, `isResultsOnly()`, `isHardSuppressed()`, `summary()` |
| `PestResult` (returned by `pest()`) | `replayed()`, `uncached()`, `affected()`, `tally()`, `output`, `exitCode`, `describe()` |
The fixture app is 3 test files / **6 tests** (`Project::TOTAL_TESTS`); `Project::EDGES` and
`Project::TESTS` describe the graph `seed()` writes. `Project::testId($file, $description)` builds a
result key. Overlays in `tests/Fixtures/Tia/overlays/<name>/` supply a different `tests/Pest.php`
that is how you configure `pest()->tia()->…` for a scenario.
**If the fixture cannot express your case, extend the fixture.** A new `GitRepo` verb, a new overlay,
or a coverage-driver guard is a legitimate part of the deliverable. Do not water down a scenario to
fit the current helpers. Phase five added `GitRepo::commit()` usage, numeric-key handling in
`Project::branchKeys()` and `GraphDelta`, and used `$project->write()` to author test files inline —
follow that pattern.
### 4.2 The sentinel discriminator — read before writing any assertion
`seed()` writes a graph **and then rewrites every cached result to `time=9.999`, `assertions=42`**
(only where assertions are non-zero — risky/skipped/incomplete statuses are *derived* from "performed
no assertions", so falsifying those would rewrite the status on replay), then snapshots. Therefore:
- `$delta->writtenCount()` is the only reliable way to tell **"replayed"** from **"executed and wrote
back the same values"**. `0` means nothing was written.
- `isResultsOnly()` = no structure moved, no `sha`/`tree` movement, nothing added or removed.
- `isHardSuppressed()` = the graph is byte-identical.
- A replayed suite reports inflated assertion counts (6 tests × 42). That is the sentinel, not a bug.
The three write tiers, unchanged since phase two:
- **COMPLETE** — may change everything.
- **RESULTS-ONLY** — may change only `baselines[<branch>].results` for tests that ran; never removes
an entry, never adds a result for a test file absent from `edges`, never touches
`sha`/`tree`/`edges`/`files`/`fingerprint`.
- **HARD-SUPPRESSED** — may change nothing.
One addition from phase five: a complete run on a **non-default** branch also sets
`baselines[<branch>].complete = true`. It is deliberately *not* set on the default branch, so a clean
green run there stays byte-identical.
### 4.3 Running them
They are in the `integration` group (`tests/Pest.php`), so `composer test:unit` skips them. **A
directory argument finds nothing** — pass files, space-separated, as separate argv entries:
```bash
PAO_DISABLE=1 php84 bin/pest tests/Features/Tia/BranchShapes.php tests/Features/Tia/CompleteRunWriteTier.php \
tests/Features/Tia/DefaultBranchReplay.php tests/Features/Tia/DefaultBranchResolution.php \
tests/Features/Tia/DefaultBranchWriteTier.php tests/Features/Tia/FilteredMode.php \
tests/Features/Tia/HostileState.php tests/Features/Tia/IssueStatuses.php \
tests/Features/Tia/PartialRunWriteTier.php tests/Features/Tia/StateReclamation.php
PAO_DISABLE=1 php bin/pest <same list>
```
**Both interpreters must be green.** `php84` is 8.4.x with **no pcov** — that is what CI has
(`.github/workflows/tests.yml` sets `coverage: none`). `php` is 8.5.x with pcov — that is the dev
machine. Any assertion that depends on a coverage driver passes locally and fails in CI. Concretely:
- A **cold recording run writes no graph at all** without pcov/xdebug (it prints `Running in TIA mode,
however TIA is skipped as it needs ext-pcov or Xdebug`). **Seed a graph, never record one**, unless
the driver *is* the point of the row — see M1, which has to solve this properly.
- A **PHP source file edit** driverless triggers `Detected PHP source changes but no coverage driver
is available` → full suite, `affected=0`. Edit *test* files, not `app/` files.
- The `terminate()` path differs by driver: with pcov the plugin reaches the complete write through
the shutdown handler, without it the run exits earlier. Defect 10 only reproduced on `php`. **Run
every probe on both before believing it.**
`PAO_DISABLE=1` is mandatory on every pest invocation and every probe: `laravel/pao` emits JSON under
agents and corrupts the captured output.
**Baseline: 161 scenario rows green on both interpreters**, at `b49ba062`. Per file:
| File | Rows |
|---|---|
| `StateReclamation.php` | 37 |
| `HostileState.php` | 25 |
| `CompleteRunWriteTier.php` | 17 |
| `DefaultBranchResolution.php` | 14 |
| `BranchShapes.php` | 14 |
| `DefaultBranchReplay.php` | 13 |
| `IssueStatuses.php` | 13 |
| `PartialRunWriteTier.php` | 12 |
| `DefaultBranchWriteTier.php` | 10 |
| `FilteredMode.php` | 6 |
Plus 80 unit/arch rows (`tests/Unit/Plugins/Tia/*`, `tests/Arch.php`). If those numbers do not
reproduce, stop and say so.
### 4.4 Measure before you assert
Never guess an expectation from reading the code. Probe first, in your scratchpad:
```php
<?php
require '/Users/nunomaduro/Work/projects/pestphp/pest/vendor/autoload.php';
use Tests\Fixtures\Tia\Project;
$p = Project::make('master');
$p->seed('master');
$p->git()->switchTo('feature-x', new: true);
$r = $p->pest('--tia');
printf("tally=[%s] keys=[%s] %s\n", $r->tally(), implode(',', $p->branchKeys()), $p->delta()->summary());
$p->destroy();
```
`PAO_DISABLE=1 php probe.php`. One project per scenario; always `destroy()`; `Project::destroyAll()`
at the end. Run every probe on **both** interpreters before believing it.
---
## Part 5 — Ground rules
- **Do not** run `composer test`. It takes minutes and you do not need it.
- **`tests/.snapshots/success.txt` and the tally in `tests/Visual/Parallel.php` are stale right now** —
phase five added 89 rows and did not regenerate them. **Report that they need regenerating and let
Nuno run `composer update:snapshots`.** Do not run it yourself.
- **Do not commit.** Leave the tree dirty.
- **Do not touch the playground's `vendor/`.** If a finding truly needs the playground (25 real tests,
real timings), say so and ask — syncing it is a manual step Nuno owns.
- Run `vendor/bin/phpstan analyse <the files you touched> --memory-limit=-1 --no-progress` and
`vendor/bin/pint <the files you touched>` before reporting. **Scope both to files you changed** —
Nuno edits `src/` live, and a repo-wide fixer will revert his work.
- The fixture projects **hardlink `src/`**. If you edit `src/` while a scenario run is in flight,
those rows go red for no reason. Finish the edit, then run.
- Keep scratch probes out of the repo.
- **Never weaken an existing assertion to make something pass.** If an existing row contradicts your
fix, that is a finding: report the contradiction and ask. Phase five hit this once (see 1.2) and
rewrote the row to pin the *narrower* contract rather than deleting it — that is the bar.
---
## Part 6 — Reporting
**Between passes**, not just at the end. Per finding:
- **Reproduced (yes / no / hypothesis only)**, with the exact command and the measured
`tally` + `delta()->summary()` on **both** interpreters.
- **Severity**, and say why in one line. The scale that matters here: *a test wrongly replayed as
passing* (worst) > *cache silently useless, full suite forever* > *graph grows / stale data* >
*cosmetic*.
- **Fixed / deferred / needs-a-decision**, and the row that pins it.
- For anything needing a decision: **one yes/no question**, no essay.
Close with:
1. The count of `tests/Features/Tia/*` green on **both** `php84` (no pcov) and `php` (pcov), against
the 161 baseline.
2. Every row you added, and what invariant from Part 2 it defends.
3. Which findings are still open, as yes/no questions.
4. That the snapshots need regenerating — **do not regenerate them.**
5. **What you looked at and found solid.** A list of attacks that did not break anything is a real
result: it tells the next phase where not to spend its time.
---
## Part 7 — Open questions carried into this phase
Answer these before or during Pass C; they change what the fixes should be.
1. A fetched remote baseline carries branch keys this clone has never heard of, and the branch GC will
prune them on the next complete write. **Should fetched keys be exempt?**
2. Two concurrent parallel runs share the `worker-edges-*` / `worker-results-*` prefixes and
`purgeWorkerPartials()` deletes by prefix. **Should partials be namespaced per run, or is "do not
run two TIA suites at once" the contract?**
3. `files` and `edges` never lose a deleted *source* file. **Cap, GC, or accept?**
4. Orphaned state dirs accumulate under `~/.pest/tia` whenever a project's origin URL changes.
**Reclaim them, or accept?**
+455
View File
@@ -0,0 +1,455 @@
# TIA default-branch fallback — phase three
## Your task
Fix [pestphp/pest#1823](https://github.com/pestphp/pest/issues/1823) — TIA's cached results never
hit for repos whose default branch is not literally `main` — then re-run the **whole** conformance
matrix against the playground app, plus the new section **L** that covers the fix.
Work in this order, and **stop where the plan says stop**:
1. **Part 1** — read the diagnosis. It is already measured; do not re-derive it, but do re-confirm
the two reproductions in Part 1.3 take ~2 minutes and prove your environment is sane.
2. **Part 2** — implement the fix in the `pestphp/pest` repo. **Do not commit. Do not touch the
playground's `vendor/`.**
3. **HARD STOP → Part 3.** Report the diff to Nuno and wait. He validates, commits, and syncs it
into the playground himself. You must not proceed until he confirms.
4. **Part 4** — verify the sync landed, then run section **L** (new) and the full phase-two matrix
(**AK**, all 156 rows) against the playground.
5. **Part 5** — report in the given format.
Per `CLAUDE.md`: **do not write new `pestphp/pest` unit tests and do not run `composer test`.** Make
the change, report it, and ask whether repo tests should be added. The section-L rows are *playground
invocations*, not repo tests — those are the deliverable and are always in scope.
---
## Part 1 — The diagnosis (already measured; commit `bfd5b756`)
### 1.1 Root cause — two independent hardcoded `'main'` literals
**(a) The read fallback.** `src/Plugins/Tia/Graph.php` — seven methods default the fallback to the
literal `'main'`:
| line | method |
|---|---|
| 579 | `recordedAtSha(string $branch, string $fallbackBranch = 'main')` |
| 614 | `getAssertions(…, string $fallbackBranch = 'main')` |
| 625 | `getTime(…, string $fallbackBranch = 'main')` |
| 636 | `getResult(…, string $fallbackBranch = 'main')` |
| 663 | `testFilesToRerun(string $branch, string $fallbackBranch = 'main')` |
| 700 | `hasUnlocatedTestsToRerun(string $branch, string $fallbackBranch = 'main')` |
| 811 | `lastRunTree(string $branch, string $fallbackBranch = 'main')` |
They all funnel into `Graph::baselineFor()` (line 819), which *does* implement a real cross-branch
fallback:
```php
if (isset($this->baselines[$branch])) return $this->baselines[$branch];
if ($branch !== $fallbackBranch && isset($this->baselines[$fallbackBranch])) return $this->baselines[$fallbackBranch];
return ['sha' => null, 'tree' => [], 'results' => []];
```
The mechanism is deliberate. The problem is that **no caller ever passes `$fallbackBranch`** — all
nine call sites in `src/Plugins/Tia.php` (lines 419, 429, 432, 785, 857, 1026, 1031, 1048, 1056) pass
only `$this->branch`. So on a `master`-named repo the second branch of `baselineFor()` can never
fire, and the whole mechanism is dead code.
`grep -rniE 'defaultBranch|symbolic-ref|init\.defaultBranch|origin/HEAD' src/` returns **nothing**
no default-branch resolution exists anywhere.
**(b) The detached-HEAD default.** `src/Plugins/Tia.php:206` declares `private string $branch = 'main';`
and `ChangedFiles::currentBranch()` (`ChangedFiles.php:208`) returns `null` for detached HEAD. So on
detached HEAD `$this->branch` stays the literal `'main'` and is used for **both reads and writes**
minting a baseline key for a branch that does not exist. This is a *write*-side bug and a separate fix
from (a).
### 1.2 Not a regression
`git log -S"fallbackBranch = 'main'"` bottoms out at `c7e32f5d feat(tia): continues to work on poc`.
This is original PoC code, untouched by phase one. Phase one's change 1 modified
`hasUnlocatedTestsToRerun()`'s file-existence check — one of the seven methods — without going near
the fallback. Do not report it as a phase-one regression.
No test anywhere exercises the mechanism: `tests/Unit/Plugins/Tia/Graph.php` uses `'main'` as the
*actual* branch name, so those assertions pass whether or not the fallback exists. A branch-name
mismatch is never tested.
### 1.3 The two reproductions (re-confirm these before you start)
Both on the playground, `master`-named default branch, zero local changes:
```
default = master default = main
1. record on default → full run (cold) → full run (cold)
2. 1st run on feature-x → 25 UNCACHED → 25 replayed ← (a)
3. 2nd run on feature-x → 25 replayed → 25 replayed
4. back on default → 25 replayed → 25 replayed
5. 1st run on feature-y → 25 UNCACHED → 25 replayed
```
```
master-only graph, then `git checkout --detach`, then `pest --tia`:
→ 25 uncached, and keys become [master,main] ← (b) spurious key
```
The cost is **one full run per new branch, forever**, with no output explaining why — the only clue
is the `N uncached` count; the headline is just `─ Experimental TIA mode enabled.`
### 1.4 Correction to phase two
Phase two reported **H6 and H7 as passing. Both were false passes.** They ran after H5, which had
renamed `master``main` and left a `main` key in the graph, so the hardcoded fallback resolved by
accident. Re-measured against a clean `master`-only graph, both fail. Section L replaces them as the
load-bearing rows; H6/H7 must be re-run **from a cold graph** this time (see Part 4.2).
---
## Part 2 — The fix to implement
### 2.1 Recommended design
**Step 1 — add a non-throwing resolver** to `src/Plugins/Tia/ChangedFiles.php`, next to
`currentBranch()`:
```php
public function defaultBranch(): ?string
```
Resolution order, each step failing soft to the next:
1. `git symbolic-ref --short refs/remotes/origin/HEAD` → strip a leading `origin/`
2. `git config --get init.defaultBranch`
3. `null`
Unlike `currentBranch()`, this must **never throw** `MissingDependency` — it is advisory. Return
`null` on any non-zero exit or empty output.
**Step 2 — add a config surface.** `src/Plugins/Tia/Configuration.php` already exposes `always()`,
`locally()`, `filtered()`, `baselined()`, `watch()`. Add:
```php
public function defaultBranch(string $branch): self
```
so `pest()->tia()->defaultBranch('master')` works in `tests/Pest.php`. Explicit config **always
wins** over autodetection — that is the escape hatch when `origin/HEAD` is unset.
**Step 3 — resolve once, in `Tia.php`.** The read path is hot (`getResult()` is called per test at
line 419), so resolution must not shell out per call. Resolve alongside `$this->branch` at
`Tia.php:1910`, under the existing `$branchResolved` guard:
```php
$this->fallbackBranch = $configuredDefaultBranch
?? $changedFiles->defaultBranch()
?? 'main';
```
**Step 4 — thread it into `Graph`.** Prefer a `Graph`-level property over editing nine call sites:
add `Graph::setFallbackBranch(string $branch)`, change the seven signatures to
`?string $fallbackBranch = null`, and resolve inside each with
`$fallbackBranch ??= $this->fallbackBranch;`. `baselineFor()` itself needs no change. This keeps the
public signatures backward-compatible and minimises blast radius.
**Step 5 — fix the detached-HEAD write.** `Tia.php:206`'s `= 'main'` default must become the
resolved default branch, so detached HEAD stops minting a phantom key.
### 2.2 Invariants the fix must not break
These are all covered by existing matrix rows — the fix is wrong if any of them moves:
- **Read-only.** The fallback must affect *reads* only. Writes go through `ensureBaseline($branch)`
and must keep using the real current branch. Otherwise H1H4/H8 ("no baseline key other than the
real branch") break.
- **H9** — a non-git dir with `--tia` must still raise
`MissingDependency: The feature "Tia mode" requires "git".` Adding a soft resolver must not
swallow that.
- **H10** — plain `pest` in a non-git dir must still run and create no baseline dir.
- **A2/A3** — cold-graph recording unchanged.
- **I1/E3** — clean+green `--tia --filtered` must still be a true zero-delta run.
- Filtered mode reads `testFilesToRerun()` and `hasUnlocatedTestsToRerun()`, so the fallback must
reach those two as well, not just `getResult()`.
### 2.3 Decisions for Nuno (raise these at the Part 3 stop)
- **D1** — Consult `origin/HEAD` at all? It requires `git remote set-head` and is absent in many CI
checkouts and all remote-less repos. Config + `init.defaultBranch` only is simpler but helps fewer
people out of the box. *Recommendation: keep it, first in the chain, since it fails soft.*
- **D2** — Single-key heuristic: if the graph holds exactly one baseline key, use it as the fallback?
Fixes the issue with zero git calls, but is implicit and surprising when several keys exist.
*Recommendation: no.*
- **D3** — Should detached HEAD write a baseline at all, or be read-only? Current behaviour mints a
key. *Recommendation: read-only.*
- **D4** — Should `pest()->tia()->defaultBranch()` validate that the branch exists, or accept any
string? *Recommendation: accept any string; a nonexistent name degrades to a full run, which is
safe.*
---
## Part 3 — HARD STOP
When the code is written:
1. Show Nuno the diff (`git -C /Users/nunomaduro/Work/projects/pestphp/pest diff`) and a one-paragraph
summary of each file's change.
2. Answer/raise the D1D4 decisions.
3. State explicitly that you have **not** committed and have **not** synced `vendor/`.
4. Ask whether repo unit tests should be added (per `CLAUDE.md`), describing the tests you have in
mind — do not write them yet.
5. **Wait.** Nuno commits and applies the change to the playground.
**Never sync the playground's `vendor/` yourself.** `vendor/pestphp/pest` there is a dist copy, not a
symlink (composer installed `dev-fix/tia-filtered as 5.2.0`), so pest-repo edits do not reach it. Say
what is stale and wait. This applies to before/after contrasts too.
Once he confirms, verify the sync actually landed before measuring anything:
```bash
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
grep -c 'defaultBranch' "$V/src/Plugins/Tia/ChangedFiles.php" # must be ≥ 1
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php src/Plugins/Tia/ChangedFiles.php \
src/Plugins/Tia/Configuration.php; do
diff -q "/Users/nunomaduro/Work/projects/pestphp/pest/$f" "$V/$f" >/dev/null \
&& echo "SAME $f" || echo "STALE $f"
done
```
If anything reports `STALE`, stop and tell him. State in your final report which pest commit produced
the playground numbers.
---
## Part 4 — Measurement
### 4.1 Environment and traps
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel` (branch `master`).
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON when
it detects an agent.
2. **Pin the interpreter to `php85`.** The playground requires PHP `>= 8.4.1` *and* pcov. Locally only
`php85` (8.5.8) has both — `php84` (8.4.23) has no pcov, and the bare Herd `php` shim has been
observed drifting to 8.3.32 mid-session, which kills every run in
`vendor/composer/platform_check.php`. Also put a `php``php85` symlink first on `PATH`: the
`--shard` list-tests probe spawns a subprocess via bare `php`, not `PHP_BINARY`.
3. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`. Scope
resets to `git checkout -- tests app` / `git clean -fd tests app`. For `phpunit.xml` and
`composer.lock`, copy aside and copy back, verifying with `shasum`.
4. **zsh does not word-split unquoted parameters.** A `$PEST` string containing a space becomes one
command name. Route every invocation through `eval` (the `pest()` helper below does this).
5. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
6. **The sentinel technique must not falsify `assertions` on zero-assertion tests.** A
risky/skipped/incomplete status is *derived* from "performed no assertions", so patching those to
`42` rewrites the status on replay and destroys the discriminator — it shows up as a phantom
`status 5→0` defect. The `sentinel.php` below only patches `assertions` where it is already
non-zero. With that, a full replay gives a clean `rewritten=0`.
7. Restore branch state after every L row. Several rows rename or detach; leaving a stray `main` key
in the graph is exactly what produced phase two's false H6/H7 passes.
### 4.2 Reference numbers
The playground already carries the phase-two fixtures at commit `fb2e77e`**do not rebuild them.**
A healthy sequential `--tia` graph is:
> **`files=27`, 10 edge keys (one per test file), self-edge on all 10, `n=25` results.**
`sha` will differ once Nuno commits the vendor sync — re-derive it once and use it throughout. The
`files`/`edges`/`n` numbers hold as long as no test fixture changes. Suite shape: 10 test files, 25
tests, including six deliberate status fixtures (skipped, todo, incomplete, risky, warning,
deprecation), a 3-row dataset, a `smoke` group, an env-driven flaky test (green unless
`FLAKY_FAIL=1`), and the annotation set (`covers`/`note`/`flaky`/`issue`/`pr`/`ticket`/`assignee`).
**Re-run H6 and H7 from a cold graph** (`rm -rf` the graph dir, record on `master` only, *then*
branch/detach). Their phase-two results are void.
### 4.3 Harness
Write these to your scratchpad. `$SP` is your own scratchpad dir.
<details>
<summary><code>lib.sh</code></summary>
```bash
#!/bin/zsh
export PAO_DISABLE=1
PG=/Users/nunomaduro/Work/projects/playground/laravel
SP="<your scratchpad>"
PHPBIN="php85"
PEST="$PHPBIN $PG/vendor/bin/pest"
cd "$PG" || exit 1
pest() { eval "$PEST $*"; } # zsh: no word-splitting, must eval
GRAPHDIR="$(pest --baseline)"
GRAPH="$GRAPHDIR/graph.json"
mkdir -p "$SP/bin" && ln -sf "$(command -v php85)" "$SP/bin/php"
export PATH="$SP/bin:$PATH" # --shard spawns bare `php`
reset_tree() { git checkout -- tests app 2>/dev/null; git clean -qfd tests app 2>/dev/null; }
seed() { rm -rf "$GRAPHDIR"; pest --tia >/dev/null 2>&1; $PHPBIN "$SP/sentinel.php" "$GRAPH" >/dev/null; cp "$GRAPH" "$SP/before.json"; }
snap() { cp "$GRAPH" "$SP/before.json"; }
delta() { $PHPBIN "$SP/cmp.php" "$SP/before.json" "$GRAPH"; }
keys() { $PHPBIN -r '$g=json_decode(file_get_contents($argv[1]),true);echo "[".implode(",",array_keys($g["baselines"]??[]))."]";' "$GRAPH"; }
tally() { sed -E $'s/\x1b\\[[0-9;]*[a-zA-Z]//g' <"$SP/out.txt" | grep -E 'Tests:' | sed -E 's/^ +//;s/Tests: +//'; }
```
</details>
<details>
<summary><code>sentinel.php</code> — the write discriminator</summary>
```php
<?php // sentinel.php <graph.json>
$p = $argv[1];
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR);
$n = 0;
foreach ($g['baselines'] ?? [] as $br => $b) {
foreach (array_keys($b['results'] ?? []) as $id) {
$g['baselines'][$br]['results'][$id]['time'] = 9.999;
// Only falsify a non-zero assertion count: risky/skipped/incomplete are
// DERIVED from "performed no assertions", so patching those to 42 would
// rewrite the status on replay and destroy the discriminator.
if ((int) ($b['results'][$id]['assertions'] ?? 0) > 0) {
$g['baselines'][$br]['results'][$id]['assertions'] = 42;
}
$n++;
}
}
file_put_contents($p, json_encode($g, JSON_THROW_ON_ERROR));
echo "sentinelled $n results\n";
```
</details>
<details>
<summary><code>oneline.php</code> — one compact tier verdict per row</summary>
```php
<?php // oneline.php <before.json> <after.json>
function load(string $p): ?array {
return is_file($p) ? json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR) : null;
}
$a = load($argv[1]); $b = load($argv[2]);
if ($a === null || $b === null) { echo 'GRAPH '.($b === null ? 'DELETED' : 'CREATED'); exit; }
function edgeSets(array $g): array {
$out = [];
foreach ($g['edges'] ?? [] as $t => $ids) { $s = array_map(fn($i) => $g['files'][$i] ?? "?$i", (array) $ids); sort($s); $out[$t] = $s; }
ksort($out); return $out;
}
$moved = [];
if (edgeSets($a) !== edgeSets($b)) $moved[] = 'edges';
if (($a['files'] ?? []) !== ($b['files'] ?? [])) $moved[] = 'files';
if (($a['fingerprint'] ?? null) !== ($b['fingerprint'] ?? null)) $moved[] = 'fingerprint';
$brA = array_keys($a['baselines'] ?? []); $brB = array_keys($b['baselines'] ?? []);
if ($brA !== $brB) $moved[] = 'branchkeys('.implode('|', $brA).'->'.implode('|', $brB).')';
$add = $rem = $wr = 0; $shaMoved = $treeMoved = false;
foreach ($brB as $br) {
$ra = $a['baselines'][$br]['results'] ?? []; $rb = $b['baselines'][$br]['results'] ?? [];
if (($a['baselines'][$br]['sha'] ?? null) !== ($b['baselines'][$br]['sha'] ?? null)) $shaMoved = true;
if (($a['baselines'][$br]['tree'] ?? null) !== ($b['baselines'][$br]['tree'] ?? null)) $treeMoved = true;
$add += count(array_diff(array_keys($rb), array_keys($ra)));
$rem += count(array_diff(array_keys($ra), array_keys($rb)));
foreach ($ra as $id => $x) {
if (! isset($rb[$id])) continue;
foreach (['status','time','assertions','message'] as $f) {
if (($x[$f] ?? null) !== ($rb[$id][$f] ?? null)) { $wr++; break; }
}
}
}
$n = 0; foreach ($brB as $br) $n = max($n, count($b['baselines'][$br]['results'] ?? []));
printf('n=%d w=%-2d +%d -%d %s%s%s', $n, $wr, $add, $rem,
$moved === [] ? 'struct:ok' : 'STRUCT:'.implode(',', $moved),
$shaMoved ? ' sha:CHANGED' : '', $treeMoved ? ' tree:chg' : '');
```
</details>
`cmp.php` (verbose per-entry version of the same) and `summarise.php` / `edgediff.php` are in
`PLAN_PHASE_TWO.md` §"Graph summariser" — reuse them for drill-downs. Reading the verdict:
- `w=` — entries actually **written**. Under sentinel patching this is the only reliable way to tell
"wrote identical values" from "wrote nothing". A full replay must give `w=0`.
- `struct:ok` + `+0 -0` — no prune, no edges/files/fingerprint movement. Required by RESULTS-ONLY.
- `STRUCT:branchkeys(...)` — a new baseline key appeared. For section L this is the headline signal.
### 4.4 Section L — new rows for this fix
Tiers, unchanged from phase two: **COMPLETE** may change everything · **RESULTS-ONLY (RO)** may
change only `baselines[<branch>].results` for tests that ran, and must never remove an entry, add a
result for a test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Every L row starts from a **cold graph recorded on the named default branch only** — verify
`keys=[<default>]` before branching. Restore branch state afterwards.
| # | Case | Target outcome |
|---|---|---|
| L1 | default `master`, record, `git switch -c feature-x`, `pest --tia`, zero changes | **all 25 replayed** (`w=0`), not `25 uncached`. The headline fix. |
| L2 | as L1 but default `main` | still all replayed — regression guard, this already worked |
| L3 | default `trunk`, then `develop` | replayed for both; the fix must not special-case two names |
| L4 | L1, then a *second* new branch `feature-y` | replayed too — the toll must not return per branch |
| L5 | L1 then edit `app/Services/Calculator.php` on `feature-x` | narrows to the 2 affected files (`CalculatorTest` + `AnnotationsTest`, which `covers` it); the other 17 replay |
| L6 | L1 with `--tia --filtered` | filtered mode reads the fallback too (`testFilesToRerun`, `hasUnlocatedTestsToRerun`) → `No affected tests found`, zero delta |
| L7 | L1 with `--tia --parallel` | fallback works in workers as well as the parent |
| L8 | L1, then `pest --tia` twice on `feature-x` | idempotent; second run also `w=0` |
| L9 | detached HEAD on a `master`-only graph | replays, and **no `main` key minted**`keys` stays `[master]`. Bug (b). |
| L10 | `pest()->tia()->defaultBranch('master')` in `tests/Pest.php`, repo default renamed away | config wins over autodetect |
| L11 | config set to a nonexistent branch (`defaultBranch('nope')`) | degrades to a full run; no crash, no phantom key |
| L12 | no remote at all (`git remote remove origin` if present) | still resolves (via `init.defaultBranch`) or degrades safely — must not throw |
| L13 | branch name with a slash (`feature/x/y`) | replayed; no key-splitting bugs |
| L14 | L1, then confirm writes | `feature-x` gets its **own** key; the `master` key is **not** written to (fallback is read-only) |
| L15 | git worktree on a new branch (the issue's scenario) | replays from the default-branch baseline |
| L16 | non-git dir, `pest --tia` | still `MissingDependency: The feature "Tia mode" requires "git".` — the soft resolver must not swallow it |
| L17 | non-git dir, plain `pest` | runs normally; no baseline dir created |
| L18 | count `git` subprocesses during one `--tia` run | default-branch resolution is cached, not one call per test. Probe by shimming `git` on `PATH` to a logging wrapper. |
L10L11 need a `tests/Pest.php` edit — that file is tracked and **outside** the `tests app` reset
scope in practice (it lives in `tests/`, so `git checkout -- tests` does restore it; verify with
`git status` after).
For L16/L17, build a throwaway non-git project — and note the trap that burned phase two: a
**symlinked** `vendor` makes Pest resolve the project root back to the playground (identical baseline
hash), silently invalidating the test. Use a hardlinked copy:
```bash
NG="$SP/nogit"; rm -rf "$NG"; mkdir -p "$NG"
cp -R composer.json composer.lock phpunit.xml artisan tests app bootstrap config routes resources storage "$NG/"
[ -f .env ] && cp .env "$NG/"
cp -Rl vendor "$NG/vendor" || cp -R vendor "$NG/vendor"
```
Confirm the baseline path differs (`nogit-<hash>`, not `laravel-4a455a95622ac0ec`), and delete both
the temp project and its `~/.pest/tia/nogit-*` dir afterwards.
### 4.5 Re-run the phase-two matrix (AK, 156 rows)
Re-run every row of `PLAN_PHASE_TWO.md` Part 2 against the fixed build. No row's status is trusted
until re-measured — the fix touches `Graph`'s read path, which nearly every row exercises. Sections
**A, B, H, I** are the load-bearing ones here (H is branch-key resolution; I is filtered mode; both
consume the changed methods directly). **H6 and H7 must be re-derived from a cold graph** (Part 1.4).
Most rows batch cheaply — phase two ran C1C20 in one call at roughly one line of output each. Use
`oneline.php` for the sweep and `cmp.php` only to drill into anomalies.
### 4.6 Known pre-existing failures — do not report as regressions
| item | status |
|---|---|
| ~~**G4 / G4b** — parallel replay clobbers cached `time` on all non-executed tests.~~ | **Struck in phase four — does not reproduce.** `flushWorkerReplay()` applies `resultTime()` worker-side before writing the partial, so the parent's verbatim read of `$result['time']` is reading values that were already corrected. Pinned by `a parallel replay keeps the recorded time of tests that did not run` (`tests/Features/Tia/CompleteRunWriteTier.php`), which sentinels every cached `time` and asserts a parallel replay writes nothing. |
| **C19**`--tia --uses=…` cannot be fixtured. TIA hard-errors on PHPUnit classes (`EnsureTiaIsRunningPestTestsOnly`), and Pest has no chainable `->uses()`. | **Expected behaviour per Nuno.** Verify the tier (`w=0`, RO, notice) and move on. Not a defect. |
| **J11**`--repeat` is not a Pest option (`Unknown option "--repeat"`). | **Don't care per Nuno.** Mark SKIP. |
| **J10**`--random-order-seed` alone exits 1 with a WARN. Identical without `--tia`. | Pre-existing Pest behaviour, unrelated. Tier still holds. |
---
## Part 5 — Reporting
Per row: **tier respected (yes/no)**, the **graph delta** under sentinel patching, and for any failure
the **pre-fix contrast** so a regression is told apart from a pre-existing defect. For a pre-fix
contrast you need `db70017c` (or `bfd5b756` for "before phase three") files swapped into the
playground's `vendor/` — that is a sync, so **ask Nuno first** and always restore afterwards.
State which pest commit produced the numbers. Close with:
1. Whether L1L18 all pass (the fix works).
2. Whether AK regressed anywhere relative to phase two's 154/156.
3. The D1D4 decisions as implemented.
4. Anything still open — including G4, which will still be failing.
Leave the playground on `master` with only the four user-modified files dirty, no stray branches, and
no leftover `~/.pest/tia/*` dirs beyond `laravel-4a455a95622ac0ec`.
+466
View File
@@ -0,0 +1,466 @@
# TIA write-tier conformance — phase two
## Your task
**Re-run all 156 rows of the matrix in Part 2 from scratch, against the playground app** at
`/Users/nunomaduro/Work/projects/playground/laravel`. Every row, including the ones already marked
VERIFIED or PASS — the point of phase two is that no row's status is trusted until it has been
re-measured against the current code. The `Phase 1` column is prior evidence and a hint at what to
watch, never a reason to skip a row.
These are **not** the `pestphp/pest` repo's own tests (`composer test`) — do not run those. Each row is
a `pest` invocation against the playground's suite, followed by a diff of the TIA graph it wrote.
Report, per row: tier respected (yes/no), the graph delta under sentinel patching, and — for any
failure — the pre-fix contrast so a regression is told apart from a pre-existing defect. Do **not**
stop at reading this file or summarising it; the deliverable is executed results.
Work in this order: **Part 1 (environment traps) → Part 1b (build the fixtures — the playground has
none of them) → Part 2 (the matrix) → Part 3 (priorities)**. The matrix is too large for one context;
take it one lettered section at a time and report as you finish each. Sections A, B, G, I and K are
the load-bearing ones — do those first if you run short.
Phase one implemented `PLAN.md` Part 1 items 14 plus three §5 items. This file turns the matrix into
a conformance check rather than a bug list.
Code under test: `pestphp/pest` at `/Users/nunomaduro/Work/projects/pestphp/pest`, branch
`fix/tia-filtered`, commit **`bfd5b756`** or later. Verify with
`grep -c 'recordsEdgesInWorkers\|recordsEdges' src/Plugins/Tia.php` → at least 3 hits. Pre-fix
baseline for every contrast is **`db70017c`**.
---
## Part 0 — What phase one changed
| # | Change | Files |
|---|---|---|
| 1 | `hasUnlocatedTestsToRerun()` stats the file, so a deleted test file is "unlocated" | `src/Plugins/Tia/Graph.php` |
| 2 | `enterReplayMode()` uses `activateLinkTracking()` under piggyback coverage | `src/Plugins/Tia.php` |
| 3 | `enterReplayMode()` stamps `TIA_PIGGYBACK_COVERAGE` for workers | `src/Plugins/Tia.php` |
| 4 | `replaceEdges(…, keepExisting:)` — piggyback edges seed empty sets, never overwrite populated ones | `Graph.php`, `Tia.php` |
| 5 | `renderFreshGraph()` stops claiming "fresh graph" when the graph is kept; reason reworded to `recording a coverage baseline` | `Tia.php` |
| 6 | `COVERAGE_REPORT_FLAGS` + `coverageReportActive()` union over `originalArguments`; new `pestCoverageActive()` keeps the coverage-cache marker/hijack on Pest's own `--coverage` | `Tia.php` |
| 7 | `Tia::recordsEdgesInWorkers()` + `WrapperRunner::handleTia()` inject `-d pcov.directory=<root>` into worker argv | `Tia.php`, `src/Plugins/Parallel/Paratest/WrapperRunner.php` |
| 8 | Sequential record runs announce structural drift via `renderFreshGraph()` | `Tia.php` |
| 9 | `Graph::getTime()` + `cachedTimeByTestId` + `resultTime()` preserve replayed durations; edge-less write guard is now `$recordsEdges = $complete && ($markKnownTestFiles \|\| $this->recordingActive)` | `Graph.php`, `Tia.php` |
Deliberately **not** done: `PLAN.md` §5 SIGINT propagation, §5 warning/deprecation `status=0`
mapping, and all of §6 (playground annotation fixtures). The vacuous C rows below stay vacuous.
### Target-outcome changes this forces
Two rows in the original matrix asserted the **old**, buggy behaviour. Their targets are updated
below — do not report them as regressions:
- **B10** was "`time` differs; statuses stable". Change 9 means replayed entries now **keep** their
recorded `time`. New target: `time` differs only for tests that actually executed.
- **B12** gains an edge-preservation assertion it never had (see K1).
### One known-failing repo test
`tests/Unit/Plugins/Tia/Graph.php:69-76` asserts `hasUnlocatedTestsToRerun('main')` is `false` for
`tests/Feature/FooTest.php` under `new Graph(sys_get_temp_dir())` — a path that does not exist. That
assertion encodes the I9 bug and **will fail** under change 1. It needs re-pointing at a root/file
that exists (e.g. `dirname(__DIR__, 4)` + `'tests/Unit/Plugins/Tia/Graph.php'`). Left untouched by
request; it is a repo-test matter, not a playground one.
---
## Part 1 — Environment, and the traps in it
**Playground:** `/Users/nunomaduro/Work/projects/playground/laravel`
```bash
cd /Users/nunomaduro/Work/projects/playground/laravel
GRAPH="$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)/graph.json" # ~/.pest/tia/laravel-4a455a95622ac0ec
```
1. **Every** pest invocation needs `PAO_DISABLE=1` — the app has `laravel/pao`, which emits JSON
when it detects an agent.
2. **Never run `git checkout .` or `git checkout -- composer.lock` in the playground.** It has four
pre-existing user-modified files — `AGENTS.md`, `CLAUDE.md`, `composer.json`, `composer.lock`
that a blanket reset would destroy. `PLAN.md`'s "reset ritual" is unsafe as written. Scope resets
to what you touched: `git checkout -- tests app` / `git clean -fd tests app`, and for
`composer.lock` (needed for the drift rows) **copy it aside and copy it back**, verifying with
`shasum`.
3. **`vendor/pestphp/pest` is a dist copy, not a symlink.** Composer installed
`dev-fix/tia-filtered as 5.2.0`, so edits in the pest repo do **not** reach the playground.
**Tell Nuno whenever a sync is needed to move forward — do not sync silently.** Say what is stale
and what the sync would be, then wait. This includes temporarily swapping in `db70017c` files for a
before/after contrast. When you report any playground result, state which commit produced it.
The sync itself, once he agrees:
```bash
PEST=/Users/nunomaduro/Work/projects/pestphp/pest
V=/Users/nunomaduro/Work/projects/playground/laravel/vendor/pestphp/pest
for f in src/Plugins/Tia.php src/Plugins/Tia/Graph.php \
src/Plugins/Parallel/Paratest/WrapperRunner.php; do cp "$PEST/$f" "$V/$f"; done
```
Verify with `grep -c recordsEdgesInWorkers "$V/src/Plugins/Tia.php"` → `1`. **Always restore the
current version before continuing** after a pre-fix contrast.
4. **Coverage driver:** pcov only, no xdebug. `ini_get('pcov.directory')` is `''` by default — that
emptiness is the entire mechanism behind G12.
5. **Suite shape as found:** 7 tests in 5 files — `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,
TrioTest}.php`, `tests/Feature/ExampleTest.php`. A healthy sequential graph is **`files=22`,
5 edge keys, self-edge on every test** (`PLAN.md`'s `files=25` is stale). Pre-fix parallel gives
`files=4` with zero self-edges. **These numbers shift the moment you add the Part 1b fixtures** —
re-derive them once, after the fixtures land, and use the new numbers throughout. The invariants
that do *not* shift: sequential and parallel must agree, and every test must have a self-edge.
6. **Sentinel patching is the only reliable discriminator** between "wrote identical values" and
"wrote nothing": rewrite every cached result to `time=9.999 assertions=42`, snapshot, run the
case, diff. A canary test absent from `edges` is unreliable under `--filter` because it never
matches the filter and so never runs.
7. Seeding a cached failure needs `--fresh` (or an env-driven flaky fixture): `--tia` on a clean
green tree replays rather than executes, so it can never cache a failure. Working recipe — break
an assertion, `pest --tia --fresh`, then restore the source.
8. Comment-only edits to a test file are **not** changes (AST-level hashing). Use semantic edits.
9. The shell is zsh — build commands with arrays or `eval`; unquoted `$args` does not word-split.
### Graph summariser
Write this to a scratch path and use it for every diff.
```php
<?php // summarise.php <graph.json> [label]
$g = json_decode((string) file_get_contents($argv[1]), true, 512, JSON_THROW_ON_ERROR);
$files = $g['files'] ?? []; $edges = $g['edges'] ?? [];
echo ($argv[2] ?? $argv[1])."\n files=".count($files).' edges keys='.count($edges)."\n";
$dirs = []; foreach ($files as $f) { $dirs[explode('/', (string) $f)[0]] = true; }
ksort($dirs); echo ' file dirs: '.implode(', ', array_keys($dirs))."\n";
ksort($edges);
foreach ($edges as $test => $ids) {
$self = 'no';
foreach ((array) $ids as $id) { if (($files[$id] ?? null) === $test) { $self = 'YES'; break; } }
echo sprintf(" %-42s n=%-3d self=%s\n", $test, count((array) $ids), $self);
}
foreach ($g['baselines'] ?? [] as $branch => $b) {
$r = $b['results'] ?? [];
echo " baseline[$branch]: n=".count($r).' sha='.substr((string) ($b['sha'] ?? '-'), 0, 7)."\n";
foreach ($r as $id => $x) {
echo sprintf(" %-58s status=%d time=%s asserts=%d file=%s\n", substr((string) $id, -58),
$x['status'], $x['time'], $x['assertions'], $x['file'] ?? '-');
}
}
```
Normalised edge-set equality (for G12 / I5 / K1):
```php
<?php // edgediff.php <a.json> <b.json>
function edges(string $p): array {
$g = json_decode((string) file_get_contents($p), true, 512, JSON_THROW_ON_ERROR); $out = [];
foreach ($g['edges'] as $t => $ids) { $s = array_map(fn ($i) => $g['files'][$i], $ids); sort($s); $out[$t] = $s; }
ksort($out); return $out;
}
$a = edges($argv[1]); $b = edges($argv[2]);
echo $a === $b ? "IDENTICAL edge sets\n" : "DIFFER\n";
foreach ($a as $t => $s) {
$m = array_diff($s, $b[$t] ?? []); $e = array_diff($b[$t] ?? [], $s);
if ($m || $e) printf(" %s: -%d +%d\n", $t, count($m), count($e));
}
```
### Per-case loop
```bash
git checkout -- tests app 2>/dev/null; git clean -qfd tests app
rm -rf "$(PAO_DISABLE=1 ./vendor/bin/pest --baseline)"
PAO_DISABLE=1 ./vendor/bin/pest --tia >/dev/null 2>&1 # seed a healthy graph
# sentinel-patch every result to time=9.999 assertions=42, snapshot, run the case, diff
```
---
## Part 1b — Fixtures you must build first
**The playground has none of the fixtures the matrix depends on.** Verified inventory: the only test
files are `tests/Unit/{CalculatorTest,ExampleTest,GreeterTest,TrioTest}.php` and
`tests/Feature/ExampleTest.php` — 7 plain tests, zero occurrences of `->group()`, `->only()`,
`->skip()`, `->todo()`, `->note()`, `->flaky()`, `->covers()`, `->uses()`, `->issue()`, `->pr()`,
`->ticket()`, `->assignee()`, datasets, or any `FLAKY_OK`-style env hook. `phpunit.xml` defines only
the `Unit` and `Feature` testsuites.
So roughly 30 rows below cannot run as written until you create the fixtures. Build them **all up
front, in one commit**, then re-derive the baseline graph shape once and use those numbers for the
whole sweep — every fixture you add changes `files`, the `edges` key count and `n`, so adding them
piecemeal invalidates earlier rows.
| Fixture to create | Rows that need it |
|---|---|
| a test with `->group('smoke')` | C7, C8, C9, C10 |
| an env-driven flaky test (passes iff `FLAKY_OK=1`) | E1, E2, E3, E4 |
| a `->skip()`ed test | E9, D8 |
| a `->todo()` test | E10, C23, C39 |
| a risky test (no assertions; pair with `--disallow-test-output`) | E11, D7 |
| a test that triggers a PHPUnit warning | D6 |
| a test calling `markTestIncomplete()` | D9 |
| a test that triggers a deprecation | D10 |
| a dataset test with ≥3 rows | E13, E14 |
| `->only()` — added and removed per case, not left in | C31, C32, G7 |
| `->covers(App\Services\Calculator::class)` | C18 |
| `->uses(...)` / `UsesClass` annotation | C19 |
| `->note(...)` | C24 |
| `->flaky()` **annotation** (distinct from the env-driven flaky test above) | C25 |
| `->issue(123)`, `->pr(1)`, `->ticket('X')`, `->assignee('X')` | C26, C27, C28, C37, C38 |
This is `PLAN.md` §6's "test-harness gaps to close before re-running", now itemised: without these,
the listed rows' graph-invariant assertions match zero tests and **prove nothing** — they pass
vacuously. Any row still marked "(vacuous)" in Part 2 is vacuous *only because* its fixture is
missing; once you add the fixture, treat the row as unverified and make it load-bearing.
Rows needing an *action* rather than a fixture — breaking `trio one` for the D rows, an uncommitted
test edit for `--dirty`, branch renames and a non-git dir for H5H10, `--mutate` against
`app/Services` for the F rows — are fine as written; `pestphp/pest-plugin-mutate` is installed.
---
## Part 2 — Full case matrix
Tiers: **COMPLETE** may change everything · **RESULTS-ONLY** (RO) may change only
`baselines[<branch>].results` for tests that ran, and must never remove an entry, add a result for a
test file absent from `edges`, or alter `sha`/`tree`/`edges`/`files`/`fingerprint` ·
**HARD-SUPPRESSED** may change nothing.
Phase-1 column: **VERIFIED** = re-run against the fixed code in the phase-one session, pre-fix
contrast captured · **PASS (sweep)** = passed in the original `db70017` sweep and *not* re-checked
since the changes — these are the bulk of phase two's work · **SKIP** = not runnable as written.
### A — Setup & sanity
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| A1 | `composer show pestphp/pest` | version `dev-fix/tia-filtered` | PASS (sweep) |
| A2 | `pest --baseline` | prints an existing dir; exit 0 | VERIFIED |
| A3 | delete graph, `pest --tia` | graph.json created; one `edges` key per test file; one result per test | VERIFIED |
| A4 | `git status` after reset ritual | clean *except the four user-modified files* (see trap 2) | VERIFIED |
| A5 | `extension_loaded("pcov")` | `true` | VERIFIED |
| A6 | delete graph, plain `pest` | no graph created | PASS (sweep) |
| A7 | delete graph, `pest --filter=adds` | no graph created | PASS (sweep) |
| A8 | two consecutive `pest --tia` | second replays everything; `sha` unchanged | VERIFIED |
### B — COMPLETE runs still write
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| B1 | `pest --tia` (clean) | replays; graph written; `sha` = `git rev-parse HEAD` | VERIFIED |
| B2 | `pest --no-tia` | results refreshed; prune applied; `sha`/`tree`/`edges` unchanged | PASS (sweep) — recheck under change 9 |
| B3 | `pest` (plain) | same as B2 | PASS (sweep) — recheck under change 9 |
| B4 | `pest --tia --fresh` | graph purged and rebuilt; `edges` rebuilt; `files` repopulated | VERIFIED |
| B5 | `pest --bail`, all green | COMPLETE — graph written, prune applied | PASS (sweep) |
| B6 | edit `Calculator.php`, `pest --tia` | `CalculatorTest` re-runs, others replay; `tree` updated | PASS (sweep) |
| B7 | delete `trio three`, `pest --tia` | its result pruned; `trio one`/`two` remain | PASS (sweep) |
| B8 | delete `GreeterTest.php`, `pest --tia` | result and edges survive (missing-file prune is `--fresh`-only) | PASS (sweep) |
| B9 | B8 then `pest --tia --fresh` | entry gone; `files` drops `Greeter.php` too | PASS (sweep) |
| B10 | `pest --tia` twice | **UPDATED TARGET:** `time` differs only for tests that executed; replayed entries keep their recorded `time`; statuses stable | VERIFIED |
| B11 | add a test file, `pest --tia` | new `edges` key **and** new result appear in the **same** run | VERIFIED (regression guard for change 9) |
| B12 | `pest --tia --coverage` | completes; graph written; coverage report prints | VERIFIED |
### C — Selection narrowing → RESULTS-ONLY
All rows: RO invariants hold. "Notice" = `TIA does not apply to partial runs — running the selected tests directly.`
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| C1 | `pest --filter="adds numbers"` | RO; no notice | PASS (sweep) |
| C2 | `pest --tia --filter="adds numbers"` | RO; notice | PASS (sweep) |
| C3 | `pest --filter="trio one"` | RO; `trio two`/`three` byte-identical | PASS (sweep) |
| C4 | `pest --filter="trio"` | RO; all three update; nothing else does | PASS (sweep) |
| C5 | `pest --exclude-filter="Feature"` | RO; no notice; no prune | PASS (sweep) |
| C6 | `pest --tia --exclude-filter="Feature"` | RO; notice | PASS (sweep) |
| C7 | `pest --group=smoke` | RO; no notice | PASS (sweep) |
| C8 | `pest --tia --group=smoke` | RO; notice | PASS (sweep) |
| C9 | `pest --exclude-group=smoke` | RO; no notice; no prune | PASS (sweep) |
| C10 | `pest --tia --exclude-group=smoke` | RO; notice | PASS (sweep) |
| C11 | `pest tests/Unit` | RO; no notice; Feature entry not pruned | PASS (sweep) |
| C12 | `pest --tia tests/Unit` | RO; notice | PASS (sweep) |
| C13 | `pest tests/Unit/TrioTest.php` | RO; no notice | PASS (sweep) |
| C14 | `pest --tia tests/Unit/TrioTest.php` | RO; notice | PASS (sweep) |
| C15 | `pest --testsuite=Unit` | RO; no notice | PASS (sweep) |
| C16 | `pest --tia --testsuite=Unit` | RO; notice | PASS (sweep) |
| C17 | `pest --tia --exclude-testsuite=Feature` | RO; notice | PASS (sweep) |
| C18 | `pest --tia --covers='App\Services\Calculator'` | RO; notice | PASS (sweep) |
| C19 | `pest --tia --uses='App\Services\Calculator'` | RO; notice | PASS (vacuous — needs fixture) |
| C20 | `pest --tia --test-suffix=Test.php` | RO even though all tests run; `edges`/`files`/`n` unchanged | PASS (sweep) |
| C21 | `pest --dirty` with an uncommitted test edit | RO; no notice | PASS (sweep) |
| C22 | `pest --tia --dirty` | RO; notice | PASS (sweep) |
| C23 | `pest --tia --todos` | RO; notice | PASS (vacuous) |
| C24 | `pest --tia --notes` | RO; notice | PASS (vacuous) |
| C25 | `pest --tia --flaky` | RO; notice | PASS (vacuous) |
| C26 | `pest --tia --issue=123` | RO; notice | PASS (vacuous) |
| C27 | `pest --tia --pr=1` | RO; notice | PASS (vacuous) |
| C28 | `pest --tia --pull-request=1` | RO; notice | PASS (vacuous) |
| C29 | `pest --tia --shard=1/2` | RO; notice; no prune | PASS (sweep) |
| C30 | `pest --tia --shard=2/2` | RO; notice; C29+C30 covers the suite; neither prunes | PASS (sweep) |
| C31 | `->only()` on `trio one`, `pest --tia` | RO; no notice; siblings untouched | PASS (sweep) |
| C32 | `->only()` on `trio one`, plain `pest` | RO; no notice | PASS (sweep) |
| C33 | `pest --tia --filtered --filter=adds` | RO; notice; filtered yields to narrowing | PASS (sweep) |
| C34 | `PEST_TIA=1 pest --filter=adds` | RO; notice | PASS (sweep) |
| C35 | `PEST_TIA_FILTERED=1 pest --filter=adds` | RO; notice (regression guard) | PASS (sweep) |
| C36 | `pest --filtered --filter=adds` | RO; notice | PASS (sweep) |
| C37 | `pest --tia --ticket=X` | RO; notice. `--ticket` **is** a real option (`src/Plugins/Snapshot.php:83`) — it narrows legitimately | PASS (vacuous) |
| C38 | `pest --tia --assignee=X` | RO; notice (`src/Plugins/Snapshot.php:81`) | PASS (vacuous) |
| C39 | `pest --tia --todo` | RO; notice; matches nothing | PASS (vacuous) |
### D — Truncation → RESULTS-ONLY
D1D5, D11D13, D16 precondition: `trio one` broken. D6D10 run against a green suite so the `--stop-on-*` trigger is the only narrowing.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| D1 | `pest --bail` | RO; `trio one` = `status=7`; siblings survive unchanged | PASS (sweep) |
| D2 | `pest --retry` | RO; siblings survive | PASS (sweep) |
| D3 | `pest --stop-on-failure` | RO | PASS (sweep) |
| D4 | `pest --stop-on-defect` | RO | PASS (sweep) |
| D5 | `pest --stop-on-error` (no error occurs) | COMPLETE — nothing stopped it | PASS (sweep) |
| D6 | `pest --stop-on-warning` | RO when it fires. Warning stored as `status=0`, not `6` | PASS (sweep) |
| D7 | `pest --stop-on-risky --disallow-test-output` | RO; `status=5` | PASS (sweep) |
| D8 | `pest --stop-on-skipped` | RO; `status=1` | PASS (sweep) |
| D9 | `pest --stop-on-incomplete` | RO; `status=2` | PASS (sweep) |
| D10 | `pest --stop-on-deprecation` | RO. Deprecation stored as `status=0`, not `4` | PASS (sweep) |
| D11 | `pest --tia --bail` | RO | PASS (sweep) |
| D12 | `pest --bail --filter=trio` | RO (narrowed *and* truncated) | PASS (sweep) |
| D13 | `stopOnFailure="true"` in `phpunit.xml`, plain `pest` | RO — caught via `stoppedEarly()`, not flag matching | PASS (sweep) |
| D14 | D13 config, green suite | COMPLETE | PASS (sweep) |
| D15 | `pest --bail`, last test in run order fails | RO — over-conservative but safe; nothing was skipped | PASS (sweep) |
| D16 | `pest --tia --filtered --bail`, broken affected test | RO | PASS (sweep) |
### E — Result merge semantics
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| E1 | cached `flaky`=7, `FLAKY_OK=1 pest --filter=flaky` | flips to `status=0`; all other results byte-identical | PASS (sweep) |
| E2 | cached `flaky`=0, `pest --filter=flaky` (env unset) | flips to `status=7` | PASS (sweep) |
| E3 | after E1, `pest --tia --filtered` clean | `No affected tests found`; zero graph delta | PASS (sweep) |
| E4 | after E2, `pest --tia --filtered` | re-runs, `from 1 previously unsuccessful test` | PASS (sweep) |
| E5 | any partial run | `assertions` and `time` update for the test that ran (it executed, so change 9 does not apply) | PASS (sweep) |
| E6 | any partial run | `message` of untouched tests unchanged | PASS (sweep) |
| E7 | two sequential partial runs on different tests | each updates only its own entry; both persist | PASS (sweep) |
| E8 | `pest --filter="trio one"` | siblings keep exact `status`/`time`/`assertions`/`message` | PASS (sweep) |
| E9 | partial run of a `->skip()`ed test | `status=1` | PASS (sweep) |
| E10 | partial run of a `->todo()` test | `status=1`, `assertions=0`, `message="__TODO__"` | PASS (sweep) |
| E11 | partial run of a risky test | `status=5` | PASS (sweep) |
| E12 | any partial run | `fingerprint` byte-identical | PASS (sweep) |
| E13 | dataset test, `--filter` matching one row | that row updates; other rows survive (the old prune bug) | PASS (sweep) |
| E14 | dataset test, full `--tia` after deleting a row | the deleted row is pruned | PASS (sweep) |
### F — Guard rails
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| F1 | test absent from `edges`, `pest --filter="brand new"` | no result recorded | VERIFIED |
| F2 | complete `pest --tia` first, then the same filter | result is recorded | VERIFIED |
| F3 | `pest --mutate --path=app/Services --covered-only` | statuses/`edges`/`sha`/`tree`/`files` identical; only `time` may change | PASS (sweep) |
| F4 | `pest --mutate --parallel --path=… --covered-only` | same (observed: zero delta) | PASS (sweep) — recheck under change 7 |
| F5 | weakened test so a mutant survives, `pest --mutate` | no status change in the baseline | PASS (sweep) |
| F6 | `pest --mutate` with the graph deleted | no graph created | PASS (sweep) |
| F7 | mutation subprocess env | carries `PEST_MUTATION_TESTING`; `--mutate` stripped from argv | PASS (sweep) |
| F8 | grep baseline after `--mutate` | no mutation-flavoured messages | PASS (sweep) |
### G — Parallel
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| G1 | `pest --tia --parallel` (clean) | COMPLETE — graph written; all 7 results present | VERIFIED |
| G2 | `pest --tia --parallel --filter=adds` | RO | PASS (sweep) |
| G3 | `pest --parallel --bail`, `trio one` broken | RO; siblings survive | PASS (sweep) |
| G4 | `pest --tia --parallel --bail` | RO | PASS (sweep) |
| G5 | `pest --tia --filtered --parallel` after a source edit | narrows to affected; writes | VERIFIED (see I5) |
| G6 | `pest --tia --parallel` | worker results reach the parent baseline | VERIFIED |
| G7 | `->only()` + `pest --tia --parallel` | RO | PASS (sweep) |
| G8 | `pest --tia --parallel --shard=1/2` | RO | PASS (sweep) |
| G9 | broken test in one worker, `pest --parallel --bail` | whole run is RO, not just that worker's slice | PASS (sweep) |
| G10 | `pest --parallel --retry` | `InvalidOption`; graph untouched | PASS (sweep) |
| G11 | `pest --tia --parallel --fresh` | graph rebuilt | VERIFIED |
| G12 | `edges` after `pest --tia` vs `pest --tia --parallel --fresh` | **equivalent edge sets** — `files=22`, self-edge on all 5 tests, both | **VERIFIED (fixed)** — pre-fix: `files=4`, 0 self-edges |
### H — Baseline key / branch resolution
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| H1 | `pest --tia` on `master` | only a `master` key | VERIFIED |
| H2 | plain `pest` on `master` | writes `master`, not `main` | PASS (sweep) |
| H3 | `pest --bail` truncated on `master` | no `main` key minted | PASS (sweep) |
| H4 | `pest --filter=adds` on `master` | no `main` key minted | PASS (sweep) |
| H5 | `git branch -m main`, full `pest --tia` | single `main` key | PASS (sweep) |
| H6 | `git checkout -b feature/x`, `pest --tia` | `feature/x` key appears; reads fall back to the existing baseline | PASS (sweep) |
| H7 | detached HEAD, `pest --tia` | falls back to the `main` key for both reads and writes; no `HEAD` key | PASS (sweep) |
| H8 | after every C and D case | no baseline key other than the real branch | PASS (sweep) |
| H9 | non-git dir, `pest --tia` | `MissingDependency` — `The feature "Tia mode" requires "git".` | PASS (sweep) |
| H10 | non-git dir, plain `pest` | runs normally; no baseline dir created | PASS (sweep) |
### I — Filtered mode
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| I1 | clean + green, `pest --tia --filtered` | `No affected tests found`; zero graph delta | PASS (sweep) |
| I2 | edit `Calculator.php` | only `CalculatorTest` runs | VERIFIED |
| I3 | cached failure, clean tree | re-runs it, `from 1 previously unsuccessful test` | PASS (sweep) |
| I4 | I2 with sentinels | `tree` updated; exactly one entry rewritten; all others retained | PASS (sweep) |
| I5 | `pest --tia --filtered --parallel` | same narrowing as I2 **and `edges` unchanged** | **VERIFIED (fixed)** — 1 affected test, edges identical |
| I6 | `PEST_TIA_FILTERED=1 pest --tia` | identical delta to I4 | PASS (sweep) |
| I7 | `pest --tia --filtered tests/Unit` | explicit path wins; filtered off; RO; notice | PASS (sweep) |
| I8 | `pest --tia --filtered --coverage-text` | filtered mode disabled by an active coverage report → full suite runs | **VERIFIED (fixed)** — pre-fix: `No affected tests found` |
| I9 | cached failure whose test file was deleted | WARN `Some cached tests due a re-run could not be located on disk` + `Running the full suite with replay instead of a filtered run` | **VERIFIED (fixed)** — pre-fix: `No tests found`, exit 0 green |
| I10 | `pest --tia --filtered` with no baseline yet | records a baseline instead of filtering | PASS (sweep) |
| I11 | edit a Blade view a Feature test renders | the Feature test is selected | PASS (sweep) |
| I12 | edit `composer.lock` | fingerprint drift → full rebuild, **and the reason is printed sequentially**: `fresh graph (composer.lock changed)` | **VERIFIED (fixed)** — pre-fix: bare `Running in TIA mode.` |
### J — Interactions & regressions
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| J1 | `pest --tia --fresh --filter=adds` | graph **not** purged; RO | PASS (sweep) |
| J2 | `pest --tia --refetch --filter=adds` | refetch not performed; RO | PASS (sweep) |
| J3 | `pest --no-tia --filter=adds` | RO — narrowing wins over `--no-tia`; no notice | PASS (sweep) |
| J4 | `pest --tia --no-tia` | TIA disabled; COMPLETE | PASS (sweep) |
| J5 | `<groups><exclude><group>integration</group></exclude></groups>`, `pest --tia` | COMPLETE, no notice — an always-in-force filter applies to baseline runs too | PASS (sweep) |
| J6 | `pest --tia --compact` | COMPLETE | PASS (sweep) |
| J7 | `pest --tia -v` | COMPLETE | PASS (sweep) |
| J8 | `pest --tia --profile` | COMPLETE | PASS (sweep) |
| J9 | `pest --tia --order-by=random` | COMPLETE | PASS (sweep) |
| J10 | `pest --tia --random-order-seed=1234` | COMPLETE | PASS (sweep) |
| J11 | `pest --tia --repeat=2` | COMPLETE — repetition is not narrowing | **SKIP** — `--repeat` is not a Pest option |
| J12 | `pest --filter=adds` twice | only the executed test's `time` differs; rest byte-identical | PASS (sweep) |
| J13 | `pest --tia --min=50` | COMPLETE (silent no-op without `--coverage`) | PASS (sweep) |
| J14 | SIGINT mid-suite | RO — an interrupted run is truncated | PASS (sweep) — signal still does not reach the re-exec'd child (not fixed) |
| J15 | delete the graph, `pest --tia --filtered` | records a baseline rather than erroring | PASS (sweep) |
| J16 | corrupt `graph.json`, `pest --tia` | recovers by rebuilding; no crash | PASS (sweep) |
### K — New rows for the phase-one fixes
These assert behaviour no original row covered. All were verified in phase one; re-run them as
regression guards.
| # | Case | Target outcome | Phase 1 |
|---|---|---|---|
| K1 | healthy `pest --tia` graph, then `pest --tia --coverage` | `edges` **byte-identical** — piggyback data may seed empty sets, never narrow populated ones; `files=22` stays `22` | **VERIFIED (fixed)** — pre-fix: `ExampleTest` 16→2 edges, self-edges lost |
| K2 | same run's headline | `Experimental TIA mode enabled / recording a coverage baseline.` — no false `fresh graph` | **VERIFIED (fixed)** |
| K3 | `pest --tia` ×3 on a clean tree | replayed entries keep their recorded `time` across all three | **VERIFIED (fixed)** — pre-fix: `0.053 → 0.001 → 0.001 → 0.001` |
| K4 | add a test file, one `pest --tia` (graph exists, fingerprint matches → replay+refresh) | result **and** edges appear in that same run; `n` grows by 1 | **VERIFIED** — regression guard for change 9 |
| K5 | add a test file, plain `pest --no-tia` | no result written for it; `n` unchanged; no edges | **VERIFIED (fixed)** — pre-fix: edge-less result inflated `n` |
| K6 | `pest --tia --parallel` worker argv | carries `-d pcov.directory=<projectRoot>`; `PEST_TIA` unset; not re-exec'd | **VERIFIED** — probe `bin/worker.php` |
| K7 | second `pest --tia --coverage` (cache primed → replay) | recorder uses link tracking only, does not clear PHPUnit's data mid-collection; coverage report intact | Not yet measured — **new work** |
| K8 | `pest --tia --parallel --coverage` | workers read `TIA_PIGGYBACK_COVERAGE`; no widened pcov scope; report intact | Not yet measured — **new work** |
| K9 | `pest --tia --filtered --coverage-html=<dir>` / `--coverage-clover=<file>` | filtered mode off, same as I8, for every flag in `COVERAGE_REPORT_FLAGS` | Not yet measured — **new work** |
---
## Part 3 — Priorities for phase two
0. **Part 1b fixtures** — nothing in C, D6D10, or E9E14 means anything until they exist.
1. **K7, K8, K9** — the only rows never measured. K7/K8 exercise changes 2 and 3, which were
reasoned about but not observed; K9 covers the seven `COVERAGE_REPORT_FLAGS` beyond
`--coverage-text`.
2. **B2, B3, F4** and all of **D** and **E** — change 9 touched the shared result-write path, and
these are the rows that exercise it hardest. `$recordsEdges` is the thing to falsify: it must be
false for every partial and every non-recording run.
3. **F3F8** — change 7 injects a `-d` into worker argv; `--mutate --parallel` is the one place that
both spawns workers and must write nothing.
4. **H1H10** — untouched by these changes; cheapest bulk confirmation.
5. Raw PHPUnit coverage flags print **no report at all** in Pest, with or without TIA
(`--no-tia --coverage-text` is equally silent). Pre-existing, unrelated to change 6 — do not
chase it as a regression, but it means I8/K9 can only assert the selection half.
Report per row: tier respected (yes/no), the graph delta under sentinel patching, and for any
failure the pre-fix contrast (write `git show db70017c:<file>` into vendor, re-run, restore) so a
regression is told apart from a pre-existing defect.
+12 -24
View File
@@ -1,19 +1,20 @@
<p align="center">
<img src="https://raw.githubusercontent.com/pestphp/art/master/v4/social.png" width="600" alt="PEST">
<img src="https://raw.githubusercontent.com/pestphp/art/master/v5/social.png" width="600" alt="PEST">
<p align="center">
<a href="https://github.com/pestphp/pest/actions"><img alt="GitHub Workflow Status (master)" src="https://img.shields.io/github/actions/workflow/status/pestphp/pest/tests.yml?branch=4.x&label=Tests%204.x"></a>
<a href="https://github.com/pestphp/pest/actions"><img alt="GitHub Workflow Status (5.x)" src="https://img.shields.io/github/actions/workflow/status/pestphp/pest/tests.yml?branch=5.x&label=Tests%205.x"></a>
<a href="https://packagist.org/packages/pestphp/pest"><img alt="Total Downloads" src="https://img.shields.io/packagist/dt/pestphp/pest"></a>
<a href="https://packagist.org/packages/pestphp/pest"><img alt="Latest Version" src="https://img.shields.io/packagist/v/pestphp/pest"></a>
<a href="https://packagist.org/packages/pestphp/pest"><img alt="License" src="https://img.shields.io/packagist/l/pestphp/pest"></a>
<a href="https://whyphp.dev"><img src="https://img.shields.io/badge/Why_PHP-in_2026-7A86E8?style=flat-square&labelColor=18181b" alt="Why PHP in 2026"></a>
<a href="https://youtube.com/@nunomaduro?sub_confirmation=1"><img alt="YouTube Channel Subscribers" src="https://img.shields.io/youtube/channel/subscribers/UCO_hYZF2gb_CyG5sA7ArlGg?style=flat&label=youtube&color=brightgreen"></a>
</p>
</p>
------
> Pest v4 Now Available: **[Read the announcement »](https://pestphp.com/docs/pest-v4-is-here-now-with-browser-testing)**.
> Pest v5 Now Available: **[Read the announcement »](https://pestphp.com/docs/pest5-now-available)**.
**Pest** is an elegant PHP testing Framework with a focus on simplicity, meticulously designed to bring back the joy of testing in PHP.
**Pest** is an elegant testing framework for PHP developers and AI agents.
- Explore our docs at **[pestphp.com »](https://pestphp.com)**
- Follow the creator Nuno Maduro:
@@ -28,26 +29,13 @@
We cannot thank our sponsors enough for their incredible support in funding Pest's development. Their contributions have been instrumental in making Pest the best it can be. For those who are interested in becoming a sponsor, please visit Nuno Maduro's Sponsor page at **[github.com/sponsors/nunomaduro](https://github.com/sponsors/nunomaduro)**.
### Platinum Sponsors
- **[CodeRabbit](https://coderabbit.ai/?ref=pestphp)**
- **[Devin](https://devin.ai/?ref=nunomaduro)**
- **[Mailtrap](https://l.rw.rw/pestphp)**
- **[Tighten](https://tighten.com/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/?utm_source=pest&utm_medium=banner&utm_campaign=pest_sponsorship)**
### Gold Sponsors
- **[CMS Max](https://cmsmax.com/?ref=pestphp)**
### Premium Sponsors
- [Zapiet](https://zapiet.com/?ref=pestphp)
- [Load Forge](https://loadforge.com/?ref=pestphp)
- [Route4Me](https://route4me.com/pt?ref=pestphp)
- [Nerdify](https://getnerdify.com/?ref=pestphp)
- [Akaunting](https://akaunting.com/?ref=pestphp)
- [LambdaTest](https://lambdatest.com/?ref=pestphp)
- **[PhpStorm](https://jb.gg/nuno)**
- **[CodeRabbit](https://coderabbit.link/nunomaduro)**
- **[SerpApi](https://serpapi.com/?ref=nunomaduro)**
- **[Typesense](https://typesense.org/?ref=nunomaduro)**
- **[Bento](https://bentonow.com/?ref=nunomaduro)**
- **[Pixel](https://wearepixel.com.au/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/?ref=nunomaduro)**
Pest is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.
+3 -3
View File
@@ -2,10 +2,10 @@
When releasing a new version of Pest there are some checks and updates that need to be done:
> **For Pest v3 you should use the `3.x` branch instead.**
> **For Pest v4 you should use the `4.x` branch instead.**
- Clear your local repository with: `git add . && git reset --hard && git checkout 4.x`
- On the GitHub repository, check the contents of [github.com/pestphp/pest/compare/{latest_version}...4.x](https://github.com/pestphp/pest/compare/{latest_version}...4.x)
- Clear your local repository with: `git add . && git reset --hard && git checkout 5.x`
- On the GitHub repository, check the contents of [github.com/pestphp/pest/compare/{latest_version}...5.x](https://github.com/pestphp/pest/compare/{latest_version}...5.x)
- Update the version number in [src/Pest.php](src/Pest.php)
- Run the tests locally using: `composer test`
- Commit the Pest file with the message: `git commit -m "release: vX.X.X"`
+23
View File
@@ -3,13 +3,16 @@
declare(strict_types=1);
use Pest\Contracts\Restarter;
use Pest\Kernel;
use Pest\Panic;
use Pest\Support\Container;
use Pest\TestCaseFilters\GitDirtyTestCaseFilter;
use Pest\TestCaseMethodFilters\AssigneeTestCaseFilter;
use Pest\TestCaseMethodFilters\IssueTestCaseFilter;
use Pest\TestCaseMethodFilters\NotesTestCaseFilter;
use Pest\TestCaseMethodFilters\PrTestCaseFilter;
use Pest\TestCaseMethodFilters\FlakyTestCaseFilter;
use Pest\TestCaseMethodFilters\TodoTestCaseFilter;
use Pest\TestSuite;
use Symfony\Component\Console\Input\ArgvInput;
@@ -23,6 +26,7 @@ use Symfony\Component\Console\Output\ConsoleOutput;
$dirty = false;
$todo = false;
$flaky = false;
$notes = false;
foreach ($arguments as $key => $value) {
@@ -57,6 +61,11 @@ use Symfony\Component\Console\Output\ConsoleOutput;
unset($arguments[$key]);
}
if ($value === '--flaky') {
$flaky = true;
unset($arguments[$key]);
}
if ($value === '--notes') {
$notes = true;
unset($arguments[$key]);
@@ -135,6 +144,7 @@ use Symfony\Component\Console\Output\ConsoleOutput;
// Get $rootPath based on $autoloadPath
$rootPath = dirname($autoloadPath, 2);
$input = new ArgvInput;
$testSuite = TestSuite::getInstance(
@@ -150,6 +160,10 @@ use Symfony\Component\Console\Output\ConsoleOutput;
$testSuite->tests->addTestCaseMethodFilter(new TodoTestCaseFilter);
}
if ($flaky) {
$testSuite->tests->addTestCaseMethodFilter(new FlakyTestCaseFilter);
}
if ($notes) {
$testSuite->tests->addTestCaseMethodFilter(new NotesTestCaseFilter);
}
@@ -181,6 +195,15 @@ use Symfony\Component\Console\Output\ConsoleOutput;
try {
$kernel = Kernel::boot($testSuite, $input, $output);
$container = Container::getInstance();
foreach (Kernel::RESTARTERS as $restarterClass) {
$restarter = $container->get($restarterClass);
assert($restarter instanceof Restarter);
$restarter->maybeRestart($rootPath, $originalArguments);
}
$result = $kernel->handle($originalArguments, $arguments);
$kernel->terminate();
+374
View File
@@ -0,0 +1,374 @@
#!/usr/bin/env node
import { readdir, readFile } from 'node:fs/promises'
import { existsSync } from 'node:fs'
import { createRequire } from 'node:module'
import { resolve, relative, extname, sep, join } from 'node:path'
import { pathToFileURL } from 'node:url'
const PAGE_EXTENSIONS = new Set([
'.vue', '.svelte',
'.tsx', '.jsx',
'.ts', '.js',
'.mts', '.cts', '.mjs', '.cjs',
])
const ASSET_EXT_RE = /\.(css|scss|sass|less|styl|stylus|svg|png|jpe?g|gif|webp|avif|ico|bmp|woff2?|ttf|eot|otf|md|mdx|txt|html|mp4|webm|mp3|wav|ogg|m4a|pdf|wasm|glsl|frag|vert)$/i
const PROJECT_ROOT = resolve(process.argv[2] ?? process.cwd())
const PAGE_DIR_CANDIDATES = [
'resources/js/Pages',
'resources/js/pages',
'assets/js/Pages',
'assets/js/pages',
'assets/Pages',
'assets/pages',
]
async function loadRolldown() {
const projectRequire = createRequire(join(PROJECT_ROOT, 'package.json'))
let path = null
try { path = projectRequire.resolve('rolldown') } catch {}
if (path === null) {
// rolldown-vite installs (vite@npm:rolldown-vite) ship rolldown as a
// dependency of the vite package rather than a top-level install.
try {
const viteRequire = createRequire(projectRequire.resolve('vite/package.json'))
path = viteRequire.resolve('rolldown')
} catch {}
}
if (path === null) return null
return await import(pathToFileURL(path).href)
}
export function stripJsonComments(raw) {
let out = ''
let inString = false
let quote = ''
let inLine = false
let inBlock = false
for (let i = 0; i < raw.length; i++) {
const c = raw[i]
const n = raw[i + 1]
if (inLine) {
if (c === '\n') { inLine = false; out += c }
continue
}
if (inBlock) {
if (c === '*' && n === '/') { inBlock = false; i++ }
continue
}
if (inString) {
out += c
if (c === '\\') { out += n ?? ''; i++; continue }
if (c === quote) inString = false
continue
}
if (c === '"' || c === "'") { inString = true; quote = c; out += c; continue }
if (c === '/' && n === '/') { inLine = true; i++; continue }
if (c === '/' && n === '*') { inBlock = true; i++; continue }
if (c === '}' || c === ']') out = out.replace(/,\s*$/, '')
out += c
}
return out
}
async function readJsonWithComments(path) {
const raw = await readFile(path, 'utf8')
return JSON.parse(stripJsonComments(raw))
}
export async function loadAliasFromTsconfig(projectRoot = PROJECT_ROOT) {
const alias = {}
for (const name of ['tsconfig.json', 'jsconfig.json']) {
const p = join(projectRoot, name)
if (!existsSync(p)) continue
let cfg
try { cfg = await readJsonWithComments(p) } catch { continue }
const baseUrl = resolve(projectRoot, cfg?.compilerOptions?.baseUrl ?? '.')
const paths = cfg?.compilerOptions?.paths ?? {}
for (const [key, targets] of Object.entries(paths)) {
if (!key.endsWith('/*')) continue
const t0 = Array.isArray(targets) ? targets[0] : null
if (typeof t0 !== 'string' || !t0.endsWith('/*')) continue
const aliasKey = key.slice(0, -2)
if (alias[aliasKey] !== undefined) continue
alias[aliasKey] = resolve(baseUrl, t0.slice(0, -2))
}
}
return alias
}
const VITE_CONFIG_FILES = [
'vite.config.ts',
'vite.config.js',
'vite.config.mjs',
'vite.config.cjs',
'vite.config.mts',
'vite.config.cts',
]
function resolveAliasTarget(projectRoot, target) {
if (target.startsWith('/')) {
// Vite resolves a leading slash against the project root, unless the
// config author used a genuinely absolute path.
return existsSync(target) ? target : resolve(projectRoot, '.' + target)
}
return resolve(projectRoot, target)
}
async function usesLaravelVitePlugin(projectRoot) {
const p = join(projectRoot, 'package.json')
if (!existsSync(p)) return false
try {
const pkg = JSON.parse(await readFile(p, 'utf8'))
return Boolean(pkg.dependencies?.['laravel-vite-plugin'] ?? pkg.devDependencies?.['laravel-vite-plugin'])
} catch {
return false
}
}
export async function loadAliasFromViteConfig(projectRoot = PROJECT_ROOT) {
const alias = {}
let source = null
for (const name of VITE_CONFIG_FILES) {
const p = join(projectRoot, name)
if (!existsSync(p)) continue
try { source = await readFile(p, 'utf8') } catch { continue }
break
}
if (source !== null) {
// The config is executable code we cannot import safely, so extract alias
// entries textually: a quoted `@…`/`~…` key, then the last quoted path in
// the value expression — covers `'@': '/resources/js'` as well as
// `'@': path.resolve(__dirname, 'resources/js')`. The value stops at a
// top-level comma; call arguments are kept via the paren group.
for (const m of source.matchAll(/['"`]([@~][\w./-]*)['"`]\s*:\s*((?:\([^)\n]*\)|[^,\n])*)/g)) {
const key = m[1]
if (alias[key] !== undefined) continue
const paths = [...m[2].matchAll(/['"`]([^'"`]+)['"`]/g)].map((p) => p[1])
const target = paths.length > 0 ? paths[paths.length - 1] : null
if (!target || target.includes('*')) continue
alias[key] = resolveAliasTarget(projectRoot, target)
}
}
// laravel-vite-plugin registers '@' → resources/js by default.
if (alias['@'] === undefined && (await usesLaravelVitePlugin(projectRoot))) {
alias['@'] = resolve(projectRoot, 'resources/js')
}
return alias
}
async function listPageFiles(pagesDir) {
if (!existsSync(pagesDir)) return []
const out = []
const walk = async (dir) => {
let entries
try { entries = await readdir(dir, { withFileTypes: true }) } catch { return }
for (const entry of entries) {
const full = resolve(dir, entry.name)
if (entry.isDirectory()) { await walk(full); continue }
if (PAGE_EXTENSIONS.has(extname(entry.name))) out.push(full)
}
}
await walk(pagesDir)
return out
}
async function discoverPagesDir() {
const override = process.env.TIA_VITE_PAGES_DIR
if (override && override.length > 0) {
return resolve(PROJECT_ROOT, override.replace(/\\/g, '/'))
}
for (const rel of PAGE_DIR_CANDIDATES) {
const abs = resolve(PROJECT_ROOT, rel)
if (!existsSync(abs)) continue
const files = await listPageFiles(abs)
if (files.length > 0) return abs
}
return null
}
function componentNameFor(pageAbs, pagesDir) {
const rel = relative(pagesDir, pageAbs).split(sep).join('/')
const ext = extname(rel)
return rel.slice(0, rel.length - ext.length)
}
function isLocalSpecifier(source, aliasKeys) {
if (source.startsWith('.') || source.startsWith('/')) return true
for (const key of aliasKeys) {
if (source === key || source.startsWith(key + '/')) return true
}
return false
}
async function main() {
const pagesDir = await discoverPagesDir()
if (pagesDir === null) {
process.stdout.write('{}')
return
}
const pages = await listPageFiles(pagesDir)
if (pages.length === 0) {
process.stdout.write('{}')
return
}
const loaded = await loadRolldown()
if (loaded === null) {
process.stdout.write('{}')
return
}
const { rolldown } = loaded
// The vite config is what the dev server actually resolves with — let it win.
const alias = { ...(await loadAliasFromTsconfig()), ...(await loadAliasFromViteConfig()) }
const aliasKeys = Object.keys(alias)
const graph = new Map()
const collector = {
name: 'pest-tia-collector',
moduleParsed(info) {
const id = info.id
if (!id || id.startsWith('\0')) return
const deps = new Set()
for (const i of info.importedIds) if (i && !i.startsWith('\0')) deps.add(i)
for (const i of info.dynamicallyImportedIds) if (i && !i.startsWith('\0')) deps.add(i)
graph.set(id, deps)
},
}
const externalBare = {
name: 'pest-tia-external-bare',
resolveId(source) {
if (!source) return null
if (isLocalSpecifier(source, aliasKeys)) return null
return { id: source, external: true }
},
}
const assetStub = {
name: 'pest-tia-asset-stub',
load(id) {
if (!id) return null
if (ASSET_EXT_RE.test(id)) {
return { code: 'export default null', moduleSideEffects: false }
}
return null
},
}
const input = Object.create(null)
for (let i = 0; i < pages.length; i++) input[`p${i}`] = pages[i]
const bundle = await rolldown({
input,
cwd: PROJECT_ROOT,
resolve: {
alias,
extensions: ['.tsx', '.ts', '.jsx', '.js', '.mts', '.cts', '.mjs', '.cjs', '.json', '.vue', '.svelte'],
},
transform: { jsx: 'preserve' },
treeshake: false,
plugins: [externalBare, assetStub, collector],
logLevel: 'silent',
onLog: () => {},
})
try {
await bundle.generate({ format: 'esm' })
} finally {
await bundle.close()
}
const reverse = new Map()
const transitiveCache = new Map()
const computeTransitive = (id, stack) => {
const cached = transitiveCache.get(id)
if (cached) return cached
if (stack.has(id)) return null
stack.add(id)
const acc = new Set()
// A set computed while skipping an in-stack (cyclic) dependency is missing
// that dependency's subtree and must not be memoized — the ancestor call
// completes it for the current traversal, but a cached copy would leak the
// incomplete set into other pages' traversals.
let complete = true
const deps = graph.get(id)
if (deps) {
for (const dep of deps) {
if (!dep || dep.startsWith('\0')) continue
if (dep.startsWith(PROJECT_ROOT)) {
const rel = relative(PROJECT_ROOT, dep).split(sep).join('/')
acc.add(rel)
}
if (stack.has(dep)) {
complete = false
continue
}
const child = computeTransitive(dep, stack)
if (child) for (const r of child) acc.add(r)
if (!transitiveCache.has(dep)) complete = false
}
}
stack.delete(id)
if (complete) transitiveCache.set(id, acc)
return acc
}
for (const page of pages) {
const pageComponent = componentNameFor(page, pagesDir)
const reachable = computeTransitive(page, new Set())
if (!reachable) continue
for (const rel of reachable) {
const bucket = reverse.get(rel) ?? new Set()
bucket.add(pageComponent)
reverse.set(rel, bucket)
}
}
const payload = Object.create(null)
const keys = [...reverse.keys()].sort()
for (const key of keys) {
payload[key] = [...reverse.get(key)].sort()
}
process.stdout.write(JSON.stringify(payload))
}
const invokedDirectly = process.argv[1] !== undefined
&& import.meta.url === pathToFileURL(process.argv[1]).href
if (invokedDirectly) {
try {
await main()
} catch (err) {
process.stderr.write(String(err?.stack ?? err ?? 'unknown error'))
process.exit(1)
}
}
+10
View File
@@ -6,6 +6,7 @@ use ParaTest\WrapperRunner\ApplicationForWrapperWorker;
use ParaTest\WrapperRunner\WrapperWorker;
use Pest\Kernel;
use Pest\Plugins\Actions\CallsHandleArguments;
use Pest\Support\Container;
use Pest\TestSuite;
use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Output\ConsoleOutput;
@@ -58,6 +59,15 @@ $bootPest = (static function (): void {
}
}
$container = Container::getInstance();
$rootPath = dirname(PHPUNIT_COMPOSER_INSTALL, 2);
foreach (Kernel::RESTARTERS as $restarterClass) {
$restarter = $container->get($restarterClass);
$restarter->maybeRestart($rootPath, $_SERVER['argv']);
}
assert(isset($getopt['status-file']) && is_string($getopt['status-file']));
$statusFile = fopen($getopt['status-file'], 'wb');
assert(is_resource($statusFile));
+30 -20
View File
@@ -17,20 +17,20 @@
}
],
"require": {
"php": "^8.3.0",
"brianium/paratest": "^7.16.0",
"nunomaduro/collision": "^8.8.3",
"nunomaduro/termwind": "^2.3.3",
"pestphp/pest-plugin": "^4.0.0",
"pestphp/pest-plugin-arch": "^4.0.0",
"pestphp/pest-plugin-mutate": "^4.0.1",
"pestphp/pest-plugin-profanity": "^4.2.1",
"phpunit/phpunit": "^12.5.4",
"symfony/process": "^7.4.3|^8.0.0"
"php": "^8.4",
"brianium/paratest": "^7.23.1",
"nunomaduro/collision": "^8.9.5",
"nunomaduro/termwind": "^2.4.0",
"pestphp/pest-plugin": "^5.0.0",
"pestphp/pest-plugin-arch": "^5.0.0",
"pestphp/pest-plugin-mutate": "^5.0.1",
"pestphp/pest-plugin-profanity": "^5.0.0",
"phpunit/phpunit": "^13.2.6",
"symfony/process": "^8.1.0"
},
"conflict": {
"filp/whoops": "<2.18.3",
"phpunit/phpunit": ">12.5.4",
"phpunit/phpunit": ">13.2.6",
"sebastian/exporter": "<7.0.0",
"webmozart/assert": "<1.11.0"
},
@@ -48,17 +48,23 @@
"Tests\\Fixtures\\Covers\\": "tests/Fixtures/Covers",
"Tests\\Fixtures\\Inheritance\\": "tests/Fixtures/Inheritance",
"Tests\\Fixtures\\Arch\\": "tests/Fixtures/Arch",
"Tests\\Fixtures\\Tia\\": "tests/Fixtures/Tia",
"Tests\\": "tests/PHPUnit/"
},
"classmap": [
"tests/Fixtures/Arch/ToBeCasedCorrectly/IncorrectCasing/incorrectCasing.php"
],
"files": [
"tests/Autoload.php"
]
},
"require-dev": {
"pestphp/pest-dev-tools": "^4.0.0",
"pestphp/pest-plugin-browser": "^4.1.1",
"pestphp/pest-plugin-type-coverage": "^4.0.3",
"psy/psysh": "^0.12.18"
"pestphp/pest-dev-tools": "^5.0.0",
"pestphp/pest-plugin-browser": "^5.0.0",
"pestphp/pest-plugin-phpstan": "^5.0.0",
"pestphp/pest-plugin-rector": "^5.0.2",
"pestphp/pest-plugin-type-coverage": "^5.0.2",
"psy/psysh": "^0.12.24"
},
"minimum-stability": "dev",
"prefer-stable": true,
@@ -73,10 +79,14 @@
"bin/pest"
],
"scripts": {
"refacto": "rector",
"lint": "pint --parallel",
"test:refacto": "rector --dry-run",
"test:lint": "pint --parallel --test",
"lint": [
"rector",
"pint --parallel"
],
"test:lint": [
"rector --dry-run",
"pint --parallel --test"
],
"test:profanity": "php bin/pest --profanity --compact",
"test:type:check": "phpstan analyse --ansi --memory-limit=-1 --debug",
"test:type:coverage": "php -d memory_limit=-1 bin/pest --type-coverage --min=100",
@@ -86,7 +96,6 @@
"test:integration": "php bin/pest --group=integration -v",
"update:snapshots": "REBUILD_SNAPSHOTS=true php bin/pest --update-snapshots",
"test": [
"@test:refacto",
"@test:lint",
"@test:type:check",
"@test:type:coverage",
@@ -116,6 +125,7 @@
"Pest\\Plugins\\Verbose",
"Pest\\Plugins\\Version",
"Pest\\Plugins\\Shard",
"Pest\\Plugins\\Tia",
"Pest\\Plugins\\Parallel"
]
},
-14
View File
@@ -1,14 +0,0 @@
version: "3.8"
services:
php:
build:
context: ./docker
volumes:
- .:/var/www/html
composer:
build:
context: ./docker
volumes:
- .:/var/www/html
entrypoint: ["composer"]
+38 -2
View File
@@ -1,6 +1,39 @@
<?php
/*
* BSD 3-Clause License
*
* Copyright (c) 2001-2023, Sebastian Bergmann
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* 3. Neither the name of the copyright holder nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
declare(strict_types=1);
/*
* This file is part of PHPUnit.
*
@@ -14,6 +47,9 @@ namespace PHPUnit\Logging\JUnit;
use DOMDocument;
use DOMElement;
use Pest\Logging\Converter;
use Pest\Support\Container;
use Pest\TestSuite;
use PHPUnit\Event\Code\Test;
use PHPUnit\Event\Code\TestMethod;
use PHPUnit\Event\EventFacadeIsSealedException;
@@ -50,7 +86,7 @@ final class JunitXmlLogger
{
private readonly Printer $printer;
private readonly \Pest\Logging\Converter $converter; // pest-added
private readonly Converter $converter; // pest-added
private DOMDocument $document;
@@ -108,7 +144,7 @@ final class JunitXmlLogger
public function __construct(Printer $printer, Facade $facade)
{
$this->printer = $printer;
$this->converter = new \Pest\Logging\Converter(\Pest\Support\Container::getInstance()->get(\Pest\TestSuite::class)->rootPath); // pest-added
$this->converter = new Converter(Container::getInstance()->get(TestSuite::class)->rootPath); // pest-added
$this->registerSubscribers($facade);
$this->createDocument();
@@ -0,0 +1,71 @@
<?php
/*
* The MIT License (MIT)
*
* Copyright (c) 2011 Brian Scaturro
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
declare(strict_types=1);
namespace ParaTest\WrapperRunner;
use PHPUnit\TextUI\Output\Printer;
use function preg_match;
/**
* @internal
*
* This file is overridden so the "T" progress character — emitted by Pest for
* "todo" tests — is routed to the progress file next to the regular characters,
* instead of being treated as unexpected output.
*/
final readonly class ProgressPrinterOutput implements Printer
{
public function __construct(
private Printer $progressPrinter,
private Printer $outputPrinter,
) {}
public function print(string $buffer): void
{
// Skip anything in \PHPUnit\TextUI\Output\Default\ProgressPrinter\ProgressPrinter::printProgress except $progress
if (
$buffer === "\n"
|| preg_match('/^ +$/', $buffer) === 1
|| preg_match('/^ \d+ \/ \d+ \(...%\)$/', $buffer) === 1
) {
return;
}
match ($buffer) {
'E', 'F', 'I', 'N', 'D', 'R', 'W', 'S', 'T', '.' => $this->progressPrinter->print($buffer),
default => $this->outputPrinter->print($buffer),
};
}
public function flush(): void
{
$this->progressPrinter->flush();
$this->outputPrinter->flush();
}
}
+2 -2
View File
@@ -90,11 +90,11 @@ final class TestSuiteLoader
(static function () use ($suiteClassFile) {
try {
include_once $suiteClassFile;
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
} catch (Throwable $e) {
Panic::with($e);
}
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
})();
$loadedClasses = array_values(
+388
View File
@@ -0,0 +1,388 @@
<?php
/*
* BSD 3-Clause License
*
* Copyright (c) 2001-2023, Sebastian Bergmann
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* 3. Neither the name of the copyright holder nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
declare(strict_types=1);
/*
* This file is part of PHPUnit.
*
* (c) Sebastian Bergmann <sebastian@phpunit.de>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/
namespace PHPUnit\Runner;
use PHPUnit\Framework\DataProviderTestSuite;
use PHPUnit\Framework\Reorderable;
use PHPUnit\Framework\Test;
use PHPUnit\Framework\TestCase;
use PHPUnit\Framework\TestSuite;
use PHPUnit\Runner\ResultCache\NullResultCache;
use PHPUnit\Runner\ResultCache\ResultCache;
use PHPUnit\Runner\ResultCache\ResultCacheId;
use function array_diff;
use function array_merge;
use function array_reverse;
use function array_splice;
use function assert;
use function count;
use function in_array;
use function max;
use function shuffle;
use function usort;
/**
* @internal This class is not covered by the backward compatibility promise for PHPUnit
*/
final class TestSuiteSorter
{
public const int ORDER_DEFAULT = 0;
public const int ORDER_RANDOMIZED = 1;
public const int ORDER_REVERSED = 2;
public const int ORDER_DEFECTS_FIRST = 3;
public const int ORDER_DURATION = 4;
public const int ORDER_SIZE = 5;
/**
* @var non-empty-array<non-empty-string, positive-int>
*/
private const array SIZE_SORT_WEIGHT = [
'small' => 1,
'medium' => 2,
'large' => 3,
'unknown' => 4,
];
/**
* @var array<string, int> Associative array of (string => DEFECT_SORT_WEIGHT) elements
*/
private array $defectSortOrder = [];
private readonly ResultCache $cache;
public function __construct(?ResultCache $cache = null)
{
$this->cache = $cache ?? new NullResultCache;
}
/**
* @throws Exception
*/
public function reorderTestsInSuite(Test $suite, int $order, bool $resolveDependencies, int $orderDefects): void
{
$allowedOrders = [
self::ORDER_DEFAULT,
self::ORDER_REVERSED,
self::ORDER_RANDOMIZED,
self::ORDER_DURATION,
self::ORDER_SIZE,
];
if (! in_array($order, $allowedOrders, true)) {
// @codeCoverageIgnoreStart
throw new InvalidOrderException;
// @codeCoverageIgnoreEnd
}
$allowedOrderDefects = [
self::ORDER_DEFAULT,
self::ORDER_DEFECTS_FIRST,
];
if (! in_array($orderDefects, $allowedOrderDefects, true)) {
// @codeCoverageIgnoreStart
throw new InvalidOrderException;
// @codeCoverageIgnoreEnd
}
if ($suite instanceof TestSuite) {
foreach ($suite as $_suite) {
$this->reorderTestsInSuite($_suite, $order, $resolveDependencies, $orderDefects);
}
if ($orderDefects === self::ORDER_DEFECTS_FIRST) {
$this->addSuiteToDefectSortOrder($suite);
}
$this->sort($suite, $order, $resolveDependencies, $orderDefects);
}
}
private function sort(TestSuite $suite, int $order, bool $resolveDependencies, int $orderDefects): void
{
if ($suite->tests() === []) {
return;
}
if ($order === self::ORDER_REVERSED) {
$suite->setTests($this->reverse($suite->tests()));
} elseif ($order === self::ORDER_RANDOMIZED) {
$suite->setTests($this->randomize($suite->tests()));
} elseif ($order === self::ORDER_DURATION) {
$suite->setTests($this->sortByDuration($suite->tests()));
} elseif ($order === self::ORDER_SIZE) {
$suite->setTests($this->sortBySize($suite->tests()));
}
if ($orderDefects === self::ORDER_DEFECTS_FIRST) {
$suite->setTests($this->sortDefectsFirst($suite->tests()));
}
if ($resolveDependencies && ! ($suite instanceof DataProviderTestSuite)) {
$tests = $suite->tests();
/** @noinspection PhpParamsInspection */
/** @phpstan-ignore argument.type */
$suite->setTests($this->resolveDependencies($tests));
}
}
private function addSuiteToDefectSortOrder(TestSuite $suite): void
{
$max = 0;
foreach ($suite->tests() as $test) {
assert($test instanceof Reorderable);
$sortId = $test->sortId();
if (! isset($this->defectSortOrder[$sortId])) {
$this->defectSortOrder[$sortId] = $this->cache->status(ResultCacheId::fromReorderable($test))->asInt();
$max = max($max, $this->defectSortOrder[$sortId]);
}
}
$this->defectSortOrder[$suite->sortId()] = $max;
}
/**
* @param list<Test> $tests
* @return list<Test>
*/
private function reverse(array $tests): array
{
return array_reverse($tests);
}
/**
* @param list<Test> $tests
* @return list<Test>
*/
private function randomize(array $tests): array
{
shuffle($tests);
return $tests;
}
/**
* @param list<Test> $tests
* @return list<Test>
*/
private function sortDefectsFirst(array $tests): array
{
usort(
$tests,
fn (Test $left, Test $right) => $this->cmpDefectPriorityAndTime($left, $right),
);
return $tests;
}
/**
* @param list<Test> $tests
* @return list<Test>
*/
private function sortByDuration(array $tests): array
{
usort(
$tests,
fn (Test $left, Test $right) => $this->cmpDuration($left, $right),
);
return $tests;
}
/**
* @param list<Test> $tests
* @return list<Test>
*/
private function sortBySize(array $tests): array
{
usort(
$tests,
fn (Test $left, Test $right) => $this->cmpSize($left, $right),
);
return $tests;
}
/**
* Comparator callback function to sort tests for "reach failure as fast as possible".
*
* 1. sort tests by defect weight defined in self::DEFECT_SORT_WEIGHT
* 2. when tests are equally defective, sort the fastest to the front
* 3. do not reorder successful tests
*/
private function cmpDefectPriorityAndTime(Test $a, Test $b): int
{
assert($a instanceof Reorderable);
assert($b instanceof Reorderable);
$priorityA = $this->defectSortOrder[$a->sortId()] ?? 0;
$priorityB = $this->defectSortOrder[$b->sortId()] ?? 0;
if ($priorityA !== $priorityB) {
// Sort defect weight descending
return $priorityB <=> $priorityA;
}
if ($priorityA > 0 || $priorityB > 0) {
return $this->cmpDuration($a, $b);
}
// do not change execution order
return 0;
}
/**
* Compares test duration for sorting tests by duration ascending.
*/
private function cmpDuration(Test $a, Test $b): int
{
if (! ($a instanceof Reorderable && $b instanceof Reorderable)) {
return 0;
}
return $this->cache->time(ResultCacheId::fromReorderable($a)) <=> $this->cache->time(ResultCacheId::fromReorderable($b));
}
/**
* Compares test size for sorting tests small->medium->large->unknown.
*/
private function cmpSize(Test $a, Test $b): int
{
$sizeA = ($a instanceof TestCase || $a instanceof DataProviderTestSuite)
? $a->size()->asString()
: 'unknown';
$sizeB = ($b instanceof TestCase || $b instanceof DataProviderTestSuite)
? $b->size()->asString()
: 'unknown';
return self::SIZE_SORT_WEIGHT[$sizeA] <=> self::SIZE_SORT_WEIGHT[$sizeB];
}
/**
* Reorder Tests within a TestCase in such a way as to resolve as many dependencies as possible.
* The algorithm will leave the tests in original running order when it can.
* For more details see the documentation for test dependencies.
*
* Short description of algorithm:
* 1. Pick the next Test from remaining tests to be checked for dependencies.
* 2. If the test has no dependencies: mark done, start again from the top
* 3. If the test has dependencies but none left to do: mark done, start again from the top
* 4. When we reach the end add any leftover tests to the end. These will be marked 'skipped' during execution.
*
* @param array<TestCase> $tests
* @return array<TestCase>
*/
private function resolveDependencies(array $tests): array
{
// Pest: Fast-path. If no test in this suite declares dependencies, the
// original O(N^2) algorithm is wasted work — it would splice each test
// one-by-one back into the same order. The check deliberately walks
// TestCase instances directly instead of calling TestSuite::requires(),
// because the latter lazily builds TestSuite::provides() via
// ExecutionOrderDependency::mergeUnique, which is O(N^2) in the total
// number of tests. With thousands of tests that single call alone can
// burn several seconds before the sort even begins. Reading the
// cached TestCase::$dependencies property stays O(N) and costs nothing
// when no test uses `->depends()` / PHPUnit `@depends`.
if (! $this->anyTestHasDependencies($tests)) {
return $tests;
}
$newTestOrder = [];
$i = 0;
$provided = [];
do {
if (array_diff($tests[$i]->requires(), $provided) === []) {
$provided = array_merge($provided, $tests[$i]->provides());
$newTestOrder = array_merge($newTestOrder, array_splice($tests, $i, 1));
$i = 0;
} else {
$i++;
}
} while ($tests !== [] && ($i < count($tests)));
return array_merge($newTestOrder, $tests);
}
/**
* Cheaply determines whether any test in the tree declares @depends.
*
* Walks `TestSuite` containers recursively and inspects each `TestCase`
* directly so it never triggers `TestSuite::provides()`, which is O(N^2)
* in the total number of aggregated tests.
*
* @param iterable<Test> $tests
*/
private function anyTestHasDependencies(iterable $tests): bool
{
foreach ($tests as $test) {
if ($test instanceof TestSuite) {
if ($this->anyTestHasDependencies($test->tests())) {
return true;
}
continue;
}
if ($test instanceof TestCase && $test->requires() !== []) {
return true;
}
}
return false;
}
}
@@ -43,7 +43,7 @@ declare(strict_types=1);
* file that was distributed with this source code.
*/
namespace Pest\Logging\TeamCity\Subscriber;
namespace PHPUnit\TextUI\Output\Default\ProgressPrinter;
use PHPUnit\Event\Test\Skipped;
use PHPUnit\Event\Test\SkippedSubscriber;
@@ -51,16 +51,20 @@ use ReflectionClass;
/**
* @internal This class is not covered by the backward compatibility promise for PHPUnit
*
* This file is overridden so PHPUnit's progress output emits a "T" before the
* regular "S" for "todo" tests — Pest's parallel result printer consumes the
* "T" and swallows the "S" that follows it.
*/
final class TestSkippedSubscriber extends Subscriber implements SkippedSubscriber
final readonly class TestSkippedSubscriber extends Subscriber implements SkippedSubscriber
{
public function notify(Skipped $event): void
{
if (str_contains($event->message(), '__TODO__')) {
if ($event->message() === '__TODO__') {
$this->printTodoItem();
}
$this->logger()->testSkipped($event);
$this->printer()->testSkipped();
}
/**
@@ -69,7 +73,7 @@ final class TestSkippedSubscriber extends Subscriber implements SkippedSubscribe
private function printTodoItem(): void
{
$mirror = new ReflectionClass($this->printer());
$printerMirror = $mirror->getMethod('printProgress');
$printerMirror->invoke($this->printer(), 'T');
$printProgress = $mirror->getMethod('printProgress');
$printProgress->invoke($this->printer(), 'T');
}
}
-78
View File
@@ -1,11 +1,5 @@
parameters:
ignoreErrors:
-
message: '#^Parameter \#1 of callable callable\(Pest\\Expectation\<string\|null\>\)\: Pest\\Arch\\Contracts\\ArchExpectation expects Pest\\Expectation\<string\|null\>, Pest\\Expectation\<string\|null\> given\.$#'
identifier: argument.type
count: 1
path: src/ArchPresets/AbstractPreset.php
-
message: '#^Trait Pest\\Concerns\\Expectable is used zero times and is not analysed\.$#'
identifier: trait.unused
@@ -24,12 +18,6 @@ parameters:
count: 1
path: src/Concerns/Testable.php
-
message: '#^Loose comparison using \!\= between \(Closure\|null\) and false will always evaluate to false\.$#'
identifier: notEqual.alwaysFalse
count: 1
path: src/Expectation.php
-
message: '#^Method Pest\\Expectation\:\:and\(\) should return Pest\\Expectation\<TAndValue\> but returns \(Pest\\Expectation&TAndValue\)\|Pest\\Expectation\<TAndValue of mixed\>\.$#'
identifier: return.type
@@ -102,78 +90,12 @@ parameters:
count: 1
path: src/PendingCalls/TestCall.php
-
message: '#^Parameter \#1 \$argv of class Symfony\\Component\\Console\\Input\\ArgvInput constructor expects list\<string\>\|null, array\<int, string\> given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel.php
-
message: '#^Parameter \#13 \$testRunnerTriggeredDeprecationEvents of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\Event\\TestRunner\\DeprecationTriggered\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#14 \$testRunnerTriggeredWarningEvents of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\Event\\TestRunner\\WarningTriggered\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#15 \$errors of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#16 \$deprecations of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#17 \$notices of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#18 \$warnings of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#19 \$phpDeprecations of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#20 \$phpNotices of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#21 \$phpWarnings of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\TestRunner\\TestResult\\Issues\\Issue\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#4 \$testErroredEvents of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\Event\\Test\\AfterLastTestMethodErrored\|PHPUnit\\Event\\Test\\BeforeFirstTestMethodErrored\|PHPUnit\\Event\\Test\\Errored\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#5 \$testFailedEvents of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\Event\\Test\\Failed\>, array given\.$#'
identifier: argument.type
count: 1
path: src/Plugins/Parallel/Paratest/WrapperRunner.php
-
message: '#^Parameter \#7 \$testSuiteSkippedEvents of class PHPUnit\\TestRunner\\TestResult\\TestResult constructor expects list\<PHPUnit\\Event\\TestSuite\\Skipped\>, array given\.$#'
identifier: argument.type
+5
View File
@@ -0,0 +1,5 @@
services:
-
class: Pest\PHPStan\HigherOrderExpectationTypeExtension
tags:
- phpstan.broker.expressionTypeResolverExtension
+2 -3
View File
@@ -1,5 +1,7 @@
includes:
- phpstan-baseline.neon
- phpstan-pest-extension.neon
- vendor/pestphp/pest-plugin-phpstan/extension.neon
parameters:
level: 7
@@ -7,6 +9,3 @@ parameters:
- src
reportUnmatchedIgnoredErrors: false
ignoreErrors:
- "#type mixed is not subtype of native#"
+2 -1
View File
@@ -18,8 +18,9 @@
<directory suffix=".php">./tests</directory>
<directory suffix=".php">./tests-external</directory>
<exclude>./tests/.snapshots</exclude>
<exclude>./tests/.tests</exclude>
<exclude>./tests/Fixtures/Inheritance</exclude>
<exclude>./tests/Fixtures/Suites</exclude>
<exclude>./tests/Fixtures/Tia</exclude>
</testsuite>
</testsuites>
<source>
+5
View File
@@ -0,0 +1,5 @@
{
"exclude": [
"tests/Fixtures/Suites"
]
}
+39
View File
@@ -2,22 +2,61 @@
declare(strict_types=1);
use Pest\Rector\Rules\UseToMatchArrayRector;
use Pest\Rector\Set\PestSetList;
use Rector\CodingStyle\Rector\ArrowFunction\ArrowFunctionDelegatingCallToFirstClassCallableRector;
use Rector\Config\RectorConfig;
use Rector\DeadCode\Rector\ClassMethod\RemoveDuplicatedReturnSelfDocblockRector;
use Rector\DeadCode\Rector\ClassMethod\RemoveEmptyClassMethodRector;
use Rector\DeadCode\Rector\ClassMethod\RemoveParentDelegatingConstructorRector;
use Rector\DeadCode\Rector\ClassMethod\RemoveReturnTagIncompatibleWithNativeTypeRector;
use Rector\DeadCode\Rector\ClassMethod\RemoveUnusedConstructorParamRector;
use Rector\DeadCode\Rector\ClassMethod\RemoveUselessUnionReturnDocblockRector;
use Rector\DeadCode\Rector\Property\RemoveUnusedPrivatePropertyRector;
use Rector\TypeDeclaration\Rector\ArrowFunction\AddArrowFunctionReturnTypeRector;
use Rector\TypeDeclaration\Rector\ClassMethod\NarrowObjectReturnTypeRector;
use Rector\TypeDeclaration\Rector\ClassMethod\ReturnNeverTypeRector;
return RectorConfig::configure()
->withPaths([
__DIR__.'/src',
__DIR__.'/tests',
])
->withSets([
PestSetList::CODING_STYLE,
])
->withSkip([
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
__DIR__.'/tests/Fixtures/Arch',
__DIR__.'/tests/Fixtures/Suites',
ReturnNeverTypeRector::class,
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
NarrowObjectReturnTypeRector::class,
RemoveParentDelegatingConstructorRector::class,
RemoveDuplicatedReturnSelfDocblockRector::class,
RemoveUselessUnionReturnDocblockRector::class,
RemoveReturnTagIncompatibleWithNativeTypeRector::class => [
__DIR__.'/src/Expectations/HigherOrderExpectation.php',
],
// Merges unrelated expectations into a single `toMatchArray()`, turning
// `toContain()` into exact matches, dropping `->not`, and mistaking a
// `toBeTrue()` failure message for an expected value. Unsafe here.
UseToMatchArrayRector::class,
// Test fixtures rely on "unused" constructors, params and properties
// (resolved via the container or read through reflection), so the
// dead-code and return-type rules below must not touch the test suite.
RemoveEmptyClassMethodRector::class => [
__DIR__.'/tests',
],
RemoveUnusedConstructorParamRector::class => [
__DIR__.'/tests',
],
RemoveUnusedPrivatePropertyRector::class => [
__DIR__.'/tests',
],
AddArrowFunctionReturnTypeRector::class => [
__DIR__.'/tests',
],
])
->withPreparedSets(
deadCode: true,
@@ -0,0 +1,196 @@
---
name: pest-testing
description: "Use this skill for Pest PHP testing in Laravel projects only. Trigger whenever any test is being written, edited, fixed, or refactored — including fixing tests that broke after a code change, adding assertions, converting PHPUnit to Pest, adding datasets, and TDD workflows. Always activate when the user asks how to write something in Pest, mentions test files or directories (tests/Feature, tests/Unit, tests/Browser), or needs browser testing, smoke testing multiple pages for JS errors, architecture tests, or faster test runs with Test Impact Analysis. Covers: test()/it()/expect() syntax, datasets, mocking, browser testing (visit/click/fill), smoke testing, arch(), Livewire component tests, RefreshDatabase, Tia (--tia), sharding, and all Pest 5 features. Do not use for factories, seeders, migrations, controllers, models, or non-test PHP code."
license: MIT
metadata:
author: laravel
---
@php
/** @var \Laravel\Boost\Install\GuidelineAssist $assist */
@endphp
# Pest Testing 5
## Documentation
Use `search-docs` for detailed Pest 5 patterns and documentation.
## Basic Usage
### Creating Tests
All tests must be written using Pest. Use `{{ $assist->artisanCommand('make:test --pest {name}') }}`.
The `{name}` argument should include only the path and test name, but should not include the test suite.
- Incorrect: `{{ $assist->artisanCommand('make:test --pest Feature/SomeFeatureTest') }}` will generate `tests/Feature/Feature/SomeFeatureTest.php`
- Correct: `{{ $assist->artisanCommand('make:test --pest SomeControllerTest') }}` will generate `tests/Feature/SomeControllerTest.php`
- Incorrect: `{{ $assist->artisanCommand('make:test --pest --unit Unit/SomeServiceTest') }}` will generate `tests/Unit/Unit/SomeServiceTest.php`
- Correct: `{{ $assist->artisanCommand('make:test --pest --unit SomeServiceTest') }}` will generate `tests/Unit/SomeServiceTest.php`
### Test Organization
- Unit/Feature tests: `tests/Feature` and `tests/Unit` directories.
- Browser tests: `tests/Browser/` directory.
- Do NOT remove tests without approval - these are core application code.
### Basic Test Structure
Pest supports both `test()` and `it()` functions. Before writing new tests, check existing test files in the same directory to match the project's convention. Use `test()` if existing tests use `test()`, or `it()` if they use `it()`.
@boostsnippet("Basic Pest Test Example", "php")
it('is true', function () {
expect(true)->toBeTrue();
});
@endboostsnippet
### Running Tests
- Run minimal tests with filter before finalizing: `{{ $assist->artisanCommand('test --compact --filter=testName') }}`.
- Run all tests: `{{ $assist->artisanCommand('test --compact') }}`.
- Run file: `{{ $assist->artisanCommand('test --compact tests/Feature/ExampleTest.php') }}`.
- Run only tests affected by recent changes (Tia): `./vendor/bin/pest --parallel --tia`.
## Assertions
Use specific assertions (`assertSuccessful()`, `assertNotFound()`) instead of `assertStatus()`:
@boostsnippet("Pest Response Assertion", "php")
it('returns all', function () {
$this->postJson('/api/docs', [])->assertSuccessful();
});
@endboostsnippet
| Use | Instead of |
|-----|------------|
| `assertSuccessful()` | `assertStatus(200)` |
| `assertNotFound()` | `assertStatus(404)` |
| `assertForbidden()` | `assertStatus(403)` |
## Mocking
Import mock function before use: `use function Pest\Laravel\mock;`
## Datasets
Use datasets for repetitive tests (validation rules, etc.):
@boostsnippet("Pest Dataset Example", "php")
it('has emails', function (string $email) {
expect($email)->not->toBeEmpty();
})->with([
'james' => 'james@laravel.com',
'taylor' => 'taylor@laravel.com',
]);
@endboostsnippet
## Pest 5 Features
| Feature | Purpose |
|---------|---------|
| Tia (Test Impact Analysis) | Rerun only tests affected by recent changes |
| Time-Balanced Sharding | Split tests across CI shards by execution time |
| New Validation Expectations | `toBeEmail()`, `toBeUlid()`, `toBeIpAddress()`, and more |
| Browser Testing | Full integration tests in real browsers |
| Smoke Testing | Validate multiple pages quickly |
| Visual Regression | Compare screenshots for visual changes |
| Architecture Testing | Enforce code conventions |
### Tia (Test Impact Analysis)
Tia reruns only tests affected by recent changes and replays cached results for the rest, dramatically reducing suite duration:
@boostsnippet("Tia Example", "shell")
./vendor/bin/pest --parallel --tia
@endboostsnippet
- Replayed tests are not skipped — cached tests store everything they produced, including covered lines and branches.
- Detects Laravel, Symfony, Livewire, and Inertia automatically.
### New Validation Expectations
Pest 5 ships eight new validation matchers, all supporting `.not` negation:
@boostsnippet("Pest 5 Validation Expectations", "php")
expect('nuno@pestphp.com')->toBeEmail();
expect('01ARZ3NDEKTSV4RRFFQ69G5FAV')->toBeUlid();
expect('192.168.1.1')->toBeIpAddress();
expect('00:1a:2b:3c:4d:5e')->toBeMacAddress();
expect('example.com')->toBeHostname();
expect('example.co.uk')->toBeDomain();
expect('Zm9vYmFy')->toBeBase64();
expect('deadbeef')->toBeHexadecimal();
@endboostsnippet
### Time-Balanced Sharding
Distribute tests across CI shards by execution time rather than count:
@boostsnippet("Pest Sharding Example", "shell")
./vendor/bin/pest --update-shards
./vendor/bin/pest --shard=1/4
@endboostsnippet
Commit `tests/.pest/shards.json` to the repository so CI shards stay consistent.
### Browser Test Example
Browser tests run in real browsers for full integration testing:
- Browser tests live in `tests/Browser/`.
- Use Laravel features like `Event::fake()`, `assertAuthenticated()`, and model factories.
- Use `RefreshDatabase` for clean state per test.
- Interact with page: click, type, scroll, select, submit, drag-and-drop, touch gestures.
- Test on multiple browsers (Chrome, Firefox, Safari) if requested.
- Test on different devices/viewports (iPhone 14 Pro, tablets) if requested.
- Switch color schemes (light/dark mode) when appropriate.
- Take screenshots or pause tests for debugging.
@boostsnippet("Pest Browser Test Example", "php")
it('may reset the password', function () {
Notification::fake();
$this->actingAs(User::factory()->create());
$page = visit('/sign-in');
$page->assertSee('Sign In')
->assertNoJavaScriptErrors()
->click('Forgot Password?')
->fill('email', 'nuno@laravel.com')
->click('Send Reset Link')
->assertSee('We have emailed your password reset link!');
Notification::assertSent(ResetPassword::class);
});
@endboostsnippet
### Smoke Testing
Quickly validate multiple pages have no JavaScript errors:
@boostsnippet("Pest Smoke Testing Example", "php")
$pages = visit(['/', '/about', '/contact']);
$pages->assertNoJavaScriptErrors()->assertNoConsoleLogs();
@endboostsnippet
### Visual Regression Testing
Capture and compare screenshots to detect visual changes.
### Architecture Testing
@boostsnippet("Architecture Test Example", "php")
arch('controllers')
->expect('App\Http\Controllers')
->toExtendNothing()
->toHaveSuffix('Controller');
@endboostsnippet
## Common Pitfalls
- Not importing `use function Pest\Laravel\mock;` before using mock
- Using `assertStatus(200)` instead of `assertSuccessful()`
- Forgetting datasets for repetitive validation tests
- Deleting tests without approval
- Forgetting `assertNoJavaScriptErrors()` in browser tests
- Prefixing `Feature/` or `Unit/` in `{name}` when using `make:test`
+2
View File
@@ -5,6 +5,8 @@
[$bgBadgeColor, $bgBadgeText] = match ($type) {
'INFO' => ['blue', 'INFO'],
'ERROR' => ['red', 'ERROR'],
'WARN' => ['yellow', 'WARN'],
'SUCCESS' => ['green', 'SUCCESS'],
};
?>
@@ -7,7 +7,7 @@
<div>
<span class="text-gray mr-1">- </span>
<span>composer require pestphp/pest-plugin-browser:^4.0 --dev</span>
<span>composer require pestphp/pest-plugin-browser:^5.0 --dev</span>
</div>
<div>
+1 -1
View File
@@ -53,7 +53,7 @@ abstract class AbstractPreset // @pest-arch-ignore-line
/**
* Runs the given callback for each namespace.
*
* @param callable(Expectation<string|null>): ArchExpectation ...$callbacks
* @param callable(Expectation<string>): ArchExpectation ...$callbacks
*/
final public function eachUserNamespace(callable ...$callbacks): void
{
+4 -1
View File
@@ -69,6 +69,7 @@ final class Laravel extends AbstractPreset
->toHaveSuffix('Request');
$this->expectations[] = expect('App\Http\Requests')
->classes()
->toExtend('Illuminate\Foundation\Http\FormRequest');
$this->expectations[] = expect('App\Http\Requests')
@@ -118,6 +119,7 @@ final class Laravel extends AbstractPreset
->toHaveMethod('handle');
$this->expectations[] = expect('App\Notifications')
->classes()
->toExtend('Illuminate\Notifications\Notification');
$this->expectations[] = expect('App')
@@ -128,6 +130,7 @@ final class Laravel extends AbstractPreset
->toHaveSuffix('ServiceProvider');
$this->expectations[] = expect('App\Providers')
->classes()
->toExtend('Illuminate\Support\ServiceProvider');
$this->expectations[] = expect('App\Providers')
@@ -150,7 +153,7 @@ final class Laravel extends AbstractPreset
->toHaveSuffix('Controller');
$this->expectations[] = expect('App\Http')
->toOnlyBeUsedIn('App\Http');
->toOnlyBeUsedIn(['App\Http', 'App\Providers']);
$this->expectations[] = expect('App\Http\Controllers')
->not->toHavePublicMethodsBesides(['__construct', '__invoke', 'index', 'show', 'create', 'store', 'edit', 'update', 'destroy', 'middleware']);
+2
View File
@@ -18,9 +18,11 @@ final class BootOverrides implements Bootstrapper
* @var array<int, string>
*/
public const array FILES = [
'ParaTest/WrapperRunner/ProgressPrinterOutput.php',
'Runner/Filter/NameFilterIterator.php',
'Runner/ResultCache/DefaultResultCache.php',
'Runner/TestSuiteLoader.php',
'Runner/TestSuiteSorter.php',
'TextUI/Command/Commands/WarmCodeCoverageCacheCommand.php',
'TextUI/Output/Default/ProgressPrinter/Subscriber/TestSkippedSubscriber.php',
'TextUI/TestSuiteFilterProcessor.php',
@@ -0,0 +1,19 @@
<?php
declare(strict_types=1);
namespace Pest\Bootstrappers;
use Pest\Contracts\Bootstrapper;
use PHPUnit\TextUI\Configuration\Builder;
/**
* @internal
*/
final class BootPhpUnitConfiguration implements Bootstrapper
{
public function boot(): void
{
(new Builder)->build(['pest']);
}
}
+17
View File
@@ -25,6 +25,23 @@ final readonly class BootSubscribers implements Bootstrapper
Subscribers\EnsureIgnorableTestCasesAreIgnored::class,
Subscribers\EnsureKernelDumpIsFlushed::class,
Subscribers\EnsureTeamCityEnabled::class,
Subscribers\EnsureTiaIsRunningPestTestsOnly::class,
Subscribers\EnsureTiaStarts::class,
Subscribers\EnsureTiaEnds::class,
Subscribers\EnsureTiaResultsAreCollected::class,
Subscribers\EnsureTiaResultIsRecordedOnPassed::class,
Subscribers\EnsureTiaResultIsRecordedOnFailed::class,
Subscribers\EnsureTiaResultIsRecordedOnErrored::class,
Subscribers\EnsureTiaResultIsRecordedOnSkipped::class,
Subscribers\EnsureTiaResultIsRecordedOnIncomplete::class,
Subscribers\EnsureTiaResultIsRecordedOnRisky::class,
Subscribers\EnsureTiaResultIsRecordedOnNoticeTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpNoticeTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnDeprecationTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpDeprecationTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnWarningTriggered::class,
Subscribers\EnsureTiaResultIsRecordedOnPhpWarningTriggered::class,
Subscribers\EnsureTiaAssertionsAreRecordedOnFinished::class,
];
/**
+4
View File
@@ -8,6 +8,8 @@ use Closure;
/**
* @internal
*
* @template T of object
*/
trait Extendable
{
@@ -20,6 +22,8 @@ trait Extendable
/**
* Register a new extend.
*
* @param-closure-this T $extend
*/
public function extend(string $name, Closure $extend): void
{
+1 -1
View File
@@ -66,6 +66,6 @@ trait Pipeable
*/
private function pipes(string $name, object $context, string $scope): array
{
return array_map(fn (Closure $pipe): \Closure => $pipe->bindTo($context, $scope), self::$pipes[$name] ?? []);
return array_map(fn (Closure $pipe): Closure => $pipe->bindTo($context, $scope), self::$pipes[$name] ?? []);
}
}
+171 -7
View File
@@ -7,14 +7,24 @@ namespace Pest\Concerns;
use Closure;
use Pest\Exceptions\DatasetArgumentsMismatch;
use Pest\Panic;
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\Collectors;
use Pest\Plugins\Tia\Enums\ReplayType;
use Pest\Plugins\Tia\Recorder;
use Pest\Preset;
use Pest\Support\ChainableClosure;
use Pest\Support\Container;
use Pest\Support\ExceptionTrace;
use Pest\Support\Reflection;
use Pest\Support\Shell;
use Pest\TestSuite;
use PHPUnit\Framework\AssertionFailedError;
use PHPUnit\Framework\Attributes\PostCondition;
use PHPUnit\Framework\IncompleteTest;
use PHPUnit\Framework\SkippedTest;
use PHPUnit\Framework\TestCase;
use PHPUnit\Framework\TestCase\ExceptionExpectation;
use PHPUnit\Framework\TestCase\OutputBuffer;
use ReflectionException;
use ReflectionFunction;
use ReflectionParameter;
@@ -73,6 +83,17 @@ trait Testable
*/
public bool $__ran = false;
/**
* The active replay mode for this test, set in `setUp()` and checked
* in `__runTest()` / `tearDown()` to skip the body and after-each.
*/
private ReplayType $__replay = ReplayType::None;
/**
* The cached assertion count to replay, captured when entering replay mode.
*/
private int $__replayAssertions = 0;
/**
* The test's test closure.
*/
@@ -129,7 +150,7 @@ trait Testable
*/
public function __addBeforeAll(?Closure $hook): void
{
if (! $hook instanceof \Closure) {
if (! $hook instanceof Closure) {
return;
}
@@ -143,7 +164,7 @@ trait Testable
*/
public function __addAfterAll(?Closure $hook): void
{
if (! $hook instanceof \Closure) {
if (! $hook instanceof Closure) {
return;
}
@@ -173,7 +194,7 @@ trait Testable
*/
private function __addHook(string $property, ?Closure $hook): void
{
if (! $hook instanceof \Closure) {
if (! $hook instanceof Closure) {
return;
}
@@ -257,8 +278,37 @@ trait Testable
self::$__latestIssues = $method->issues;
self::$__latestPrs = $method->prs;
/** @var Tia $tia */
$tia = Container::getInstance()->get(Tia::class);
$status = $tia->getStatus(self::$__filename, $this->valueObjectForEvents()->id());
$replay = ReplayType::fromStatus($status);
if ($replay !== ReplayType::None) {
assert($status !== null);
$this->__replay = $replay;
match ($replay) {
ReplayType::Pass, ReplayType::Risky => $this->__beginReplay($replay, $tia),
ReplayType::Skipped => $this->markTestSkipped($status->message()),
ReplayType::Incomplete => $this->markTestIncomplete($status->message()),
ReplayType::Failure => throw new AssertionFailedError($status->message() ?: 'Cached failure'),
};
return;
}
$recorder = Container::getInstance()->get(Recorder::class);
assert($recorder instanceof Recorder);
if ($recorder->isActive()) {
$recorder->beginTest($this::class, $this->name(), self::$__filename);
}
parent::setUp();
Collectors::armAll($recorder);
$beforeEach = TestSuite::getInstance()->beforeEach->get(self::$__filename)[1];
if ($this->__beforeEach instanceof Closure) {
@@ -268,6 +318,13 @@ trait Testable
$this->__callClosure($beforeEach, $arguments);
}
private function __beginReplay(ReplayType $replay, Tia $tia): void
{
$this->__replay = $replay;
$this->__replayAssertions = $tia->getAssertionCount($this->valueObjectForEvents()->id());
$this->__ran = true;
}
/**
* Initialize test case properties from TestSuite.
*/
@@ -300,6 +357,12 @@ trait Testable
*/
protected function tearDown(...$arguments): void
{
if ($this->__replay !== ReplayType::None) {
TestSuite::getInstance()->test = null;
return;
}
$afterEach = TestSuite::getInstance()->afterEach->get(self::$__filename);
if ($this->__afterEach instanceof Closure) {
@@ -325,10 +388,106 @@ trait Testable
*/
private function __runTest(Closure $closure, ...$args): mixed
{
if ($this->__replay === ReplayType::Pass || $this->__replay === ReplayType::Risky) {
if ($this->__replay === ReplayType::Pass && $this->__replayAssertions === 0) {
$this->expectNotToPerformAssertions();
}
$this->addToAssertionCount($this->__replayAssertions);
return null;
}
$arguments = $this->__resolveTestArguments($args);
$this->__ensureDatasetArgumentNameAndNumberMatches($arguments);
return $this->__callClosure($closure, $arguments);
$method = TestSuite::getInstance()->tests->get(self::$__filename)->getMethod($this->name());
if ($method->flakyTries === null) {
return $this->__callClosure($closure, $arguments);
}
$lastException = null;
$initialProperties = get_object_vars($this);
for ($attempt = 1; $attempt <= $method->flakyTries; $attempt++) {
try {
return $this->__callClosure($closure, $arguments);
} catch (Throwable $e) {
if ($e instanceof SkippedTest
|| $e instanceof IncompleteTest
|| $this->__isExpectedException($e)) {
throw $e;
}
$lastException = $e;
if ($attempt < $method->flakyTries) {
if ($this->__snapshotChanges !== []) {
throw $e;
}
$this->tearDown();
Closure::bind(fn (): array => $this->mockObjects = [], $this, TestCase::class)();
foreach (array_keys(array_diff_key(get_object_vars($this), $initialProperties)) as $property) {
unset($this->{$property});
}
$outputBuffer = Closure::bind(fn () => $this->outputBuffer, $this, TestCase::class)();
if ($outputBuffer->hasExpectation()) {
ob_clean();
Closure::bind(function (): void {
$this->expectedString = null;
$this->expectedRegularExpression = null;
}, $outputBuffer, OutputBuffer::class)();
}
$this->setUp();
}
}
}
throw $lastException;
}
/**
* Determines if the given exception matches PHPUnit's expected exception.
*/
private function __isExpectedException(Throwable $e): bool
{
$expectation = Closure::bind(fn () => $this->exceptionExpectation, $this, TestCase::class)();
$read = fn (string $property): mixed => Closure::bind(fn () => $this->{$property}, $expectation, ExceptionExpectation::class)();
$expectedClass = $read('expectedException');
if ($expectedClass !== null) {
return $e instanceof $expectedClass;
}
$expectedMessage = $read('expectedMessage');
if ($expectedMessage !== null) {
return str_contains($e->getMessage(), (string) $expectedMessage);
}
$expectedMessageRegex = $read('expectedMessageRegularExpression');
if ($expectedMessageRegex !== null) {
return preg_match($expectedMessageRegex, $e->getMessage()) === 1;
}
$expectedCode = $read('expectedCode');
if ($expectedCode !== null) {
return $e->getCode() === $expectedCode;
}
return false;
}
/**
@@ -350,7 +509,8 @@ trait Testable
}
$underlyingTest = Reflection::getFunctionVariable($this->__test, 'closure');
$testParameterTypes = array_values(Reflection::getFunctionArguments($underlyingTest));
$testParameterTypesByName = Reflection::getFunctionArguments($underlyingTest);
$testParameterTypes = array_values($testParameterTypesByName);
if (count($arguments) !== 1) {
foreach ($arguments as $argumentIndex => $argumentValue) {
@@ -358,7 +518,11 @@ trait Testable
continue;
}
if (in_array($testParameterTypes[$argumentIndex], [Closure::class, 'callable', 'mixed'])) {
$parameterType = is_string($argumentIndex)
? $testParameterTypesByName[$argumentIndex]
: $testParameterTypes[$argumentIndex];
if (in_array($parameterType, [Closure::class, 'callable', 'mixed'])) {
continue;
}
@@ -384,7 +548,7 @@ trait Testable
return [$boundDatasetResult];
}
return array_values($boundDatasetResult);
return $boundDatasetResult;
}
/**
+19 -3
View File
@@ -33,7 +33,7 @@ final readonly class Configuration
*/
public function in(string ...$targets): UsesCall
{
return (new UsesCall($this->filename, []))->in(...$targets);
return new UsesCall($this->filename, [])->in(...$targets);
}
/**
@@ -60,7 +60,7 @@ final readonly class Configuration
*/
public function group(string ...$groups): UsesCall
{
return (new UsesCall($this->filename, []))->group(...$groups);
return new UsesCall($this->filename, [])->group(...$groups);
}
/**
@@ -68,7 +68,7 @@ final readonly class Configuration
*/
public function only(): void
{
(new BeforeEachCall(TestSuite::getInstance(), $this->filename))->only();
new BeforeEachCall(TestSuite::getInstance(), $this->filename)->only();
}
/**
@@ -119,6 +119,22 @@ final readonly class Configuration
return new Browser\Configuration;
}
/**
* Gets the evals configuration.
*/
public function evals(): Evals\Configuration // @phpstan-ignore-line
{
return new Evals\Configuration; // @phpstan-ignore-line
}
/**
* Gets the TIA (Test Impact Analysis) configuration.
*/
public function tia(): Plugins\Tia\Configuration
{
return new Plugins\Tia\Configuration;
}
/**
* Proxies calls to the uses method.
*
+7 -4
View File
@@ -59,12 +59,15 @@ final class Project
/**
* Sets the test project to GitLab.
*/
public function gitlab(string $project): self
public function gitlab(string $project, string $hostname = 'gitlab.com'): self
{
$this->issues = "https://gitlab.com/{$project}/issues/%s";
$this->prs = "https://gitlab.com/{$project}/merge_requests/%s";
// Simple way to ensure only the host is used
$hostname = parse_url($hostname, PHP_URL_HOST) ?? $hostname;
$this->assignees = 'https://gitlab.com/%s';
$this->issues = "https://{$hostname}/{$project}/-/work_items/%s";
$this->prs = "https://{$hostname}/{$project}/-/merge_requests/%s";
$this->assignees = "https://{$hostname}/%s";
return $this;
}
+16
View File
@@ -0,0 +1,16 @@
<?php
declare(strict_types=1);
namespace Pest\Contracts;
/**
* @internal
*/
interface Restarter
{
/**
* @param array<int, string> $arguments
*/
public function maybeRestart(string $projectRoot, array $arguments): void;
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllAlreadyExist extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllWithinDescribe extends InvalidArgumentException implements E
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterBeforeTestFunction extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct('After method cannot be used with before the [test|it] functions in the filename `['.$filename.']`.');
parent::__construct('After method cannot be used with before the [test|it] functions in the filename ['.$filename.'].');
}
}
+54
View File
@@ -0,0 +1,54 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use Pest\Support\View;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class BaselineFetchFailed extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct(
private readonly string $headline,
private readonly string $hint,
private readonly bool $hasAnchor = false,
) {
parent::__construct($headline);
}
public function render(OutputInterface $output): void
{
View::renderUsing($output);
if (! $this->hasAnchor) {
View::render('components.badge', ['type' => 'ERROR', 'content' => $this->headline]);
$this->renderChild($output, $this->hint.' Or use [--fresh] to record locally.');
$output->writeln('');
return;
}
$this->renderChild($output, $this->headline);
$this->renderChild($output, $this->hint.' Or use [--fresh] to record locally.');
$output->writeln('');
}
public function exitCode(): int
{
return 1;
}
private function renderChild(OutputInterface $output, string $text): void
{
$output->writeln(sprintf(' <fg=gray>─ %s</>', $text));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllAlreadyExist extends InvalidArgumentException implements Ex
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllWithinDescribe extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetAlreadyExists extends InvalidArgumentException implements Exc
*/
public function __construct(string $name, string $scope)
{
parent::__construct(sprintf('A dataset with the name `%s` already exists in scope [%s].', $name, $scope));
parent::__construct(sprintf('A dataset with the name [%s] already exists in scope [%s].', $name, $scope));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetDoesNotExist extends InvalidArgumentException implements Exce
*/
public function __construct(string $name)
{
parent::__construct(sprintf("A dataset with the name `%s` does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
parent::__construct(sprintf("A dataset with the name [%s] does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
}
}
+19
View File
@@ -0,0 +1,19 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use RuntimeException;
use Throwable;
/**
* @internal
*/
final class DatasetProviderError extends RuntimeException
{
public function __construct(Throwable $previous)
{
parent::__construct($previous->getMessage(), (int) $previous->getCode(), $previous);
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class FileOrFolderNotFound extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The file or folder with the name `%s` could not be found.', $filename));
parent::__construct(sprintf('The file or folder with the name [%s] could not be found.', $filename));
}
}
+41
View File
@@ -0,0 +1,41 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use InvalidArgumentException;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Symfony\Component\Console\Exception\ExceptionInterface;
/**
* @internal
*/
final class InvalidTestClassName extends InvalidArgumentException implements ExceptionInterface, RenderlessEditor, RenderlessTrace
{
/**
* Creates a new Exception instance for the given class name.
*/
public static function fromClassName(string $filename, string $className): self
{
return new self(sprintf(
'The test file [%s] would create the class [%s], which is not a valid PHP class name. Please rename the test file.',
$filename,
$className,
));
}
/**
* Creates a new Exception instance for the given namespace.
*/
public static function fromNamespace(string $filename, string $namespace, string $part): self
{
return new self(sprintf(
'The test file [%s] would create the namespace [%s], which is not a valid PHP namespace, as [%s] may not be used as a namespace name. Please rename the folder in question.',
$filename,
$namespace,
$part,
));
}
}
+32
View File
@@ -0,0 +1,32 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use InvalidArgumentException;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class NoAffectedTestsFound extends InvalidArgumentException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=blue> INFO </> No affected tests found.',
'',
]);
}
public function exitCode(): int
{
return 0;
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestAlreadyExist extends InvalidArgumentException implements Excepti
*/
public function __construct(string $fileName, string $description)
{
parent::__construct(sprintf('A test with the description `%s` already exists in the filename `%s`.', $description, $fileName));
parent::__construct(sprintf('A test with the description [%s] already exists in the filename [%s].', $description, $fileName));
}
}
@@ -19,6 +19,6 @@ final class TestCaseClassOrTraitNotFound extends InvalidArgumentException implem
*/
public function __construct(string $testCaseClass)
{
parent::__construct(sprintf('The class `%s` was not found.', $testCaseClass));
parent::__construct(sprintf('The class [%s] was not found.', $testCaseClass));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestDescriptionMissing extends InvalidArgumentException implements E
*/
public function __construct(string $fileName)
{
parent::__construct(sprintf('Test description is missing in the filename `%s`.', $fileName));
parent::__construct(sprintf('Test description is missing in the filename [%s].', $fileName));
}
}
@@ -0,0 +1,52 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresDefaultBranch extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct()
{
parent::__construct(
'Tia mode could not determine the default branch every other branch falls back to reading.',
);
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode could not determine the default branch.',
'',
' It is the branch whose baseline every other branch falls back to reading, and',
' nothing in this checkout names it: the repository has a remote, but no',
' <fg=yellow>origin/HEAD</>, and no CI provider stated it either. Guessing would re-run the',
' whole suite on every new branch while reporting it as a cache hit.',
'',
' Name the branch in <fg=yellow>tests/Pest.php</>:',
'',
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
'',
' Or let git answer, once per clone:',
'',
' <fg=yellow>git remote set-head origin --auto</>',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
+46
View File
@@ -0,0 +1,46 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresPestTests extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct(private readonly string $className, string $filename)
{
parent::__construct(sprintf(
'Tia mode requires only functional based Pest tests, but encountered PHPUnit class [%s] in [%s].',
$className,
$filename,
));
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires Pest tests.',
'',
sprintf(' Encountered PHPUnit class <fg=yellow>%s</>', $this->className),
sprintf(' in <fg=gray>%s</>.', $this->file),
'',
' Convert it to a Pest test, or run without Tia.',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
+47
View File
@@ -0,0 +1,47 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresRemote extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct()
{
parent::__construct(
'Tia mode requires a repository with a remote, so the default branch every other branch falls back to reading can be resolved.',
);
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires a repository with a remote.',
'',
' Without one there is no way to tell which branch is the default — the branch',
' whose baseline every other branch falls back to reading — so the first run on',
' each new branch would silently re-run the whole suite.',
'',
' Add a remote, or name the branch yourself in <fg=yellow>tests/Pest.php</>:',
'',
' <fg=yellow>pest()->tia()->defaultBranch(\'master\');</>',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
@@ -0,0 +1,44 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Pest\Contracts\Panicable;
use RuntimeException;
use Symfony\Component\Console\Exception\ExceptionInterface;
use Symfony\Component\Console\Output\OutputInterface;
/**
* @internal
*/
final class TiaRequiresRepositoryRoot extends RuntimeException implements ExceptionInterface, Panicable, RenderlessEditor, RenderlessTrace
{
public function __construct(private readonly string $subdirectoryPrefix)
{
parent::__construct(sprintf(
'Tia mode requires the project root to be the git repository root, but it sits in the subdirectory [%s] of a larger repo.',
$this->subdirectoryPrefix,
));
}
public function render(OutputInterface $output): void
{
$output->writeln([
'',
' <fg=white;options=bold;bg=red> ERROR </> Tia mode requires the git repository root.',
'',
sprintf(' This project sits in a subdirectory of a larger repo <fg=yellow>%s</>.', $this->subdirectoryPrefix),
'',
' Give the project its own git repository to use Tia.',
'',
]);
}
public function exitCode(): int
{
return 1;
}
}
+71 -20
View File
@@ -18,6 +18,7 @@ use Pest\Arch\Expectations\ToOnlyUse;
use Pest\Arch\Expectations\ToUse;
use Pest\Arch\Expectations\ToUseNothing;
use Pest\Arch\PendingArchExpectation;
use Pest\Arch\Support\Composer;
use Pest\Arch\Support\FileLineFinder;
use Pest\Concerns\Extendable;
use Pest\Concerns\Pipeable;
@@ -52,7 +53,9 @@ use ReflectionProperty;
*/
final class Expectation
{
/** @use Extendable<self<TValue>> */
use Extendable;
use Pipeable;
use Retrievable;
@@ -109,7 +112,7 @@ final class Expectation
if (function_exists('dump')) {
dump($this->value, ...$arguments);
} else {
var_dump($this->value);
var_dump($this->value, ...$arguments);
}
return $this;
@@ -117,16 +120,22 @@ final class Expectation
/**
* Dump the expectation value and end the script.
*
* @return never
*/
public function dd(mixed ...$arguments): void
public function dd(mixed ...$arguments): never
{
if (function_exists('dd')) {
dd($this->value, ...$arguments);
}
var_dump($this->value);
if (getenv('PARATEST') !== false || isset($_SERVER['COLLISION_PRINTER'])) {
ob_start();
var_dump($this->value, ...$arguments);
$output = ob_get_clean();
throw new ExpectationFailedException($output);
}
var_dump($this->value, ...$arguments);
exit(1);
}
@@ -134,7 +143,7 @@ final class Expectation
/**
* Dump the expectation value when the result of the condition is truthy.
*
* @param (\Closure(TValue): bool)|bool $condition
* @param (Closure(TValue): bool)|bool $condition
* @return self<TValue>
*/
public function ddWhen(Closure|bool $condition, mixed ...$arguments): Expectation
@@ -151,7 +160,7 @@ final class Expectation
/**
* Dump the expectation value when the result of the condition is falsy.
*
* @param (\Closure(TValue): bool)|bool $condition
* @param (Closure(TValue): bool)|bool $condition
* @return self<TValue>
*/
public function ddUnless(Closure|bool $condition, mixed ...$arguments): Expectation
@@ -235,7 +244,7 @@ final class Expectation
if ($callbacks[$index] instanceof Closure) {
$callbacks[$index](new self($value), new self($key));
} else {
(new self($value))->toEqual($callbacks[$index]);
new self($value)->toEqual($callbacks[$index]);
}
$index = isset($callbacks[$index + 1]) ? $index + 1 : 0;
@@ -667,6 +676,41 @@ final class Expectation
throw InvalidExpectation::fromMethods(['toHavePrivateMethods']);
}
/**
* Asserts that the given expectation target is cased correctly.
*/
public function toBeCasedCorrectly(): ArchExpectation
{
return Targeted::make(
$this,
function (ObjectDescription $object): bool {
if (! isset($object->reflectionClass)) {
return false;
}
$realPath = realpath($object->path);
if ($realPath === false) {
return false;
}
foreach (Composer::allNamespacesWithDirectories() as $directory => $namespace) {
if (str_starts_with($realPath, $directory)) {
$relativePath = substr($realPath, strlen($directory) + 1);
$relativePath = explode('.', $relativePath)[0];
$classFromPath = $namespace.'\\'.str_replace(DIRECTORY_SEPARATOR, '\\', $relativePath);
return $classFromPath === $object->reflectionClass->getName();
}
}
return false;
},
'to be cased correctly',
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
}
/**
* Asserts that the given expectation target is enum.
*/
@@ -781,7 +825,22 @@ final class Expectation
return false;
}
if (! in_array($trait, $object->reflectionClass->getTraitNames(), true)) {
$currentClass = $object->reflectionClass;
$usedTraits = [];
do {
$classTraits = $currentClass->getTraits();
foreach ($classTraits as $traitReflection) {
$usedTraits[$traitReflection->getName()] = $traitReflection->getName();
$nestedTraits = $traitReflection->getTraits();
foreach ($nestedTraits as $nestedTrait) {
$usedTraits[$nestedTrait->getName()] = $nestedTrait->getName();
}
}
} while ($currentClass = $currentClass->getParentClass());
if (! array_key_exists($trait, $usedTraits)) {
return false;
}
}
@@ -862,15 +921,7 @@ final class Expectation
return Targeted::make(
$this,
function (ObjectDescription $object) use ($interfaces): bool {
foreach ($interfaces as $interface) {
if (! isset($object->reflectionClass) || ! $object->reflectionClass->implementsInterface($interface)) {
return false;
}
}
return true;
},
fn (ObjectDescription $object): bool => array_all($interfaces, fn (string $interface): bool => isset($object->reflectionClass) && $object->reflectionClass->implementsInterface($interface)),
"to implement '".implode("', '", $interfaces)."'",
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
@@ -1085,8 +1136,8 @@ final class Expectation
$this,
fn (ObjectDescription $object): bool => isset($object->reflectionClass)
&& $object->reflectionClass->isEnum()
&& (new ReflectionEnum($object->name))->isBacked() // @phpstan-ignore-line
&& (string) (new ReflectionEnum($object->name))->getBackingType() === $backingType, // @phpstan-ignore-line
&& new ReflectionEnum($object->name)->isBacked() // @phpstan-ignore-line
&& (string) new ReflectionEnum($object->name)->getBackingType() === $backingType, // @phpstan-ignore-line
'to be '.$backingType.' backed enum',
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
+2 -2
View File
@@ -112,7 +112,7 @@ final class HigherOrderExpectation
* Dynamically calls methods on the class with the given arguments.
*
* @param array<int, mixed> $arguments
* @return self<TOriginalValue, mixed>|self<TOriginalValue, TValue>
* @return self<TOriginalValue, mixed>
*/
public function __call(string $name, array $arguments): self
{
@@ -127,7 +127,7 @@ final class HigherOrderExpectation
/**
* Accesses properties in the value or in the expectation.
*
* @return self<TOriginalValue, mixed>|self<TOriginalValue, TValue>
* @return self<TOriginalValue, mixed>
*/
public function __get(string $name): self
{
+6 -24
View File
@@ -576,15 +576,7 @@ final readonly class OppositeExpectation
return Targeted::make(
$original,
function (ObjectDescription $object) use ($traits): bool {
foreach ($traits as $trait) {
if (isset($object->reflectionClass) && in_array($trait, $object->reflectionClass->getTraitNames(), true)) {
return false;
}
}
return true;
},
fn (ObjectDescription $object): bool => array_all($traits, fn (string $trait): bool => ! isset($object->reflectionClass) || ! in_array($trait, $object->reflectionClass->getTraitNames(), true)),
"not to use traits '".implode("', '", $traits)."'",
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
@@ -604,15 +596,7 @@ final readonly class OppositeExpectation
return Targeted::make(
$original,
function (ObjectDescription $object) use ($interfaces): bool {
foreach ($interfaces as $interface) {
if (isset($object->reflectionClass) && $object->reflectionClass->implementsInterface($interface)) {
return false;
}
}
return true;
},
fn (ObjectDescription $object): bool => array_all($interfaces, fn (string $interface): bool => ! isset($object->reflectionClass) || ! $object->reflectionClass->implementsInterface($interface)),
"not to implement '".implode("', '", $interfaces)."'",
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
@@ -814,13 +798,11 @@ final readonly class OppositeExpectation
$exporter = Exporter::default();
$toString = fn (mixed $argument): string => $exporter->shortenedExport($argument);
throw new ExpectationFailedException(sprintf(
'Expecting %s not %s %s.',
$toString($this->original->value),
$exporter->shortenedExport($this->original->value),
strtolower((string) preg_replace('/(?<!\ )[A-Z]/', ' $0', $name)),
implode(' ', array_map(fn (mixed $argument): string => $toString($argument), $arguments)),
implode(' ', array_map(fn (mixed $argument): string => $exporter->export($argument), $arguments)),
));
}
@@ -852,8 +834,8 @@ final readonly class OppositeExpectation
$original,
fn (ObjectDescription $object): bool => isset($object->reflectionClass) === false
|| ! $object->reflectionClass->isEnum()
|| ! (new \ReflectionEnum($object->name))->isBacked() // @phpstan-ignore-line
|| (string) (new \ReflectionEnum($object->name))->getBackingType() !== $backingType, // @phpstan-ignore-line
|| ! new \ReflectionEnum($object->name)->isBacked() // @phpstan-ignore-line
|| (string) new \ReflectionEnum($object->name)->getBackingType() !== $backingType, // @phpstan-ignore-line
'not to be '.$backingType.' backed enum',
FileLineFinder::where(fn (string $line): bool => str_contains($line, 'class')),
);
+28 -7
View File
@@ -9,6 +9,7 @@ use Pest\Concerns;
use Pest\Contracts\HasPrintableTestCaseName;
use Pest\Evaluators\Attributes;
use Pest\Exceptions\DatasetMissing;
use Pest\Exceptions\InvalidTestClassName;
use Pest\Exceptions\ShouldNotHappen;
use Pest\Exceptions\TestAlreadyExist;
use Pest\Exceptions\TestClosureMustNotBeStatic;
@@ -17,6 +18,7 @@ use Pest\Factories\Concerns\HigherOrderable;
use Pest\Support\Reflection;
use Pest\Support\Str;
use Pest\TestSuite;
use PHPUnit\Framework\Attributes\TestDox;
use PHPUnit\Framework\TestCase;
use RuntimeException;
@@ -58,6 +60,11 @@ final class TestCaseFactory
Concerns\Expectable::class,
];
/**
* The namespace for the test case, overrides the path-based namespace when set.
*/
public ?string $namespace = null;
/**
* Creates a new Factory instance.
*/
@@ -88,7 +95,8 @@ final class TestCaseFactory
$filename = (string) preg_replace_callback('~^(?P<drive>[a-z]+:\\\)~i', static fn (array $match): string => strtolower($match['drive']), $filename);
}
$filename = str_replace('\\\\', '\\', addslashes((string) realpath($filename)));
$realpath = (string) realpath($filename);
$filename = str_replace('\\\\', '\\', addslashes($realpath));
$rootPath = TestSuite::getInstance()->rootPath;
$relativePath = str_replace($rootPath.DIRECTORY_SEPARATOR, '', $filename);
@@ -110,8 +118,8 @@ final class TestCaseFactory
$relativePath = (string) preg_replace('|%[a-fA-F0-9][a-fA-F0-9]|', '', $relativePath);
// Remove escaped quote sequences (maintain namespace)
$relativePath = str_replace(array_map(fn (string $quote): string => sprintf('\\%s', $quote), ['\'', '"']), '', $relativePath);
// Limit to A-Z, a-z, 0-9, '_', '-'.
$relativePath = (string) preg_replace('/[^A-Za-z0-9\\\\]/', '', $relativePath);
// Limit to Unicode letters and numbers.
$relativePath = (string) preg_replace('/[^\p{L}\p{N}\\\\]/u', '', $relativePath);
$classFQN = 'P\\'.$relativePath;
@@ -126,16 +134,26 @@ final class TestCaseFactory
$partsFQN = explode('\\', $classFQN);
$className = array_pop($partsFQN);
$namespace = implode('\\', $partsFQN);
$namespace = $this->namespace ?? implode('\\', $partsFQN);
$baseClass = sprintf('\%s', $this->class);
if (trim($className) === '') {
$className = 'InvalidTestName'.Str::random();
} elseif (! Str::isValidClassName($className)) {
throw InvalidTestClassName::fromClassName($this->filename, $className);
}
if ($this->namespace === null) {
foreach ($partsFQN as $partFQN) {
if (! Str::isValidIdentifier($partFQN)) {
throw InvalidTestClassName::fromNamespace($this->filename, $namespace, $partFQN);
}
}
}
$this->attributes = [
new Attribute(
\PHPUnit\Framework\Attributes\TestDox::class,
TestDox::class,
[$this->filename],
),
...$this->attributes,
@@ -143,6 +161,8 @@ final class TestCaseFactory
$attributesCode = Attributes::code($this->attributes);
$filenameLiteral = var_export($realpath, true);
$methodsCode = implode('', array_map(
fn (TestCaseMethodFactory $methodFactory): string => $methodFactory->buildForEvaluation(),
$methods
@@ -152,6 +172,7 @@ final class TestCaseFactory
$classCode = <<<PHP
namespace $namespace;
use Pest\Exceptions\DatasetProviderError as __PestDatasetProviderError;
use Pest\Repositories\DatasetsRepository as __PestDatasets;
use Pest\TestSuite as __PestTestSuite;
@@ -160,7 +181,7 @@ final class TestCaseFactory
final class $className extends $baseClass implements $hasPrintableTestCaseClassFQN {
$traitsCode
private static \$__filename = '$filename';
public static \$__filename = $filenameLiteral;
$methodsCode
}
@@ -191,7 +212,7 @@ final class TestCaseFactory
if (
$method->closure instanceof \Closure &&
(new \ReflectionFunction($method->closure))->isStatic()
new \ReflectionFunction($method->closure)->isStatic()
) {
throw new TestClosureMustNotBeStatic($method);
+22 -5
View File
@@ -9,10 +9,14 @@ use Pest\Evaluators\Attributes;
use Pest\Exceptions\ShouldNotHappen;
use Pest\Factories\Concerns\HigherOrderable;
use Pest\Repositories\DatasetsRepository;
use Pest\Support\Description;
use Pest\Support\Str;
use Pest\TestSuite;
use PHPUnit\Framework\Assert;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\Depends;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\Attributes\TestDox;
use PHPUnit\Framework\TestCase;
/**
@@ -32,7 +36,7 @@ final class TestCaseMethodFactory
/**
* The test's describing, if any.
*
* @var array<int, \Pest\Support\Description>
* @var array<int, Description>
*/
public array $describing = [];
@@ -46,6 +50,11 @@ final class TestCaseMethodFactory
*/
public int $repetitions = 1;
/**
* The test's number of flaky retry tries.
*/
public ?int $flakyTries = null;
/**
* Determines if the test is a "todo".
*/
@@ -192,11 +201,11 @@ final class TestCaseMethodFactory
$this->attributes = [
new Attribute(
\PHPUnit\Framework\Attributes\Test::class,
Test::class,
[],
),
new Attribute(
\PHPUnit\Framework\Attributes\TestDox::class,
TestDox::class,
[str_replace('*/', '{@*}', $this->description)],
),
...$this->attributes,
@@ -206,7 +215,7 @@ final class TestCaseMethodFactory
$depend = Str::evaluable($this->describing === [] ? $depend : Str::describe($this->describing, $depend));
$this->attributes[] = new Attribute(
\PHPUnit\Framework\Attributes\Depends::class,
Depends::class,
[$depend],
);
}
@@ -226,6 +235,10 @@ final class TestCaseMethodFactory
$attributesCode
public function $methodName(...\$arguments)
{
if (count(\$arguments) === 1 && \$arguments[0] instanceof __PestDatasetProviderError) {
throw \$arguments[0]->getPrevious() ?? \$arguments[0];
}
return \$this->__runTest(
\$this->__test,
...\$arguments,
@@ -252,7 +265,11 @@ final class TestCaseMethodFactory
public static function $dataProviderName()
{
return __PestDatasets::get(self::\$__filename, "$methodName");
try {
return __PestDatasets::get(self::\$__filename, "$methodName");
} catch (\Throwable \$throwable) {
return [[new __PestDatasetProviderError(\$throwable)]];
}
}
EOF;
+19 -37
View File
@@ -4,7 +4,6 @@ declare(strict_types=1);
use Pest\Browser\Api\ArrayablePendingAwaitablePage;
use Pest\Browser\Api\PendingAwaitablePage;
use Pest\Concerns\Expectable;
use Pest\Configuration;
use Pest\Exceptions\AfterAllWithinDescribe;
use Pest\Exceptions\BeforeAllWithinDescribe;
@@ -48,7 +47,7 @@ if (! function_exists('beforeAll')) {
function beforeAll(Closure $closure): void
{
if (DescribeCall::describing() !== []) {
$filename = Backtrace::file();
$filename = Backtrace::testFile();
throw new BeforeAllWithinDescribe($filename);
}
@@ -61,13 +60,11 @@ if (! function_exists('beforeEach')) {
/**
* Runs the given closure before each test in the current file.
*
* @param-closure-this TestCase $closure
*
* @return HigherOrderTapProxy<Expectable|TestCall|TestCase>|Expectable|TestCall|TestCase|mixed
* @param-closure-this TestCall $closure
*/
function beforeEach(?Closure $closure = null): BeforeEachCall
{
$filename = Backtrace::file();
$filename = Backtrace::testFile();
return new BeforeEachCall(TestSuite::getInstance(), $filename, $closure);
}
@@ -92,8 +89,6 @@ if (! function_exists('describe')) {
* Adds the given closure as a group of tests. The first argument
* is the group description; the second argument is a closure
* that contains the group tests.
*
* @return HigherOrderTapProxy<Expectable|TestCall|TestCase>|Expectable|TestCall|TestCase|mixed
*/
function describe(string $description, Closure $tests): DescribeCall
{
@@ -112,7 +107,7 @@ if (! function_exists('uses')) {
*/
function uses(string ...$classAndTraits): UsesCall
{
$filename = Backtrace::file();
$filename = Backtrace::testFile();
return new UsesCall($filename, array_values($classAndTraits));
}
@@ -124,7 +119,7 @@ if (! function_exists('pest')) {
*/
function pest(): Configuration
{
return new Configuration(Backtrace::file());
return new Configuration(Backtrace::testFile());
}
}
@@ -134,13 +129,13 @@ if (! function_exists('test')) {
* is the test description; the second argument is
* a closure that contains the test expectations.
*
* @param-closure-this TestCase $closure
* @param-closure-this TestCall $closure
*
* @return Expectable|TestCall|TestCase|mixed
* @return ($description is string ? TestCall : HigherOrderTapProxy|TestCall)
*/
function test(?string $description = null, ?Closure $closure = null): HigherOrderTapProxy|TestCall
{
if ($description === null && TestSuite::getInstance()->test instanceof \PHPUnit\Framework\TestCase) {
if ($description === null && TestSuite::getInstance()->test instanceof TestCase) {
return new HigherOrderTapProxy(TestSuite::getInstance()->test);
}
@@ -156,34 +151,23 @@ if (! function_exists('it')) {
* is the test description; the second argument is
* a closure that contains the test expectations.
*
* @param-closure-this TestCase $closure
*
* @return Expectable|TestCall|TestCase|mixed
* @param-closure-this TestCall $closure
*/
function it(string $description, ?Closure $closure = null): TestCall
{
$description = sprintf('it %s', $description);
/** @var TestCall $test */
$test = test($description, $closure);
return $test;
return test($description, $closure);
}
}
if (! function_exists('todo')) {
/**
* Creates a new test that is marked as "todo".
*
* @return Expectable|TestCall|TestCase|mixed
*/
function todo(string $description): TestCall
{
$test = test($description);
assert($test instanceof TestCall);
return $test->todo();
return test($description)->todo();
}
}
@@ -191,13 +175,11 @@ if (! function_exists('afterEach')) {
/**
* Runs the given closure after each test in the current file.
*
* @param-closure-this TestCase $closure
*
* @return Expectable|HigherOrderTapProxy<Expectable|TestCall|TestCase>|TestCall|mixed
* @param-closure-this TestCall $closure
*/
function afterEach(?Closure $closure = null): AfterEachCall
{
$filename = Backtrace::file();
$filename = Backtrace::testFile();
return new AfterEachCall(TestSuite::getInstance(), $filename, $closure);
}
@@ -210,7 +192,7 @@ if (! function_exists('afterAll')) {
function afterAll(Closure $closure): void
{
if (DescribeCall::describing() !== []) {
$filename = Backtrace::file();
$filename = Backtrace::testFile();
throw new AfterAllWithinDescribe($filename);
}
@@ -227,7 +209,7 @@ if (! function_exists('covers')) {
*/
function covers(array|string ...$classesOrFunctions): void
{
$filename = Backtrace::file();
$filename = Backtrace::testFile();
$beforeEachCall = (new BeforeEachCall(TestSuite::getInstance(), $filename));
@@ -236,7 +218,7 @@ if (! function_exists('covers')) {
/** @var MutationTestRunner $runner */
$runner = Container::getInstance()->get(MutationTestRunner::class);
/** @var \Pest\Mutate\Repositories\ConfigurationRepository $configurationRepository */
/** @var ConfigurationRepository $configurationRepository */
$configurationRepository = Container::getInstance()->get(ConfigurationRepository::class);
$everything = $configurationRepository->cliConfiguration->toArray()['everything'] ?? false;
$classes = $configurationRepository->cliConfiguration->toArray()['classes'] ?? false;
@@ -256,14 +238,14 @@ if (! function_exists('mutates')) {
*/
function mutates(array|string ...$targets): void
{
$filename = Backtrace::file();
$filename = Backtrace::testFile();
$beforeEachCall = (new BeforeEachCall(TestSuite::getInstance(), $filename));
$beforeEachCall->group('__pest_mutate_only');
/** @var MutationTestRunner $runner */
$runner = Container::getInstance()->get(MutationTestRunner::class);
/** @var \Pest\Mutate\Repositories\ConfigurationRepository $configurationRepository */
/** @var ConfigurationRepository $configurationRepository */
$configurationRepository = Container::getInstance()->get(ConfigurationRepository::class);
$everything = $configurationRepository->cliConfiguration->toArray()['everything'] ?? false;
$classes = $configurationRepository->cliConfiguration->toArray()['classes'] ?? false;
@@ -320,7 +302,7 @@ if (! function_exists('visit')) {
*/
function visit(array|string $url, array $options = []): ArrayablePendingAwaitablePage|PendingAwaitablePage
{
if (! class_exists(\Pest\Browser\Configuration::class)) {
if (! class_exists(Pest\Browser\Configuration::class)) {
PluginBrowser::install();
exit(0);
+33 -9
View File
@@ -27,8 +27,13 @@ use Whoops\Exception\Inspector;
/**
* @internal
*/
final readonly class Kernel
final class Kernel
{
/**
* Either the kernel is terminated or not.
*/
private bool $terminated = false;
/**
* The Kernel bootstrappers.
*
@@ -36,6 +41,8 @@ final readonly class Kernel
*/
private const array BOOTSTRAPPERS = [
Bootstrappers\BootOverrides::class,
Bootstrappers\BootPhpUnitConfiguration::class,
Plugins\Tia\Bootstrapper::class,
Bootstrappers\BootSubscribers::class,
Bootstrappers\BootFiles::class,
Bootstrappers\BootView::class,
@@ -43,15 +50,22 @@ final readonly class Kernel
Bootstrappers\BootExcludeList::class,
];
/**
* The Kernel restarters — resolved and invoked from `bin/pest`
* before any other Pest class is touched, so the list is exposed
* on the Kernel rather than driven from `bin/pest` directly.
*
* @var array<int, class-string<Contracts\Restarter>>
*/
public const array RESTARTERS = [
Restarters\XdebugRestarter::class,
Restarters\PcovRestarter::class,
];
/**
* Creates a new Kernel instance.
*/
public function __construct(
private Application $application,
private OutputInterface $output,
) {
//
}
public function __construct(private readonly Application $application, private readonly OutputInterface $output) {}
/**
* Boots the Kernel.
@@ -112,9 +126,13 @@ final readonly class Kernel
$configuration = Registry::get();
$result = Facade::result();
return CallsAddsOutput::execute(
$result = CallsAddsOutput::execute(
Result::exitCode($configuration, $result),
);
$this->terminate();
return $result;
}
/**
@@ -122,6 +140,12 @@ final readonly class Kernel
*/
public function terminate(): void
{
if ($this->terminated) {
return;
}
$this->terminated = true;
$preBufferOutput = Container::getInstance()->get(KernelDump::class);
assert($preBufferOutput instanceof KernelDump);
@@ -139,7 +163,7 @@ final readonly class Kernel
$this->terminate();
if (is_array($error = error_get_last())) {
if (! in_array($error['type'], [E_ERROR, E_CORE_ERROR], true)) {
if (! in_array($error['type'], [E_ERROR, E_COMPILE_ERROR, E_CORE_ERROR], true)) {
return;
}
+10 -2
View File
@@ -4,6 +4,7 @@ declare(strict_types=1);
namespace Pest;
use Laravel\Pao\Execution;
use Pest\Support\View;
use Symfony\Component\Console\Output\OutputInterface;
@@ -28,6 +29,10 @@ final class KernelDump
*/
public function enable(): void
{
if (class_exists(Execution::class) && Execution::running()) {
return;
}
ob_start(function (string $message): string {
$this->buffer .= $message;
@@ -68,6 +73,10 @@ final class KernelDump
$type = 'INFO';
if (is_array($error = error_get_last()) && in_array($error['type'], [E_ERROR, E_COMPILE_ERROR, E_CORE_ERROR], true)) {
return;
}
if ($this->isInternalError($this->buffer)) {
$type = 'ERROR';
$this->buffer = str_replace(
@@ -107,7 +116,6 @@ final class KernelDump
*/
private function isInternalError(string $output): bool
{
return str_contains($output, 'An error occurred inside PHPUnit.')
|| str_contains($output, 'Fatal error');
return str_contains($output, 'An error occurred inside PHPUnit.');
}
}
+25 -22
View File
@@ -12,12 +12,9 @@ use PHPUnit\Event\Code\Test;
use PHPUnit\Event\Code\TestMethod;
use PHPUnit\Event\Code\Throwable;
use PHPUnit\Event\Test\AfterLastTestMethodErrored;
use PHPUnit\Event\Test\AfterLastTestMethodFailed;
use PHPUnit\Event\Test\BeforeFirstTestMethodErrored;
use PHPUnit\Event\Test\ConsideredRisky;
use PHPUnit\Event\Test\Errored;
use PHPUnit\Event\Test\Failed;
use PHPUnit\Event\Test\MarkedIncomplete;
use PHPUnit\Event\Test\Skipped;
use PHPUnit\Event\Test\BeforeFirstTestMethodFailed;
use PHPUnit\Event\TestSuite\TestSuite;
use PHPUnit\Event\TestSuite\TestSuiteForTestMethodWithDataProvider;
use PHPUnit\Framework\Exception as FrameworkException;
@@ -151,7 +148,7 @@ final readonly class Converter
{
if ($testSuite instanceof TestSuiteForTestMethodWithDataProvider) {
$firstTest = $this->getFirstTest($testSuite);
if ($firstTest instanceof \PHPUnit\Event\Code\TestMethod) {
if ($firstTest instanceof TestMethod) {
return $this->getTestMethodNameWithoutDatasetSuffix($firstTest);
}
}
@@ -179,7 +176,7 @@ final readonly class Converter
public function getTestSuiteLocation(TestSuite $testSuite): ?string
{
$firstTest = $this->getFirstTest($testSuite);
if (! $firstTest instanceof \PHPUnit\Event\Code\TestMethod) {
if (! $firstTest instanceof TestMethod) {
return null;
}
$path = $firstTest->testDox()->prettifiedClassName();
@@ -252,23 +249,29 @@ final readonly class Converter
...$result->testMarkedIncompleteEvents(),
];
$numberOfNotPassedTests = count(
array_unique(
array_map(
function (AfterLastTestMethodErrored|BeforeFirstTestMethodErrored|Errored|Failed|Skipped|ConsideredRisky|MarkedIncomplete $event): string {
if ($event instanceof BeforeFirstTestMethodErrored
|| $event instanceof AfterLastTestMethodErrored) {
return $event->testClassName();
}
$notPassedTests = [];
return $this->getTestCaseLocation($event->test());
},
$events
)
)
);
foreach ($events as $event) {
if ($event instanceof AfterLastTestMethodErrored) {
// PHPUnit's collector does not count these towards `numberOfTestsRun`...
continue;
}
if ($event instanceof AfterLastTestMethodFailed) {
// PHPUnit's collector does not count these towards `numberOfTestsRun`...
continue;
}
if ($event instanceof BeforeFirstTestMethodErrored || $event instanceof BeforeFirstTestMethodFailed) {
$notPassedTests[] = $event->testClassName();
$numberOfPassedTests = $result->numberOfTestsRun() - $numberOfNotPassedTests;
continue;
}
$notPassedTests[] = $this->getTestCaseLocation($event->test());
}
$numberOfPassedTests = $result->numberOfTestsRun()
- count(array_unique($notPassedTests))
- $result->numberOfTestSkippedByTestSuiteSkippedEvents();
return $this->stateGenerator->fromPhpUnitTestResult($numberOfPassedTests, $result);
}
@@ -14,6 +14,10 @@ final class TestSkippedSubscriber extends Subscriber implements SkippedSubscribe
{
public function notify(Skipped $event): void
{
if ($event->message() === '__TODO__') {
return; // "todo" tests are reported in the summary, not as ignored tests...
}
$this->logger()->testSkipped($event);
}
}
+7 -1
View File
@@ -200,7 +200,7 @@ final class TeamCityLogger
public function testFinished(Finished $event): void
{
if (! $this->time instanceof \PHPUnit\Event\Telemetry\HRTime) {
if (! $this->time instanceof HRTime) {
throw ShouldNotHappen::fromMessage('Start time has not been set.');
}
@@ -241,6 +241,12 @@ final class TeamCityLogger
$telemetry->memoryUsageSinceStart(),
$telemetry->durationSincePrevious(),
$telemetry->memoryUsageSincePrevious(),
$telemetry->userCpuTimeSinceStart(),
$telemetry->systemCpuTimeSinceStart(),
$telemetry->totalCpuTimeSinceStart(),
$telemetry->userCpuTimeSincePrevious(),
$telemetry->systemCpuTimeSincePrevious(),
$telemetry->totalCpuTimeSincePrevious(),
);
}
+160 -12
View File
@@ -9,10 +9,12 @@ use Closure;
use Countable;
use DateTimeInterface;
use Error;
use Illuminate\Testing\TestResponse;
use InvalidArgumentException;
use JsonSerializable;
use Pest\Exceptions\InvalidExpectationValue;
use Pest\Matchers\Any;
use Pest\Plugins\Snapshot;
use Pest\Support\Arr;
use Pest\Support\Exporter;
use Pest\Support\NullClosure;
@@ -842,7 +844,7 @@ final class Expectation
is_object($this->value) && method_exists($this->value, 'toSnapshot') => $this->value->toSnapshot(),
is_object($this->value) && method_exists($this->value, '__toString') => $this->value->__toString(),
is_object($this->value) && method_exists($this->value, 'toString') => $this->value->toString(),
$this->value instanceof \Illuminate\Testing\TestResponse => $this->value->getContent(), // @phpstan-ignore-line
$this->value instanceof TestResponse => $this->value->getContent(), // @phpstan-ignore-line
is_array($this->value) => json_encode($this->value, JSON_THROW_ON_ERROR | JSON_PRETTY_PRINT),
$this->value instanceof Traversable => json_encode(iterator_to_array($this->value), JSON_THROW_ON_ERROR | JSON_PRETTY_PRINT),
$this->value instanceof JsonSerializable => json_encode($this->value->jsonSerialize(), JSON_THROW_ON_ERROR | JSON_PRETTY_PRINT),
@@ -850,18 +852,31 @@ final class Expectation
default => InvalidExpectationValue::expected('array|object|string'),
};
if ($snapshots->has()) {
[$filename, $content] = $snapshots->get();
Assert::assertSame(
strtr($content, ["\r\n" => "\n", "\r" => "\n"]),
strtr($string, ["\r\n" => "\n", "\r" => "\n"]),
$message === '' ? "Failed asserting that the string value matches its snapshot ($filename)." : $message
);
} else {
if (! $snapshots->has()) {
$filename = $snapshots->save($string);
TestSuite::getInstance()->registerSnapshotChange("Snapshot created at [$filename]");
} else {
[$filename, $content] = $snapshots->get();
$normalizedContent = strtr($content, ["\r\n" => "\n", "\r" => "\n"]);
$normalizedString = strtr($string, ["\r\n" => "\n", "\r" => "\n"]);
if (Snapshot::$updateSnapshots && $normalizedContent !== $normalizedString) {
$snapshots->save($string);
TestSuite::getInstance()->registerSnapshotChange("Snapshot updated at [$filename]");
} else {
if (Snapshot::$updateSnapshots) {
TestSuite::getInstance()->registerSnapshotChange("Snapshot unchanged at [$filename]");
}
Assert::assertSame(
$normalizedContent,
$normalizedString,
$message === '' ? "Failed asserting that the string value matches its snapshot ($filename)." : $message
);
}
}
return $this;
@@ -921,7 +936,7 @@ final class Expectation
if ($exception instanceof Closure) {
$callback = $exception;
$parameters = (new ReflectionFunction($exception))->getParameters();
$parameters = new ReflectionFunction($exception)->getParameters();
if (count($parameters) !== 1) {
throw new InvalidArgumentException('The given closure must have a single parameter type-hinted as the class string.');
@@ -939,6 +954,7 @@ final class Expectation
} catch (Throwable $e) {
if ($exception instanceof Throwable) {
// @phpstan-ignore-next-line
expect($e)
->toBeInstanceOf($exception::class, $message)
->and($e->getMessage())->toBe($exceptionMessage ?? $exception->getMessage(), $message);
@@ -983,7 +999,7 @@ final class Expectation
*/
private function export(mixed $value): string
{
if (! $this->exporter instanceof \Pest\Support\Exporter) {
if (! $this->exporter instanceof Exporter) {
$this->exporter = Exporter::default();
}
@@ -1126,6 +1142,22 @@ final class Expectation
return $this;
}
/**
* Asserts that the value is a ULID.
*
* @return self<TValue>
*/
public function toBeUlid(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
Assert::assertTrue(Str::isUlid($this->value), $message);
return $this;
}
/**
* Asserts that the value is between 2 specified values
*
@@ -1139,6 +1171,22 @@ final class Expectation
return $this;
}
/**
* Asserts that the value is an email address.
*
* @return self<TValue>
*/
public function toBeEmail(string $message = ''): self
{
if ($message === '') {
$message = "Failed asserting that {$this->value} is an email address.";
}
Assert::assertTrue(Str::isEmail((string) $this->value), $message);
return $this;
}
/**
* Asserts that the value is a url
*
@@ -1171,4 +1219,104 @@ final class Expectation
return $this;
}
/**
* Asserts that the value is an IP address.
*
* @return self<TValue>
*/
public function toBeIpAddress(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
Assert::assertTrue((bool) filter_var($this->value, FILTER_VALIDATE_IP), $message);
return $this;
}
/**
* Asserts that the value is a MAC address.
*
* @return self<TValue>
*/
public function toBeMacAddress(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
Assert::assertTrue((bool) filter_var($this->value, FILTER_VALIDATE_MAC), $message);
return $this;
}
/**
* Asserts that the value is a hostname.
*
* @return self<TValue>
*/
public function toBeHostname(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
Assert::assertTrue((bool) filter_var($this->value, FILTER_VALIDATE_DOMAIN, FILTER_FLAG_HOSTNAME), $message);
return $this;
}
/**
* Asserts that the value is a domain name.
*
* @return self<TValue>
*/
public function toBeDomain(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
$isValid = filter_var($this->value, FILTER_VALIDATE_DOMAIN, FILTER_FLAG_HOSTNAME) !== false
&& str_contains($this->value, '.');
Assert::assertTrue($isValid, $message);
return $this;
}
/**
* Asserts that the value is a base64-encoded string.
*
* @return self<TValue>
*/
public function toBeBase64(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
$decoded = base64_decode($this->value, true);
Assert::assertTrue($decoded !== false && base64_encode($decoded) === $this->value, $message);
return $this;
}
/**
* Asserts that the value is a hexadecimal string.
*
* @return self<TValue>
*/
public function toBeHexadecimal(string $message = ''): self
{
if (! is_string($this->value)) {
InvalidExpectationValue::expected('string');
}
Assert::assertTrue(ctype_xdigit($this->value), $message);
return $this;
}
}
@@ -0,0 +1,57 @@
<?php
declare(strict_types=1);
namespace Pest\PHPStan;
use Pest\Expectations\HigherOrderExpectation;
use PhpParser\Node\Expr;
use PhpParser\Node\Expr\PropertyFetch;
use PhpParser\Node\Identifier;
use PHPStan\Analyser\Scope;
use PHPStan\Reflection\ReflectionProvider;
use PHPStan\Type\ExpressionTypeResolverExtension;
use PHPStan\Type\ObjectType;
use PHPStan\Type\Type;
/**
* Prevents native declared properties of HigherOrderExpectation (like $original,
* $expectation, $opposite, $shouldReset) from being incorrectly resolved as
* higher-order value property accesses by downstream ExpressionTypeResolverExtensions.
*
* This extension must be registered BEFORE the pest-plugin-phpstan HigherOrderExpectationTypeExtension.
*
* @internal
*/
final readonly class HigherOrderExpectationTypeExtension implements ExpressionTypeResolverExtension
{
public function __construct(
private ReflectionProvider $reflectionProvider,
) {}
public function getType(Expr $expr, Scope $scope): ?Type
{
if (! $expr instanceof PropertyFetch || ! $expr->name instanceof Identifier) {
return null;
}
$varType = $scope->getType($expr->var);
if (! new ObjectType(HigherOrderExpectation::class)->isSuperTypeOf($varType)->yes()) {
return null;
}
if (! $this->reflectionProvider->hasClass(HigherOrderExpectation::class)) {
return null;
}
$propertyName = $expr->name->name;
$classReflection = $this->reflectionProvider->getClass(HigherOrderExpectation::class);
if (! $classReflection->hasNativeProperty($propertyName)) {
return null;
}
return $varType->getProperty($propertyName, $scope)->getReadableType();
}
}
+4 -2
View File
@@ -4,6 +4,8 @@ declare(strict_types=1);
namespace Pest\PendingCalls\Concerns;
use Pest\Support\Description;
/**
* @internal
*/
@@ -12,14 +14,14 @@ trait Describable
/**
* Note: this is property is not used; however, it gets added automatically by rector php.
*
* @var array<int, \Pest\Support\Description>
* @var array<int, Description>
*/
public array $__describing;
/**
* The describing of the test case.
*
* @var array<int, \Pest\Support\Description>
* @var array<int, Description>
*/
public array $describing = [];
}
+11 -7
View File
@@ -5,7 +5,6 @@ declare(strict_types=1);
namespace Pest\PendingCalls;
use Closure;
use Pest\Support\Backtrace;
use Pest\Support\Description;
use Pest\TestSuite;
@@ -53,7 +52,11 @@ final class DescribeCall
*/
public function __destruct()
{
unset($this->currentBeforeEachCall);
// Ensure BeforeEachCall destructs before creating tests
// by moving to local scope and clearing the reference
$beforeEach = $this->currentBeforeEachCall;
$this->currentBeforeEachCall = null;
unset($beforeEach); // Trigger destructor immediately
self::$describing[] = $this->description;
@@ -71,12 +74,13 @@ final class DescribeCall
*/
public function __call(string $name, array $arguments): self
{
$filename = Backtrace::file();
if (! $this->currentBeforeEachCall instanceof BeforeEachCall) {
$this->currentBeforeEachCall = new BeforeEachCall(TestSuite::getInstance(), $this->filename);
if (! $this->currentBeforeEachCall instanceof \Pest\PendingCalls\BeforeEachCall) {
$this->currentBeforeEachCall = new BeforeEachCall(TestSuite::getInstance(), $filename);
$this->currentBeforeEachCall->describing[] = $this->description;
$this->currentBeforeEachCall->describing = array_merge(
DescribeCall::describing(),
[$this->description]
);
}
$this->currentBeforeEachCall->{$name}(...$arguments);
+22 -4
View File
@@ -22,6 +22,10 @@ use Pest\Support\NullClosure;
use Pest\Support\Str;
use Pest\TestSuite;
use PHPUnit\Framework\AssertionFailedError;
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\Attributes\CoversFunction;
use PHPUnit\Framework\Attributes\CoversTrait;
use PHPUnit\Framework\Attributes\Group;
use PHPUnit\Framework\TestCase;
/**
@@ -211,7 +215,7 @@ final class TestCall // @phpstan-ignore-line
{
foreach ($groups as $group) {
$this->testCaseMethod->attributes[] = new Attribute(
\PHPUnit\Framework\Attributes\Group::class,
Group::class,
[$group],
);
}
@@ -408,6 +412,20 @@ final class TestCall // @phpstan-ignore-line
return $this;
}
/**
* Marks the test as flaky, retrying it up to the given number of times.
*/
public function flaky(int $tries = 3): self
{
if ($tries < 1) {
throw new InvalidArgumentException('The number of tries must be greater than 0.');
}
$this->testCaseMethod->flakyTries = $tries;
return $this;
}
/**
* Marks the test as "todo".
*/
@@ -604,7 +622,7 @@ final class TestCall // @phpstan-ignore-line
{
foreach ($classes as $class) {
$this->testCaseFactoryAttributes[] = new Attribute(
\PHPUnit\Framework\Attributes\CoversClass::class,
CoversClass::class,
[$class],
);
}
@@ -627,7 +645,7 @@ final class TestCall // @phpstan-ignore-line
{
foreach ($traits as $trait) {
$this->testCaseFactoryAttributes[] = new Attribute(
\PHPUnit\Framework\Attributes\CoversTrait::class,
CoversTrait::class,
[$trait],
);
}
@@ -650,7 +668,7 @@ final class TestCall // @phpstan-ignore-line
{
foreach ($functions as $function) {
$this->testCaseFactoryAttributes[] = new Attribute(
\PHPUnit\Framework\Attributes\CoversFunction::class,
CoversFunction::class,
[$function],
);
}
+1 -3
View File
@@ -53,9 +53,7 @@ final class UsesCall
$this->targets = [$filename];
}
/**
* @deprecated Use `pest()->printer()->compact()` instead.
*/
#[\Deprecated(message: 'Use `pest()->printer()->compact()` instead.')]
public function compact(): self
{
DefaultPrinter::compact(true);
+1 -1
View File
@@ -6,7 +6,7 @@ namespace Pest;
function version(): string
{
return '4.3.1';
return '5.0.3';
}
function testDirectory(string $file = ''): string
+1 -1
View File
@@ -50,7 +50,7 @@ final class Cache implements HandlesArguments
}
}
if (! $this->hasArgument('--parallel', $arguments)) {
if (! $this->hasArgument('--parallel', $arguments) && ! $this->hasArgument('--do-not-cache-result', $arguments) && ! $this->hasArgument('--cache-result', $arguments)) {
return $this->pushArgument('--cache-result', $arguments);
}
+33 -3
View File
@@ -50,10 +50,40 @@ trait HandleArguments
*/
public function popArgument(string $argument, array $arguments): array
{
$arguments = array_flip($arguments);
$key = array_search($argument, $arguments, true);
unset($arguments[$argument]);
while ($key !== false) {
unset($arguments[$key]);
$key = array_search($argument, $arguments, true);
}
return array_values(array_flip($arguments));
return array_values($arguments);
}
/**
* Pops the given argument and its value from the arguments, returning the value.
*
* @param array<int, string> $arguments
*/
public function popArgumentValue(string $argument, array &$arguments): ?string
{
foreach ($arguments as $key => $value) {
if (str_contains($value, "$argument=")) {
unset($arguments[$key]);
$arguments = array_values($arguments);
return substr($value, strlen($argument) + 1);
}
if ($value === $argument && isset($arguments[$key + 1])) {
$result = $arguments[$key + 1];
unset($arguments[$key], $arguments[$key + 1]);
$arguments = array_values($arguments);
return $result;
}
}
return null;
}
}
+17 -5
View File
@@ -17,12 +17,16 @@ use Symfony\Component\Console\Output\OutputInterface;
*/
final class Coverage implements AddsOutput, HandlesArguments
{
use Concerns\HandleArguments;
private const string COVERAGE_OPTION = 'coverage';
private const string MIN_OPTION = 'min';
private const string EXACTLY_OPTION = 'exactly';
private const string ONLY_COVERED_OPTION = 'only-covered';
/**
* Whether it should show the coverage or not.
*/
@@ -43,6 +47,11 @@ final class Coverage implements AddsOutput, HandlesArguments
*/
public ?float $coverageExactly = null;
/**
* Whether it should show only covered files.
*/
public bool $showOnlyCovered = false;
/**
* Creates a new Plugin instance.
*/
@@ -57,7 +66,7 @@ final class Coverage implements AddsOutput, HandlesArguments
public function handleArguments(array $originals): array
{
$arguments = [...[''], ...array_values(array_filter($originals, function (string $original): bool {
foreach ([self::COVERAGE_OPTION, self::MIN_OPTION, self::EXACTLY_OPTION] as $option) {
foreach ([self::COVERAGE_OPTION, self::MIN_OPTION, self::EXACTLY_OPTION, self::ONLY_COVERED_OPTION] as $option) {
if ($original === sprintf('--%s', $option)) {
return true;
}
@@ -70,16 +79,15 @@ final class Coverage implements AddsOutput, HandlesArguments
return false;
}))];
$originals = array_flip($originals);
foreach ($arguments as $argument) {
unset($originals[$argument]);
$originals = $this->popArgument($argument, $originals);
}
$originals = array_flip($originals);
$inputs = [];
$inputs[] = new InputOption(self::COVERAGE_OPTION, null, InputOption::VALUE_NONE);
$inputs[] = new InputOption(self::MIN_OPTION, null, InputOption::VALUE_REQUIRED);
$inputs[] = new InputOption(self::EXACTLY_OPTION, null, InputOption::VALUE_REQUIRED);
$inputs[] = new InputOption(self::ONLY_COVERED_OPTION, null, InputOption::VALUE_NONE);
$input = new ArgvInput($arguments, new InputDefinition($inputs));
if ((bool) $input->getOption(self::COVERAGE_OPTION)) {
@@ -120,6 +128,10 @@ final class Coverage implements AddsOutput, HandlesArguments
$this->coverageExactly = (float) $exactlyOption;
}
if ((bool) $input->getOption(self::ONLY_COVERED_OPTION)) {
$this->showOnlyCovered = true;
}
if ($_SERVER['COLLISION_PRINTER_COMPACT'] ?? false) {
$this->compact = true;
}
@@ -144,7 +156,7 @@ final class Coverage implements AddsOutput, HandlesArguments
exit(1);
}
$coverage = \Pest\Support\Coverage::report($this->output, $this->compact);
$coverage = \Pest\Support\Coverage::report($this->output, $this->compact, $this->showOnlyCovered);
$exitCode = (int) ($coverage < $this->coverageMin);
if ($exitCode === 0 && $this->coverageExactly !== null) {
+55
View File
@@ -107,6 +107,48 @@ final readonly class Help implements HandlesArguments
'desc' => 'Initialise a standard Pest configuration',
]], ...$content['Configuration']];
$content['AI'] = [
[
'arg' => '--ai',
'desc' => 'Run a code snippet as a fully scaffolded test for AI verification',
],
];
$content['Tia'] = [
[
'arg' => '--tia',
'desc' => 'Re-run only the tests affected by your changes, replaying the rest from cache',
],
[
'arg' => '--no-tia',
'desc' => 'Disable test impact analysis for this run',
],
[
'arg' => '--tia --fresh',
'desc' => 'Discard the recorded dependency graph and record it again',
],
[
'arg' => '--tia --filtered',
'desc' => 'Narrow the run to the affected test files only',
],
[
'arg' => '--tia --locally',
'desc' => 'Enable test impact analysis on local machines only',
],
[
'arg' => '--tia --baselined',
'desc' => 'Fetch the shared dependency graph recorded by the CI baseline',
],
[
'arg' => '--tia --refetch',
'desc' => 'Force a fresh fetch of the shared dependency graph',
],
[
'arg' => '--baseline',
'desc' => 'Output to standard output the test impact analysis storage directory',
],
];
$content['Execution'] = [...[
[
'arg' => '--parallel',
@@ -116,6 +158,10 @@ final readonly class Help implements HandlesArguments
'arg' => '--update-snapshots',
'desc' => 'Update snapshots for tests using the "toMatchSnapshot" expectation',
],
[
'arg' => '--update-shards',
'desc' => 'Update shards.json with test timing data for time-balanced sharding',
],
], ...$content['Execution']];
$content['Selection'] = [[
@@ -145,6 +191,9 @@ final readonly class Help implements HandlesArguments
], [
'arg' => '--dirty',
'desc' => 'Only run tests that have uncommitted changes according to Git',
], [
'arg' => '--flaky',
'desc' => 'Output to standard output tests marked as flaky',
], ...$content['Selection']];
$content['Reporting'] = [...$content['Reporting'], ...[
@@ -160,6 +209,12 @@ final readonly class Help implements HandlesArguments
], [
'arg' => '--coverage --min',
'desc' => 'Set the minimum required coverage percentage, and fail if not met',
], [
'arg' => '--coverage --exactly',
'desc' => 'Set the exact required coverage percentage, and fail if not met',
], [
'arg' => '--coverage --only-covered',
'desc' => 'Hide files with 0% coverage from the code coverage report',
], ...$content['Code Coverage']];
$content['Mutation Testing'] = [[
+19 -9
View File
@@ -34,7 +34,7 @@ final class Parallel implements HandlesArguments
/**
* @var string[]
*/
private const array UNSUPPORTED_ARGUMENTS = ['--todo', '--todos', '--retry', '--notes', '--issue', '--pr', '--pull-request'];
private const array UNSUPPORTED_ARGUMENTS = ['--todo', '--todos', '--retry', '--notes', '--issue', '--pr', '--pull-request', '--flaky'];
/**
* Whether the given command line arguments indicate that the test suite should be run in parallel.
@@ -127,7 +127,9 @@ final class Parallel implements HandlesArguments
$arguments
);
$exitCode = $this->paratestCommand()->run(new ArgvInput($filteredArguments), new CleanConsoleOutput);
$filteredArguments = $this->processTeamcityArguments($filteredArguments);
$exitCode = $this->paratestCommand()->run(new ArgvInput(array_values($filteredArguments)), new CleanConsoleOutput);
return CallsAddsOutput::execute($exitCode);
}
@@ -176,13 +178,7 @@ final class Parallel implements HandlesArguments
{
$arguments = new ArgvInput;
foreach (self::UNSUPPORTED_ARGUMENTS as $unsupportedArgument) {
if ($arguments->hasParameterOption($unsupportedArgument)) {
return true;
}
}
return false;
return array_any(self::UNSUPPORTED_ARGUMENTS, fn (string|array $unsupportedArgument): bool => $arguments->hasParameterOption($unsupportedArgument));
}
/**
@@ -197,4 +193,18 @@ final class Parallel implements HandlesArguments
return $this->popArgument('-p', $arguments);
}
/**
* @param string[] $arguments
* @return string[]
*/
public function processTeamcityArguments(array $arguments): array
{
$argv = new ArgvInput;
if ($argv->hasParameterOption('--teamcity')) {
$arguments[] = '--teamcity';
}
return $arguments;
}
}
+3 -2
View File
@@ -7,6 +7,7 @@ namespace Pest\Plugins\Parallel\Handlers;
use Closure;
use Composer\InstalledVersions;
use Illuminate\Testing\ParallelRunner;
use Orchestra\Testbench\TestCase;
use ParaTest\Options;
use ParaTest\RunnerInterface;
use Pest\Contracts\Plugins\HandlesArguments;
@@ -39,13 +40,13 @@ final class Laravel implements HandlesArguments
* Executes the given closure when running Laravel.
*
* @param array<int, string> $arguments
* @param CLosure(array<int, string>): array<int, string> $closure
* @param Closure(array<int, string>): array<int, string> $closure
* @return array<int, string>
*/
private function whenUsingLaravel(array $arguments, Closure $closure): array
{
$isLaravelApplication = InstalledVersions::isInstalled('laravel/framework', false);
$isLaravelPackage = class_exists(\Orchestra\Testbench\TestCase::class);
$isLaravelPackage = class_exists(TestCase::class);
if ($isLaravelApplication && ! $isLaravelPackage) {
return $closure($arguments);
+18 -11
View File
@@ -81,7 +81,9 @@ final class ResultPrinter
public function flush(): void {}
};
$this->compactPrinter = CompactPrinter::default();
$this->compactPrinter = CompactPrinter::default(
decorated: ! in_array('--colors=never', $_SERVER['argv'] ?? [], true),
);
if (! $this->options->configuration->hasLogfileTeamcity()) {
return;
@@ -92,14 +94,13 @@ final class ResultPrinter
$this->teamcityLogFileHandle = $teamcityLogFileHandle;
}
/** @param list<SplFileInfo> $teamcityFiles */
public function printFeedback(
SplFileInfo $progressFile,
SplFileInfo $outputFile,
array $teamcityFiles
?SplFileInfo $teamcityFile,
): void {
if ($this->options->needsTeamcity) {
$teamcityProgress = $this->tailMultiple($teamcityFiles);
if ($this->options->needsTeamcity && $teamcityFile instanceof SplFileInfo) {
$teamcityProgress = $this->tailMultiple([$teamcityFile]);
if ($this->teamcityLogFileHandle !== null) {
fwrite($this->teamcityLogFileHandle, $teamcityProgress);
@@ -119,10 +120,6 @@ final class ResultPrinter
$unexpectedOutput = $this->tail($outputFile);
if ($unexpectedOutput !== '') {
if (preg_match('/^T+$/', $unexpectedOutput) > 0) {
return;
}
$this->output->write($unexpectedOutput);
}
@@ -171,8 +168,18 @@ final class ResultPrinter
$state = (new StateGenerator)->fromPhpUnitTestResult($this->passedTests, $testResult);
$this->compactPrinter->errors($state);
$this->compactPrinter->recap($state, $testResult, $duration, $this->options);
if ($testResult->numberOfTestsRun() === 0 && $state->testSuiteTestsCount() === 0) {
$this->output->writeln([
'',
' <fg=white;options=bold;bg=blue> INFO </> No tests found.',
'',
]);
}
if (! isset($_SERVER['PEST_PARALLEL_NO_OUTPUT'])) {
$this->compactPrinter->errors($state);
$this->compactPrinter->recap($state, $testResult, $duration, $this->options);
}
}
private function printFeedbackItem(string $item): void
+199 -18
View File
@@ -7,13 +7,14 @@ namespace Pest\Plugins\Parallel\Paratest;
use const DIRECTORY_SEPARATOR;
use NunoMaduro\Collision\Adapters\Phpunit\Support\ResultReflection;
use ParaTest\Coverage\CoverageMerger;
use ParaTest\JUnit\LogMerger;
use ParaTest\JUnit\Writer;
use ParaTest\Options;
use ParaTest\RunnerInterface;
use ParaTest\WrapperRunner\MissingResultsException;
use ParaTest\WrapperRunner\SuiteLoader;
use ParaTest\WrapperRunner\WrapperWorker;
use Pest\Plugins\Tia;
use Pest\Result;
use Pest\TestSuite;
use PHPUnit\Event\Facade as EventFacade;
@@ -25,11 +26,17 @@ use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
use PHPUnit\TestRunner\TestResult\TestResult;
use PHPUnit\TextUI\Configuration\CodeCoverageFilterRegistry;
use PHPUnit\Util\ExcludeList;
use ReflectionProperty;
use SebastianBergmann\CodeCoverage\Node\Builder;
use SebastianBergmann\CodeCoverage\Serialization\Merger;
use SebastianBergmann\CodeCoverage\StaticAnalysis\FileAnalyser;
use SebastianBergmann\CodeCoverage\StaticAnalysis\ParsingSourceAnalyser;
use SebastianBergmann\Timer\Timer;
use SplFileInfo;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Process\PhpExecutableFinder;
use function array_filter;
use function array_merge;
use function array_merge_recursive;
use function array_shift;
@@ -37,8 +44,11 @@ use function assert;
use function count;
use function dirname;
use function file_get_contents;
use function filesize;
use function is_file;
use function max;
use function realpath;
use function str_starts_with;
use function unlink;
use function unserialize;
use function usleep;
@@ -48,6 +58,11 @@ use function usleep;
*/
final class WrapperRunner implements RunnerInterface
{
/**
* The merged test result from the parallel run.
*/
public static ?TestResult $result = null;
/**
* The time to sleep between cycles.
*/
@@ -77,6 +92,18 @@ final class WrapperRunner implements RunnerInterface
/** @var array<int,int> */
private array $batches = [];
/** @var array<non-empty-string,true> */
private array $requiredTestResultFiles = [];
/** @var array<non-empty-string,true> */
private array $requiredCoverageFiles = [];
/** @var list<SplFileInfo> */
private array $statusFiles = [];
/** @var list<SplFileInfo> */
private array $progressFiles = [];
/** @var list<SplFileInfo> */
private array $unexpectedOutputFiles = [];
@@ -129,8 +156,10 @@ final class WrapperRunner implements RunnerInterface
/** @var array<int, non-empty-string> $parameters */
$parameters = $this->handleLaravelHerd($parameters);
$parameters = $this->handleTia($parameters);
$parameters[] = $wrapper;
$parameters[] = '--test-directory='.TestSuite::getInstance()->testPath;
$this->parameters = $parameters;
$this->codeCoverageFilterRegistry = new CodeCoverageFilterRegistry;
@@ -139,7 +168,6 @@ final class WrapperRunner implements RunnerInterface
public function run(): int
{
$directory = dirname(__DIR__);
assert($directory !== '');
ExcludeList::addDirectory($directory);
TestResultFacade::init();
EventFacade::instance()->seal();
@@ -176,6 +204,19 @@ final class WrapperRunner implements RunnerInterface
return $parameters;
}
/**
* @param array<int, non-empty-string> $parameters
* @return array<int, non-empty-string>
*/
private function handleTia(array $parameters): array
{
if (! Tia::recordsEdgesInWorkers()) {
return $parameters;
}
return array_merge($parameters, ['-d', 'pcov.directory='.TestSuite::getInstance()->rootPath]);
}
private function startWorkers(): void
{
for ($token = 1; $token <= $this->options->processes; $token++) {
@@ -206,7 +247,7 @@ final class WrapperRunner implements RunnerInterface
if (
$this->exitcode > 0
&& $this->options->configuration->stopOnFailure()
&& $this->options->configuration->stopOnFailureThreshold() > 0
) {
$this->pending = [];
} elseif (($pending = array_shift($this->pending)) !== null) {
@@ -221,11 +262,23 @@ final class WrapperRunner implements RunnerInterface
private function flushWorker(WrapperWorker $worker): void
{
if ($worker->hasExecutedTests()) {
$testResultFile = $worker->testResultFile->getPathname();
if ($testResultFile !== '') {
$this->requiredTestResultFiles[$testResultFile] = true;
}
if (isset($worker->coverageFile) && $worker->coverageFile->getPathname() !== '') {
$this->requiredCoverageFiles[$worker->coverageFile->getPathname()] = true;
}
}
$this->exitcode = max($this->exitcode, $worker->getExitCode());
$this->printer->printFeedback(
$worker->progressFile,
$worker->unexpectedOutputFile,
$this->teamcityFiles,
$worker->teamcityFile ?? null,
);
$worker->reset();
}
@@ -268,10 +321,15 @@ final class WrapperRunner implements RunnerInterface
$worker->start();
$this->batches[$token] = 0;
$this->unexpectedOutputFiles[] = $worker->unexpectedOutputFile;
$this->statusFiles[] = $worker->statusFile;
$this->progressFiles[] = $worker->progressFile;
$this->unexpectedOutputFiles[] = $worker->unexpectedOutputFile;
$this->testResultFiles[] = $worker->testResultFile;
if (isset($worker->resultCacheFile)) {
$this->resultCacheFiles[] = $worker->resultCacheFile;
}
if (isset($worker->junitFile)) {
$this->junitFiles[] = $worker->junitFile;
}
@@ -304,6 +362,20 @@ final class WrapperRunner implements RunnerInterface
private function complete(TestResult $testResultSum): int
{
$missingTestResultFiles = [];
foreach ($this->requiredTestResultFiles as $filePath => $true) {
if (is_file($filePath)) {
continue;
}
$missingTestResultFiles[] = $filePath;
}
if ($missingTestResultFiles !== []) {
throw MissingResultsException::create($missingTestResultFiles, 'test_result');
}
foreach ($this->testResultFiles as $testResultFile) {
if (! $testResultFile->isFile()) {
continue;
@@ -338,6 +410,20 @@ final class WrapperRunner implements RunnerInterface
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredWarningEvents(), $testResult->testRunnerTriggeredWarningEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssueDeprecationEvents(), $testResult->testRunnerTriggeredIssueDeprecationEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssueErrorEvents(), $testResult->testRunnerTriggeredIssueErrorEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssueNoticeEvents(), $testResult->testRunnerTriggeredIssueNoticeEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssuePhpDeprecationEvents(), $testResult->testRunnerTriggeredIssuePhpDeprecationEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssuePhpNoticeEvents(), $testResult->testRunnerTriggeredIssuePhpNoticeEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssuePhpWarningEvents(), $testResult->testRunnerTriggeredIssuePhpWarningEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->testRunnerTriggeredIssueWarningEvents(), $testResult->testRunnerTriggeredIssueWarningEvents()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->errors(), $testResult->errors()),
// @phpstan-ignore-next-line
array_merge_recursive($testResultSum->deprecations(), $testResult->deprecations()),
@@ -373,8 +459,15 @@ final class WrapperRunner implements RunnerInterface
$testResultSum->testRunnerTriggeredNoticeEvents(),
array_values(array_filter(
$testResultSum->testRunnerTriggeredWarningEvents(),
fn (WarningTriggered $event): bool => ! str_contains($event->message(), 'No tests found')
fn (WarningTriggered $event): bool => ! str_contains($event->message(), 'No tests found in class')
)),
$testResultSum->testRunnerTriggeredIssueDeprecationEvents(),
$testResultSum->testRunnerTriggeredIssueErrorEvents(),
$testResultSum->testRunnerTriggeredIssueNoticeEvents(),
$testResultSum->testRunnerTriggeredIssuePhpDeprecationEvents(),
$testResultSum->testRunnerTriggeredIssuePhpNoticeEvents(),
$testResultSum->testRunnerTriggeredIssuePhpWarningEvents(),
$testResultSum->testRunnerTriggeredIssueWarningEvents(),
$testResultSum->errors(),
$testResultSum->deprecations(),
$testResultSum->notices(),
@@ -385,6 +478,8 @@ final class WrapperRunner implements RunnerInterface
$testResultSum->numberOfIssuesIgnoredByBaseline(),
);
self::$result = $testResultSum;
if ($this->options->configuration->cacheResult()) {
$resultCacheSum = new DefaultResultCache($this->options->configuration->testResultCacheFile());
foreach ($this->resultCacheFiles as $resultCacheFile) {
@@ -408,8 +503,11 @@ final class WrapperRunner implements RunnerInterface
$exitcode = Result::exitCode($this->options->configuration, $testResultSum);
$this->clearFiles($this->statusFiles);
$this->clearFiles($this->progressFiles);
$this->clearFiles($this->unexpectedOutputFiles);
$this->clearFiles($this->testResultFiles);
$this->clearFiles($this->resultCacheFiles);
$this->clearFiles($this->coverageFiles);
$this->clearFiles($this->junitFiles);
$this->clearFiles($this->teamcityFiles);
@@ -424,6 +522,20 @@ final class WrapperRunner implements RunnerInterface
return;
}
$missingCoverageFiles = [];
foreach ($this->requiredCoverageFiles as $filePath => $true) {
if (is_file($filePath) && filesize($filePath) !== 0) {
continue;
}
$missingCoverageFiles[] = $filePath;
}
if ($missingCoverageFiles !== []) {
throw MissingResultsException::create($missingCoverageFiles, 'coverage');
}
$coverageManager = new CodeCoverage;
$coverageManager->init(
$this->options->configuration,
@@ -439,10 +551,33 @@ final class WrapperRunner implements RunnerInterface
return;
}
$coverageMerger = new CoverageMerger($coverageManager->codeCoverage());
foreach ($this->coverageFiles as $coverageFile) {
$coverageMerger->addCoverageFromFile($coverageFile);
$coverageFiles = [];
foreach ($this->coverageFiles as $fileInfo) {
$realPath = $fileInfo->getRealPath();
if ($realPath !== false && $realPath !== '') {
$coverageFiles[] = $realPath;
}
}
$serializedCoverage = (new Merger)->merge($coverageFiles);
$report = (new Builder(new FileAnalyser(new ParsingSourceAnalyser, false, false)))->build(
$serializedCoverage['codeCoverage'],
$serializedCoverage['testResults'],
$serializedCoverage['basePath'],
);
$codeCoverage = $coverageManager->codeCoverage();
$codeCoverage->excludeUncoveredFiles();
$mergedData = $serializedCoverage['codeCoverage'];
$basePath = $serializedCoverage['basePath'];
if ($basePath !== '') {
foreach ($mergedData->coveredFiles() as $relativePath) {
$mergedData->renameFile($relativePath, $basePath.DIRECTORY_SEPARATOR.$relativePath);
}
}
$codeCoverage->setData($mergedData);
$codeCoverage->setTests($serializedCoverage['testResults']);
(new ReflectionProperty(\SebastianBergmann\CodeCoverage\CodeCoverage::class, 'cachedReport'))->setValue($codeCoverage, $report);
$coverageManager->generateReports(
$this->printer->printer,
@@ -483,15 +618,61 @@ final class WrapperRunner implements RunnerInterface
*/
private function getTestFiles(SuiteLoader $suiteLoader): array
{
/** @var array<string, non-empty-string> $files */
$files = [
...array_values(array_filter(
$suiteLoader->tests,
fn (string $filename): bool => ! str_ends_with($filename, "eval()'d code")
)),
...TestSuite::getInstance()->tests->getFilenames(),
];
/** @var array<string, null> $files */
$files = [];
return $files; // @phpstan-ignore-line
foreach (array_filter(
$suiteLoader->tests,
fn (string $filename): bool => ! str_ends_with($filename, "eval()'d code")
) as $filename) {
$resolved = realpath($filename) ?: $filename;
$files[$resolved] = null;
}
foreach (TestSuite::getInstance()->tests->getFilenames() as $filename) {
if ($this->shouldIncludeBootstrappedTestFile($filename)) {
$resolved = realpath($filename)
?: realpath($this->options->cwd.DIRECTORY_SEPARATOR.$filename)
?: $filename;
$files[$resolved] = null;
}
}
return array_keys($files); // @phpstan-ignore-line
}
private function shouldIncludeBootstrappedTestFile(string $filename): bool
{
if (! $this->options->configuration->hasCliArguments()) {
return true;
}
$resolvedFilename = realpath($filename);
if ($resolvedFilename === false) {
$resolvedFilename = realpath($this->options->cwd.DIRECTORY_SEPARATOR.$filename);
}
if ($resolvedFilename === false) {
return false;
}
foreach ($this->options->configuration->cliArguments() as $path) {
$resolvedPath = realpath($path);
if ($resolvedPath === false) {
continue;
}
if ($resolvedFilename === $resolvedPath) {
return true;
}
if (is_dir($resolvedPath) && str_starts_with($resolvedFilename, $resolvedPath.DIRECTORY_SEPARATOR)) {
return true;
}
}
return false;
}
}
@@ -7,6 +7,7 @@ namespace Pest\Plugins\Parallel\Support;
use NunoMaduro\Collision\Adapters\Phpunit\State;
use NunoMaduro\Collision\Adapters\Phpunit\Style;
use ParaTest\Options;
use PHPUnit\Event\Telemetry\CpuTime;
use PHPUnit\Event\Telemetry\GarbageCollectorStatus;
use PHPUnit\Event\Telemetry\HRTime;
use PHPUnit\Event\Telemetry\Info;
@@ -62,12 +63,12 @@ final class CompactPrinter
/**
* Creates a new instance of the Compact Printer.
*/
public static function default(): self
public static function default(bool $decorated = true): self
{
return new self(
terminal(),
new ConsoleOutput(decorated: true),
new Style(new ConsoleOutput(decorated: true)),
new ConsoleOutput(decorated: $decorated),
new Style(new ConsoleOutput(decorated: $decorated)),
terminal()->width() - 4,
);
}
@@ -147,11 +148,20 @@ final class CompactPrinter
MemoryUsage::fromBytes(0),
MemoryUsage::fromBytes(0),
$garbageCollectorStatus,
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
),
$telemetryDuration,
MemoryUsage::fromBytes(0),
\PHPUnit\Event\Telemetry\Duration::fromSecondsAndNanoseconds(0, 0),
MemoryUsage::fromBytes(0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
CpuTime::fromSecondsAndNanoseconds(0, 0),
);
$this->style->writeRecap($state, $telemetry, $testResult);
+460 -14
View File
@@ -6,7 +6,13 @@ namespace Pest\Plugins;
use Pest\Contracts\Plugins\AddsOutput;
use Pest\Contracts\Plugins\HandlesArguments;
use Pest\Contracts\Plugins\Terminable;
use Pest\Exceptions\InvalidOption;
use Pest\Subscribers\EnsureShardTimingFinished;
use Pest\Subscribers\EnsureShardTimingsAreCollected;
use Pest\Subscribers\EnsureShardTimingStarted;
use Pest\TestSuite;
use PHPUnit\Event;
use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
@@ -15,12 +21,19 @@ use Symfony\Component\Process\Process;
/**
* @internal
*/
final class Shard implements AddsOutput, HandlesArguments
final class Shard implements AddsOutput, HandlesArguments, Terminable
{
use Concerns\HandleArguments;
private const string SHARD_OPTION = 'shard';
/**
* The maximum length allowed for the filter argument.
* While ARG_MAX can be 2MB, individual arguments are often limited to 128KB (MAX_ARG_STRLEN).
* Practical limits in CI environments (like Docker or pipeline runners) can be even lower.
*/
private const int MAX_FILTER_LENGTH = 32768;
/**
* The shard index and total number of shards.
*
@@ -33,6 +46,40 @@ final class Shard implements AddsOutput, HandlesArguments
*/
private static ?array $shard = null;
/**
* Whether to update the shards.json file.
*/
private static bool $updateShards = false;
/**
* Whether time-balanced sharding was used.
*/
private static bool $timeBalanced = false;
/**
* Whether the shards.json file is outdated.
*/
private static bool $shardsOutdated = false;
/**
* Whether the test suite passed.
*/
private static bool $passed = false;
/**
* Collected timings from workers or subscribers.
*
* @var array<string, float>|null
*/
private static ?array $collectedTimings = null;
/**
* The canonical list of test classes from --list-tests.
*
* @var list<string>|null
*/
private static ?array $knownTests = null;
/**
* Creates a new Plugin instance.
*/
@@ -47,6 +94,19 @@ final class Shard implements AddsOutput, HandlesArguments
*/
public function handleArguments(array $arguments): array
{
if ($this->hasArgument('--update-shards', $arguments)) {
return $this->handleUpdateShards($arguments);
}
if (Parallel::isWorker() && Parallel::getGlobal('UPDATE_SHARDS') === true) {
self::$updateShards = true;
Event\Facade::instance()->registerSubscriber(new EnsureShardTimingStarted);
Event\Facade::instance()->registerSubscriber(new EnsureShardTimingFinished);
return $arguments;
}
if (! $this->hasArgument('--shard', $arguments)) {
return $arguments;
}
@@ -63,7 +123,25 @@ final class Shard implements AddsOutput, HandlesArguments
/** @phpstan-ignore-next-line */
$tests = $this->allTests($arguments);
$testsToRun = (array_chunk($tests, max(1, (int) ceil(count($tests) / $total))))[$index - 1] ?? [];
$timings = $this->loadShardsFile();
if ($timings !== null) {
$knownTests = array_values(array_filter($tests, fn (string $test): bool => isset($timings[$test])));
$newTests = array_values(array_diff($tests, $knownTests));
$partitions = $this->partitionByTime($knownTests, $timings, $total);
foreach ($newTests as $i => $test) {
$partitions[$i % $total][] = $test;
}
$testsToRun = $partitions[$index - 1] ?? [];
self::$timeBalanced = true;
self::$shardsOutdated = $newTests !== [];
} else {
$isInCurrentShard = fn (int $key): bool => $key % $total === ($index - 1);
$testsToRun = array_values(array_filter($tests, $isInCurrentShard, ARRAY_FILTER_USE_KEY));
}
self::$shard = [
'index' => $index,
@@ -72,7 +150,45 @@ final class Shard implements AddsOutput, HandlesArguments
'testsCount' => count($tests),
];
return [...$arguments, '--filter', $this->buildFilterArgument($testsToRun)];
if ($testsToRun === []) {
return $arguments;
}
$filter = $this->buildFilterArgument($testsToRun);
$this->ensureFilterLengthIsSafe($filter);
return [...$arguments, '--filter', $filter];
}
/**
* Handles the --update-shards argument.
*
* @param array<int, string> $arguments
* @return array<int, string>
*/
private function handleUpdateShards(array $arguments): array
{
if ($this->hasArgument('--shard', $arguments)) {
throw new InvalidOption('The [--update-shards] option cannot be combined with [--shard].');
}
$arguments = $this->popArgument('--update-shards', $arguments);
self::$updateShards = true;
/** @phpstan-ignore-next-line */
self::$knownTests = $this->allTests($arguments);
if ($this->hasArgument('--parallel', $arguments) || $this->hasArgument('-p', $arguments)) {
Parallel::setGlobal('UPDATE_SHARDS', true);
Parallel::setGlobal('SHARD_RUN_ID', uniqid('pest-shard-', true));
} else {
Event\Facade::instance()->registerSubscriber(new EnsureShardTimingStarted);
Event\Facade::instance()->registerSubscriber(new EnsureShardTimingFinished);
}
return $arguments;
}
/**
@@ -83,15 +199,14 @@ final class Shard implements AddsOutput, HandlesArguments
*/
private function allTests(array $arguments): array
{
$output = (new Process([
'php',
...$this->removeParallelArguments($arguments),
'--list-tests',
]))->mustRun()->getOutput();
$command = $this->buildListTestsCommand(
$arguments,
TestSuite::getInstance()->testPath,
);
preg_match_all('/ - (?:P\\\\)?(Tests\\\\[^:]+)::/', $output, $matches);
$output = new Process($command)->setTimeout(120)->mustRun()->getOutput();
return array_values(array_unique($matches[1]));
return $this->parseListTestsOutput($output);
}
/**
@@ -100,15 +215,97 @@ final class Shard implements AddsOutput, HandlesArguments
*/
private function removeParallelArguments(array $arguments): array
{
return array_filter($arguments, fn (string $argument): bool => ! in_array($argument, ['--parallel', '-p'], strict: true));
return array_values(array_filter(
$arguments,
fn (string $argument): bool => ! in_array($argument, ['--parallel', '-p'], strict: true)
&& ! str_starts_with($argument, '--processes'),
));
}
/**
* Builds the subprocess command used to enumerate tests via `--list-tests`.
*
* @param list<string> $arguments
* @return list<string>
*/
private function buildListTestsCommand(array $arguments, string $testPath): array
{
$filtered = $this->removeParallelArguments($arguments);
return ['php', ...$filtered, '--test-directory='.$testPath, '--list-tests'];
}
/**
* Parses `--list-tests` output into a unique list of test class FQCNs.
*
* @return list<string>
*/
private function parseListTestsOutput(string $output): array
{
preg_match_all('/ - (?:P\\\\)?([A-Za-z_]\w*(?:\\\\[A-Za-z_]\w*)*)::/', $output, $matches);
return array_values(array_unique($matches[1]));
}
/**
* Builds the filter argument for the given tests to run.
*
* @param array<int, string> $testsToRun
*/
private function buildFilterArgument(mixed $testsToRun): string
private function buildFilterArgument(array $testsToRun): string
{
return addslashes(implode('|', $testsToRun));
if ($testsToRun === []) {
return '';
}
/** @var array<string, mixed> $tree */
$tree = [];
foreach ($testsToRun as $class) {
$parts = explode('\\', $class);
$current = &$tree;
foreach ($parts as $part) {
if (! isset($current[$part])) {
$current[$part] = [];
}
$current = &$current[$part];
}
}
$buildRegex = function (array $tree) use (&$buildRegex): string {
$parts = [];
foreach ($tree as $key => $sub) {
$subRegex = $buildRegex($sub);
if ($subRegex === '') {
$parts[] = preg_quote($key, '/');
} else {
$parts[] = preg_quote($key, '/').'\\\\'.(count($sub) > 1 ? '('.$subRegex.')' : $subRegex);
}
}
return implode('|', $parts);
};
return $buildRegex($tree);
}
/**
* Ensures that the filter length is safe for the current environment.
*
* @throws InvalidOption
*/
private function ensureFilterLengthIsSafe(string $filter): void
{
$maxLength = (int) (getenv('PEST_SHARD_MAX_FILTER_LENGTH') ?: self::MAX_FILTER_LENGTH);
if (strlen($filter) > $maxLength) {
throw new InvalidOption(sprintf(
'The generated filter for this shard is too long (%d characters). '.
'This can cause issues with some environments (limit is %d characters). '.
'Please increase the number of shards (e.g., use 1/4 instead of 1/2) to reduce the filter length.',
strlen($filter),
$maxLength
));
}
}
/**
@@ -116,6 +313,22 @@ final class Shard implements AddsOutput, HandlesArguments
*/
public function addOutput(int $exitCode): int
{
self::$passed = $exitCode === 0;
if (self::$updateShards && self::$passed && ! Parallel::isWorker()) {
self::$collectedTimings = $this->collectTimings();
$count = self::$knownTests !== null
? count(array_intersect_key(self::$collectedTimings, array_flip(self::$knownTests)))
: count(self::$collectedTimings);
$this->output->writeln(sprintf(
' <fg=gray>Shards:</> <fg=default>shards.json updated with timings for %d test class%s.</>',
$count,
$count === 1 ? '' : 'es',
));
}
if (self::$shard === null) {
return $exitCode;
}
@@ -128,17 +341,250 @@ final class Shard implements AddsOutput, HandlesArguments
] = self::$shard;
$this->output->writeln(sprintf(
' <fg=gray>Shard:</> <fg=default>%d of %d</> — %d file%s ran, out of %d.',
' <fg=gray>Shard:</> <fg=default>%d of %d</> — %d file%s ran, out of %d%s.',
$index,
$total,
$testsRan,
$testsRan === 1 ? '' : 's',
$testsCount,
self::$timeBalanced ? ' <fg=gray>(time-balanced)</>' : '',
));
if (self::$shardsOutdated) {
$this->output->writeln(' <fg=yellow;options=bold>WARN</> <fg=default>The [tests/.pest/shards.json] file is out of date. Run [--update-shards] to update it.</>');
}
return $exitCode;
}
/**
* Terminates the plugin.
*/
public function terminate(): void
{
if (! self::$updateShards) {
return;
}
if (Parallel::isWorker()) {
$this->writeWorkerTimings();
return;
}
if (! self::$passed) {
return;
}
$timings = self::$collectedTimings ?? $this->collectTimings();
if ($timings === []) {
return;
}
$this->writeTimings($timings);
}
/**
* Collects timings from subscribers or worker temp files.
*
* @return array<string, float>
*/
private function collectTimings(): array
{
$runId = Parallel::getGlobal('SHARD_RUN_ID');
if (is_string($runId)) {
return $this->readWorkerTimings($runId);
}
return EnsureShardTimingsAreCollected::timings();
}
/**
* Writes the current worker's timing data to a temp file.
*/
private function writeWorkerTimings(): void
{
$timings = EnsureShardTimingsAreCollected::timings();
if ($timings === []) {
return;
}
$runId = Parallel::getGlobal('SHARD_RUN_ID');
if (! is_string($runId)) {
return;
}
$path = sys_get_temp_dir().DIRECTORY_SEPARATOR.'__pest_sharding_'.$runId.'-'.getmypid().'.json';
file_put_contents($path, json_encode($timings, JSON_THROW_ON_ERROR));
}
/**
* Reads and merges timing data from all worker temp files.
*
* @return array<string, float>
*/
private function readWorkerTimings(string $runId): array
{
$pattern = sys_get_temp_dir().DIRECTORY_SEPARATOR.'__pest_sharding_'.$runId.'-*.json';
$files = glob($pattern);
if ($files === false || $files === []) {
return [];
}
$merged = [];
foreach ($files as $file) {
$contents = file_get_contents($file);
if ($contents === false) {
continue;
}
$timings = json_decode($contents, true);
if (is_array($timings)) {
$merged = array_merge($merged, $timings);
}
unlink($file);
}
return $merged;
}
/**
* Returns the path to shards.json.
*/
private function shardsPath(): string
{
$testSuite = TestSuite::getInstance();
return implode(DIRECTORY_SEPARATOR, [$testSuite->rootPath, $testSuite->testPath, '.pest', 'shards.json']);
}
/**
* Loads the timings from shards.json.
*
* @return array<string, float>|null
*/
private function loadShardsFile(): ?array
{
$path = $this->shardsPath();
if (! file_exists($path)) {
return null;
}
$contents = file_get_contents($path);
if ($contents === false) {
throw new InvalidOption('The [tests/.pest/shards.json] file could not be read. Delete it or run [--update-shards] to regenerate.');
}
$data = json_decode($contents, true);
if (! is_array($data) || ! isset($data['timings']) || ! is_array($data['timings'])) {
throw new InvalidOption('The [tests/.pest/shards.json] file is corrupted. Delete it or run [--update-shards] to regenerate.');
}
return $data['timings'];
}
/**
* Partitions tests across shards using the LPT (Longest Processing Time) algorithm.
*
* @param list<string> $tests
* @param array<string, float> $timings
* @return list<list<string>>
*/
private function partitionByTime(array $tests, array $timings, int $total): array
{
$knownTimings = array_filter(
array_map(fn (string $test): ?float => $timings[$test] ?? null, $tests),
fn (?float $t): bool => $t !== null,
);
$median = $knownTimings !== [] ? $this->median(array_values($knownTimings)) : 1.0;
$testsWithTimings = array_map(
fn (string $test): array => ['test' => $test, 'time' => $timings[$test] ?? $median],
$tests,
);
usort($testsWithTimings, fn (array $a, array $b): int => $b['time'] <=> $a['time']);
/** @var list<list<string>> */
$bins = array_fill(0, $total, []);
/** @var non-empty-list<float> */
$binTimes = array_fill(0, $total, 0.0);
foreach ($testsWithTimings as $item) {
$minIndex = array_search(min($binTimes), $binTimes, strict: true);
assert(is_int($minIndex));
$bins[$minIndex][] = $item['test'];
$binTimes[$minIndex] += $item['time'];
}
return $bins;
}
/**
* Calculates the median of an array of floats.
*
* @param list<float> $values
*/
private function median(array $values): float
{
sort($values);
$count = count($values);
$middle = (int) floor($count / 2);
if ($count % 2 === 0) {
return ($values[$middle - 1] + $values[$middle]) / 2;
}
return $values[$middle];
}
/**
* Writes the timings to shards.json.
*
* @param array<string, float> $timings
*/
private function writeTimings(array $timings): void
{
$path = $this->shardsPath();
$directory = dirname($path);
if (! is_dir($directory)) {
mkdir($directory, 0755, true);
}
if (self::$knownTests !== null) {
$knownSet = array_flip(self::$knownTests);
$timings = array_intersect_key($timings, $knownSet);
}
ksort($timings);
$canonical = self::$knownTests ?? array_keys($timings);
sort($canonical);
file_put_contents($path, json_encode([
'timings' => $timings,
'checksum' => md5(implode("\n", $canonical)),
'updated_at' => date('c'),
], JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES)."\n");
}
/**
* Returns the shard information.
*
+98 -4
View File
@@ -5,7 +5,6 @@ declare(strict_types=1);
namespace Pest\Plugins;
use Pest\Contracts\Plugins\HandlesArguments;
use Pest\Exceptions\InvalidOption;
use Pest\TestSuite;
/**
@@ -15,21 +14,116 @@ final class Snapshot implements HandlesArguments
{
use Concerns\HandleArguments;
/**
* Whether snapshots should be updated on this run.
*/
public static bool $updateSnapshots = false;
/**
* {@inheritDoc}
*/
public function handleArguments(array $arguments): array
{
if (Parallel::isWorker() && Parallel::getGlobal('UPDATE_SNAPSHOTS') === true) {
self::$updateSnapshots = true;
return $arguments;
}
if (! $this->hasArgument('--update-snapshots', $arguments)) {
return $arguments;
}
if ($this->hasArgument('--parallel', $arguments)) {
throw new InvalidOption('The [--update-snapshots] option is not supported when running in parallel.');
self::$updateSnapshots = true;
if ($this->isFullRun($arguments)) {
TestSuite::getInstance()->snapshots->flush();
}
TestSuite::getInstance()->snapshots->flush();
if ($this->hasArgument('--parallel', $arguments) || $this->hasArgument('-p', $arguments)) {
Parallel::setGlobal('UPDATE_SNAPSHOTS', true);
}
return $this->popArgument('--update-snapshots', $arguments);
}
/**
* Options that take a value as the next argument (rather than via "=value").
*
* @var list<string>
*/
private const array FLAGS_WITH_VALUES = [
'--filter',
'--group',
'--exclude-group',
'--test-suffix',
'--covers',
'--uses',
'--cache-directory',
'--cache-result-file',
'--configuration',
'--colors',
'--test-directory',
'--bootstrap',
'--order-by',
'--random-order-seed',
'--log-junit',
'--log-teamcity',
'--log-events-text',
'--log-events-verbose-text',
'--coverage-clover',
'--coverage-cobertura',
'--coverage-crap4j',
'--coverage-html',
'--coverage-php',
'--coverage-text',
'--coverage-xml',
'--assignee',
'--issue',
'--ticket',
'--pr',
'--pull-request',
'--retry',
'--shard',
'--repeat',
];
/**
* Determines whether the command targets the entire suite (no filter, no path).
*
* @param array<int, string> $arguments
*/
private function isFullRun(array $arguments): bool
{
if ($this->hasArgument('--filter', $arguments)) {
return false;
}
$tokens = array_slice($arguments, 1);
$skipNext = false;
foreach ($tokens as $arg) {
if ($skipNext) {
$skipNext = false;
continue;
}
if ($arg === '') {
continue;
}
if ($arg[0] === '-') {
if (in_array($arg, self::FLAGS_WITH_VALUES, true)) {
$skipNext = true;
}
continue;
}
return false;
}
return true;
}
}
+2327
View File
File diff suppressed because it is too large Load Diff
+621
View File
@@ -0,0 +1,621 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
use Pest\Exceptions\BaselineFetchFailed;
use Pest\Panic;
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\Contracts\State;
use Pest\Support\View;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Process\Process;
/**
* @internal
*/
final readonly class BaselineSync
{
private const string WORKFLOW_FILE = 'tia-baseline.yml';
private const string ARTIFACT_NAME = 'pest-tia-baseline';
private const string GRAPH_ASSET = Tia::KEY_GRAPH;
private const string COVERAGE_ASSET = Tia::KEY_COVERAGE_CACHE;
private const string DOWNLOAD_CACHE_DIR = 'artifacts';
private const int DOWNLOAD_CACHE_MAX_ENTRIES = 5;
private const int FETCH_COOLDOWN_SECONDS = 86400;
private const array DIAGNOSES = [
'network' => [
'pattern' => '/could not resolve host|connection refused|connection reset|temporary failure in name resolution|network is unreachable|no route to host|i\/o timeout|tls handshake|getaddrinfo/i',
'message' => 'network error (offline or DNS unreachable). Try again when connected.',
],
'gh-auth' => [
'pattern' => '/authentication failed|not logged in|requires authentication|bad credentials|401/i',
'message' => 'authentication failed — run `gh auth login` and retry.',
],
'rate-limit' => [
'pattern' => '/rate limit|too many requests|secondary rate limit/i',
'message' => 'GitHub API rate limit hit — try again later.',
],
'not-found' => [
'pattern' => '/404|not found|repository not found/i',
'message' => 'workflow or artifact not found in repo.',
],
'forbidden' => [
'pattern' => '/403|forbidden|access denied/i',
'message' => 'access denied — check that your `gh` token has repo + actions read scope.',
],
];
public function __construct(
private State $state,
private OutputInterface $output,
) {}
private function renderBadge(string $type, string $content): void
{
View::render('components.badge', ['type' => $type, 'content' => $content]);
}
private function renderChild(string $text): void
{
$this->output->writeln(sprintf(' <fg=gray>─ %s</>', $text));
}
public function fetchIfAvailable(string $projectRoot, bool $force = false, bool $hasAnchor = false): bool
{
$repo = $this->detectGitHubRepo($projectRoot);
if ($repo === null) {
return false;
}
if (! $force && ($remaining = $this->cooldownRemaining()) !== null) {
$this->renderBadge('WARN', sprintf(
'Last fetch found no baseline — next auto-retry in %s. Override with --refetch.',
$this->formatDuration($remaining),
));
return false;
}
$result = $this->download($repo, $projectRoot, $hasAnchor);
$payload = $result['payload'];
$failureKind = $result['failureKind'];
if ($payload === null) {
if ($failureKind === 'no-runs' || $failureKind === null) {
$this->startCooldown();
$this->emitPublishInstructions();
}
return false;
}
if (! $this->state->write(Tia::KEY_GRAPH, $payload['graph'])) {
return false;
}
if ($payload['coverage'] !== null) {
$this->state->write(Tia::KEY_COVERAGE_CACHE, $payload['coverage']);
}
$this->clearCooldown();
return true;
}
private function cooldownRemaining(): ?int
{
$raw = $this->state->read(Tia::KEY_FETCH_COOLDOWN);
if ($raw === null) {
return null;
}
$decoded = json_decode($raw, true);
if (! is_array($decoded) || ! isset($decoded['until']) || ! is_int($decoded['until'])) {
return null;
}
$remaining = $decoded['until'] - time();
return $remaining > 0 ? $remaining : null;
}
private function startCooldown(): void
{
$this->state->write(Tia::KEY_FETCH_COOLDOWN, (string) json_encode([
'until' => time() + self::FETCH_COOLDOWN_SECONDS,
]));
}
private function clearCooldown(): void
{
$this->state->delete(Tia::KEY_FETCH_COOLDOWN);
}
private function formatDuration(int $seconds): string
{
if ($seconds >= 3600) {
return (int) round($seconds / 3600).'h';
}
if ($seconds >= 60) {
return (int) round($seconds / 60).'m';
}
return $seconds.'s';
}
private function emitPublishInstructions(): void
{
if ($this->isCi()) {
$this->renderBadge('INFO', 'No baseline yet — this run will produce one.');
return;
}
$this->renderBadge('WARN', 'No baseline published yet — recording locally.');
$this->renderChild('See https://pestphp.com/docs/tia for how to publish one from CI.');
}
private function isCi(): bool
{
return getenv('GITHUB_ACTIONS') === 'true'
|| getenv('GITLAB_CI') === 'true'
|| getenv('CIRCLECI') === 'true';
}
private function detectGitHubRepo(string $projectRoot): ?string
{
$gitConfig = $projectRoot.DIRECTORY_SEPARATOR.'.git'.DIRECTORY_SEPARATOR.'config';
if (! is_file($gitConfig)) {
return null;
}
$content = @file_get_contents($gitConfig);
if ($content === false) {
return null;
}
if (preg_match('/\[remote "origin"\][^\[]*?url\s*=\s*(\S+)/s', $content, $match) !== 1) {
return null;
}
$url = $match[1];
if (preg_match('#^git@github\.com:([\w.-]+/[\w.-]+?)(?:\.git)?$#', $url, $m) === 1) {
return $m[1];
}
if (preg_match('#^https?://github\.com/([\w.-]+/[\w.-]+?)(?:\.git)?/?$#', $url, $m) === 1) {
return $m[1];
}
if (preg_match('#^ssh://(?:[^@/]+@)?github\.com(?::\d+)?/([\w.-]+/[\w.-]+?)(?:\.git)?/?$#i', $url, $m) === 1) {
return $m[1];
}
return null;
}
/**
* @return array{payload: array{graph: string, coverage: ?string, sizeOnDisk: int}|null, failureKind: ?string}
*/
private function download(string $repo, string $projectRoot, bool $hasAnchor = false): array
{
$this->validateGhDependencies($hasAnchor);
[$runId, $listError] = $this->latestSuccessfulRunIdWithError($repo);
if ($listError !== null) {
$this->panicOnClassifiedError($listError, 'Failed to query baseline runs', $hasAnchor);
$this->renderBadge('WARN', sprintf(
'Failed to query baseline runs — %s',
$listError['message'],
));
return ['payload' => null, 'failureKind' => $listError['kind']];
}
if ($runId === null) {
return ['payload' => null, 'failureKind' => 'no-runs'];
}
$runCacheDir = $this->downloadCacheDir($projectRoot).DIRECTORY_SEPARATOR.$this->safeRunId($runId);
if (is_file($runCacheDir.DIRECTORY_SEPARATOR.self::GRAPH_ASSET)) {
@touch($runCacheDir);
$this->renderChild(sprintf(
'Using cached baseline from %s (run %s).',
$repo,
$runId,
));
return ['payload' => $this->readArtifact($runCacheDir), 'failureKind' => null];
}
if (! @mkdir($runCacheDir, 0755, true) && ! is_dir($runCacheDir)) {
return ['payload' => null, 'failureKind' => null];
}
$download = $this->downloadArtifact($repo, $runId, $runCacheDir, $hasAnchor);
if (! $download['success']) {
return ['payload' => null, 'failureKind' => $download['failureKind']];
}
$payload = $this->validateDownloadedArtifact($runCacheDir, $hasAnchor);
$this->trimDownloadCache($projectRoot);
return ['payload' => $payload, 'failureKind' => null];
}
/**
* @param array{kind: string, message: string} $diagnosis
*/
private function panicOnClassifiedError(array $diagnosis, string $contextPrefix, bool $hasAnchor): void
{
if (! in_array($diagnosis['kind'], ['forbidden', 'not-found'], true)) {
return;
}
Panic::with(new BaselineFetchFailed(
sprintf('%s — %s', $contextPrefix, $diagnosis['message']),
'Verify workflow tia-baseline.yml, artifact pest-tia-baseline, and gh token scope.',
$hasAnchor,
));
}
private function validateGhDependencies(bool $hasAnchor): void
{
if (! $this->commandExists('gh')) {
Panic::with(new BaselineFetchFailed(
'GitHub CLI (gh) not found — cannot fetch baseline.',
'Install it from https://cli.github.com.',
$hasAnchor,
));
}
if (! $this->ghAuthenticated()) {
Panic::with(new BaselineFetchFailed(
'GitHub CLI (gh) is not authenticated — cannot fetch baseline.',
'Run `gh auth login` and retry.',
$hasAnchor,
));
}
}
/**
* @return array{success: bool, failureKind: ?string}
*/
private function downloadArtifact(string $repo, string $runId, string $runCacheDir, bool $hasAnchor): array
{
$artifactSize = $this->artifactSize($repo, $runId);
$this->output->writeln('');
$this->renderChild($artifactSize !== null
? sprintf(
'Downloading TIA baseline (%s) from %s…',
$this->formatSize($artifactSize),
$repo,
)
: sprintf(
'Downloading TIA baseline from %s…',
$repo,
));
$process = new Process([
'gh', 'run', 'download', $runId,
'-R', $repo,
'-n', self::ARTIFACT_NAME,
'-D', $runCacheDir,
]);
$process->setTimeout(900.0);
$process->start();
$startedAt = microtime(true);
$tick = 0;
while ($process->isRunning()) {
$this->renderDownloadProgress($startedAt, $tick++);
usleep(120_000);
}
$process->wait();
$this->clearProgressLine();
if ($process->isSuccessful()) {
return ['success' => true, 'failureKind' => null];
}
$this->cleanup($runCacheDir);
$diagnosis = $this->classifyGhError($process->getErrorOutput().$process->getOutput());
$this->panicOnClassifiedError($diagnosis, 'Baseline download failed', $hasAnchor);
$this->renderBadge('WARN', sprintf(
'Baseline download failed — %s',
$diagnosis['message'],
));
return ['success' => false, 'failureKind' => $diagnosis['kind']];
}
/**
* @return array{graph: string, coverage: ?string, sizeOnDisk: int}
*/
private function validateDownloadedArtifact(string $runCacheDir, bool $hasAnchor): array
{
$payload = $this->readArtifact($runCacheDir);
if ($payload === null) {
$this->cleanup($runCacheDir);
Panic::with(new BaselineFetchFailed(
'Baseline downloaded but the artifact is missing expected files (graph.json).',
'Your CI publish step is broken — check the workflow that uploads pest-tia-baseline.',
$hasAnchor,
));
}
return $payload;
}
private function artifactSize(string $repo, string $runId): ?int
{
$process = new Process([
'gh', 'api',
sprintf('repos/%s/actions/runs/%s/artifacts', $repo, $runId),
'--jq', sprintf(
'.artifacts[] | select(.name == "%s") | .size_in_bytes', // @pest-ignore-type
self::ARTIFACT_NAME,
),
]);
$process->setTimeout(30.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
$size = trim($process->getOutput());
return is_numeric($size) ? (int) $size : null;
}
private function renderDownloadProgress(float $startedAt, int $tick): void
{
static $frames = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
$elapsed = max(0.0, microtime(true) - $startedAt);
$frame = $frames[$tick % count($frames)];
$this->output->write(sprintf(
"\r\033[K <fg=gray>%s %.1fs elapsed</>",
$frame,
$elapsed,
));
}
private function clearProgressLine(): void
{
$this->output->write("\r\033[K");
}
private function dirSize(string $dir): int
{
if (! is_dir($dir)) {
return 0;
}
$total = 0;
$iterator = new \RecursiveIteratorIterator(
new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS),
);
/** @var \SplFileInfo $entry */
foreach ($iterator as $entry) {
if ($entry->isFile()) {
$total += $entry->getSize();
}
}
return $total;
}
/**
* @return array{graph: string, coverage: ?string, sizeOnDisk: int}|null
*/
private function readArtifact(string $dir): ?array
{
$graphPath = $dir.DIRECTORY_SEPARATOR.self::GRAPH_ASSET;
$coveragePath = $dir.DIRECTORY_SEPARATOR.self::COVERAGE_ASSET;
$graph = is_file($graphPath) ? @file_get_contents($graphPath) : false;
if ($graph === false) {
return null;
}
$coverage = is_file($coveragePath) ? @file_get_contents($coveragePath) : false;
return [
'graph' => $graph,
'coverage' => $coverage === false ? null : $coverage,
'sizeOnDisk' => $this->dirSize($dir),
];
}
private function downloadCacheDir(string $projectRoot): string
{
return Storage::tempDir($projectRoot).DIRECTORY_SEPARATOR.self::DOWNLOAD_CACHE_DIR;
}
private function safeRunId(string $runId): string
{
$sanitised = preg_replace('/[^A-Za-z0-9_-]/', '', $runId) ?? '';
return $sanitised === '' ? 'unknown' : $sanitised;
}
private function trimDownloadCache(string $projectRoot): void
{
$root = $this->downloadCacheDir($projectRoot);
if (! is_dir($root)) {
return;
}
$entries = @scandir($root);
if ($entries === false) {
return;
}
$candidates = [];
foreach ($entries as $entry) {
if (in_array($entry, ['.', '..'], true)) {
continue;
}
$path = $root.DIRECTORY_SEPARATOR.$entry;
if (! is_dir($path)) {
continue;
}
$mtime = @filemtime($path);
$candidates[] = ['path' => $path, 'mtime' => $mtime === false ? 0 : $mtime];
}
if (count($candidates) <= self::DOWNLOAD_CACHE_MAX_ENTRIES) {
return;
}
usort(
$candidates,
static fn (array $a, array $b): int => $b['mtime'] <=> $a['mtime'],
);
foreach (array_slice($candidates, self::DOWNLOAD_CACHE_MAX_ENTRIES) as $stale) {
$this->cleanup($stale['path']);
}
}
/**
* @return array{0: ?string, 1: ?array{kind: string, message: string}}
*/
private function latestSuccessfulRunIdWithError(string $repo): array
{
$process = new Process([
'gh', 'run', 'list',
'-R', $repo,
'--workflow', self::WORKFLOW_FILE,
'--status', 'success',
'--limit', '1',
'--json', 'databaseId',
'--jq', '.[0].databaseId // empty',
]);
$process->setTimeout(30.0);
$process->run();
if (! $process->isSuccessful()) {
return [null, $this->classifyGhError($process->getErrorOutput().$process->getOutput())];
}
$runId = trim($process->getOutput());
return [$runId === '' ? null : $runId, null];
}
private function ghAuthenticated(): bool
{
$process = new Process(['gh', 'auth', 'status']);
$process->setTimeout(10.0);
$process->run();
return $process->isSuccessful();
}
/**
* @return array{kind: string, message: string}
*/
private function classifyGhError(string $output): array
{
$output = trim($output);
if ($output === '') {
return ['kind' => 'unknown', 'message' => 'unknown error'];
}
foreach (self::DIAGNOSES as $kind => $diagnosis) {
if (preg_match($diagnosis['pattern'], $output) === 1) {
return ['kind' => $kind, 'message' => $diagnosis['message']];
}
}
return ['kind' => 'unknown', 'message' => trim(strtok($output, "\n"))];
}
private function commandExists(string $cmd): bool
{
$process = new Process(['which', $cmd]);
$process->run();
return $process->isSuccessful();
}
private function cleanup(string $dir): void
{
if (! is_dir($dir)) {
return;
}
$iterator = new \RecursiveIteratorIterator(
new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS),
\RecursiveIteratorIterator::CHILD_FIRST,
);
/** @var \SplFileInfo $entry */
foreach ($iterator as $entry) {
if ($entry->isDir()) {
@rmdir($entry->getPathname());
} else {
@unlink($entry->getPathname());
}
}
@rmdir($dir);
}
private function formatSize(int $bytes): string
{
if ($bytes >= 1024 * 1024) {
return sprintf('%.1f MB', $bytes / 1024 / 1024);
}
if ($bytes >= 1024) {
return sprintf('%.1f KB', $bytes / 1024);
}
return $bytes.' B';
}
}
+28
View File
@@ -0,0 +1,28 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
use Pest\Contracts\Bootstrapper as BootstrapperContract;
use Pest\Plugins\Tia\Contracts\State;
use Pest\Support\Container;
use Pest\TestSuite;
/**
* @internal
*/
final readonly class Bootstrapper implements BootstrapperContract
{
public function __construct(private Container $container) {}
public function boot(): void
{
$testSuite = $this->container->get(TestSuite::class);
assert($testSuite instanceof TestSuite);
$tempDir = Storage::tempDir($testSuite->rootPath);
$this->container->add(State::class, new FileState($tempDir));
}
}
+423
View File
@@ -0,0 +1,423 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
use Pest\Exceptions\MissingDependency;
use Symfony\Component\Process\Process;
/**
* @internal
*/
final readonly class ChangedFiles
{
public function __construct(private string $projectRoot) {}
/**
* @param array<int, string> $files project-relative paths.
* @param array<string, string> $lastRunTree path → content hash from last run.
* @return array<int, string>
*/
public function filterUnchangedSinceLastRun(array $files, array $lastRunTree): array
{
if ($lastRunTree === []) {
return $files;
}
$candidates = array_fill_keys($files, true);
foreach (array_keys($lastRunTree) as $snapshotted) {
$candidates[$snapshotted] = true;
}
$remaining = [];
foreach (array_keys($candidates) as $file) {
$snapshot = $lastRunTree[$file] ?? null;
$current = $this->currentHash($file);
if ($snapshot === null || $current === null || $current !== $snapshot) {
$remaining[] = $file;
}
}
return $remaining;
}
private function currentHash(string $relativePath): ?string
{
$absolute = $this->projectRoot.DIRECTORY_SEPARATOR.$relativePath;
if (! is_file($absolute)) {
return null;
}
$hash = ContentHash::of($absolute);
return $hash === false ? null : $hash;
}
/**
* @param array<int, string> $files
* @return array<string, string> path → xxh128 content hash
*/
public function snapshotTree(array $files): array
{
$out = [];
foreach ($files as $file) {
$absolute = $this->projectRoot.DIRECTORY_SEPARATOR.$file;
if (! is_file($absolute)) {
$out[$file] = '';
continue;
}
$hash = ContentHash::of($absolute);
if ($hash !== false) {
$out[$file] = $hash;
}
}
return $out;
}
/**
* @return array<int, string>|null `null` when git is unavailable, or when
*/
public function since(?string $sha): ?array
{
$files = [];
if ($sha !== null && $sha !== '') {
if (! $this->shaIsReachable($sha)) {
return null;
}
$files = array_merge($files, $this->diffSinceSha($sha));
}
$files = array_merge($files, $this->workingTreeChanges());
$unique = [];
foreach ($files as $file) {
if ($file === '') {
continue;
}
$unique[$file] = true;
}
$candidates = array_keys($this->filterIgnored($unique));
if ($sha !== null && $sha !== '') {
return $this->filterBehaviourallyUnchanged($candidates, $sha);
}
return $candidates;
}
/**
* @param array<int, string> $files
* @return array<int, string>
*/
private function filterBehaviourallyUnchanged(array $files, string $sha): array
{
$remaining = [];
foreach ($files as $file) {
$currentHash = $this->currentHash($file);
if ($currentHash === null) {
$remaining[] = $file;
continue;
}
$baselineContent = $this->contentAtSha($sha, $file);
if ($baselineContent === null) {
$remaining[] = $file;
continue;
}
if ($currentHash !== ContentHash::ofContent($file, $baselineContent)) {
$remaining[] = $file;
}
}
return $remaining;
}
private function contentAtSha(string $sha, string $path): ?string
{
$process = new Process(['git', 'show', $sha.':'.$path], $this->projectRoot);
$process->setTimeout(5.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
return $process->getOutput();
}
/**
* @param array<string, true> $candidates
* @return array<string, true>
*/
private function filterIgnored(array $candidates): array
{
if ($candidates === []) {
return $candidates;
}
$process = new Process(
['git', 'check-ignore', '--no-index', '-z', '--stdin'],
$this->projectRoot,
);
$process->setTimeout(5.0);
$process->setInput(implode("\x00", array_keys($candidates)));
$process->run();
$exitCode = $process->getExitCode();
if ($exitCode !== 0 && $exitCode !== 1) {
throw new MissingDependency('Tia mode', 'git');
}
$output = $process->getOutput();
if ($output === '') {
return $candidates;
}
foreach (explode("\x00", rtrim($output, "\x00")) as $ignored) {
if ($ignored !== '') {
unset($candidates[$ignored]);
}
}
return $candidates;
}
public function currentBranch(): ?string
{
$process = new Process(['git', 'rev-parse', '--abbrev-ref', 'HEAD'], $this->projectRoot);
$process->run();
if (! $process->isSuccessful()) {
throw new MissingDependency('Tia mode', 'git');
}
$branch = trim($process->getOutput());
return $branch === '' || $branch === 'HEAD' ? null : $branch;
}
public function defaultBranch(): ?string
{
$head = $this->gitOutput(['git', 'symbolic-ref', '--short', 'refs/remotes/origin/HEAD']);
if ($head !== null) {
$branch = preg_replace('#^origin/#', '', $head);
if (is_string($branch) && $branch !== '') {
return $branch;
}
}
$configured = $this->gitOutput(['git', 'config', '--get', 'init.defaultBranch']);
if ($configured === null) {
return null;
}
$exists = $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/heads/'.$configured]) !== null
|| $this->gitOutput(['git', 'rev-parse', '--verify', '--quiet', 'refs/remotes/origin/'.$configured]) !== null;
return $exists ? $configured : null;
}
/**
* Every branch name this checkout knows, local and remote alike. Remotes
* count: a branch that only lives on the origin is still a branch someone
* will check out, and its baseline must survive.
*
* @return list<string>|null `null` when git cannot answer.
*/
public function branchNames(): ?array
{
$process = new Process(
['git', 'for-each-ref', '--format=%(refname)', 'refs/heads', 'refs/remotes'],
$this->projectRoot,
);
$process->setTimeout(5.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
$names = [];
foreach ($this->splitLines($process->getOutput()) as $ref) {
if (str_starts_with($ref, 'refs/heads/')) {
$names[substr($ref, strlen('refs/heads/'))] = true;
continue;
}
if (! str_starts_with($ref, 'refs/remotes/')) {
continue;
}
$tail = substr($ref, strlen('refs/remotes/'));
$slash = strpos($tail, '/');
if ($slash === false) {
continue;
}
$branch = substr($tail, $slash + 1);
if ($branch !== '' && $branch !== 'HEAD') {
$names[$branch] = true;
}
}
return array_keys($names);
}
public function hasRemote(): bool
{
return $this->gitOutput(['git', 'remote']) !== null;
}
/**
* @param array<int, string> $command
*/
private function gitOutput(array $command): ?string
{
$process = new Process($command, $this->projectRoot);
$process->setTimeout(5.0);
$process->run();
if (! $process->isSuccessful()) {
return null;
}
$output = trim($process->getOutput());
return $output === '' ? null : $output;
}
private function shaIsReachable(string $sha): bool
{
$process = new Process(
['git', 'merge-base', '--is-ancestor', $sha, 'HEAD'],
$this->projectRoot,
);
$process->run();
return $process->getExitCode() === 0;
}
/**
* @return array<int, string>
*/
private function diffSinceSha(string $sha): array
{
$process = new Process(
['git', 'diff', '--name-only', $sha.'..HEAD'],
$this->projectRoot,
);
$process->run();
if (! $process->isSuccessful()) {
throw new MissingDependency('Tia mode', 'git');
}
return $this->splitLines($process->getOutput());
}
/**
* @return array<int, string>
*/
private function workingTreeChanges(): array
{
$process = new Process(
['git', 'status', '--porcelain', '-z', '--untracked-files=all'],
$this->projectRoot,
);
$process->run();
if (! $process->isSuccessful()) {
throw new MissingDependency('Tia mode', 'git');
}
$output = $process->getOutput();
if ($output === '') {
return [];
}
$records = explode("\x00", rtrim($output, "\x00"));
$files = [];
$count = count($records);
for ($i = 0; $i < $count; $i++) {
$record = $records[$i];
if (strlen($record) < 4) {
continue;
}
$status = substr($record, 0, 2);
$path = substr($record, 3);
if ($status[0] === 'R' || $status[0] === 'C') {
$files[] = $path;
if (isset($records[$i + 1]) && $records[$i + 1] !== '') {
$files[] = $records[$i + 1];
$i++;
}
continue;
}
$files[] = $path;
}
return $files;
}
public function currentSha(): ?string
{
$process = new Process(['git', 'rev-parse', 'HEAD'], $this->projectRoot);
$process->run();
if (! $process->isSuccessful()) {
throw new MissingDependency('Tia mode', 'git');
}
$sha = trim($process->getOutput());
return $sha === '' ? null : $sha;
}
/**
* @return array<int, string>
*/
private function splitLines(string $output): array
{
$lines = preg_split('/\R+/', trim($output), flags: PREG_SPLIT_NO_EMPTY);
return $lines === false ? [] : $lines;
}
}
+59
View File
@@ -0,0 +1,59 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
/**
* @internal
*/
final class CiDefaultBranch
{
public static function detect(): ?string
{
return self::fromGitLab() ?? self::fromGitHubEvent();
}
private static function fromGitLab(): ?string
{
return self::environment('CI_DEFAULT_BRANCH');
}
private static function fromGitHubEvent(): ?string
{
$path = self::environment('GITHUB_EVENT_PATH');
if ($path === null || ! is_file($path) || ! is_readable($path)) {
return null;
}
$contents = @file_get_contents($path);
if ($contents === false) {
return null;
}
$payload = json_decode($contents, true);
if (! is_array($payload) || ! is_array($payload['repository'] ?? null)) {
return null;
}
$branch = $payload['repository']['default_branch'] ?? null;
return is_string($branch) && $branch !== '' ? $branch : null;
}
private static function environment(string $name): ?string
{
$value = getenv($name);
if (! is_string($value)) {
return null;
}
$value = trim($value);
return $value === '' ? null : $value;
}
}
+28
View File
@@ -0,0 +1,28 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
use Pest\Plugins\Tia\Edges\BladeEdges;
use Pest\Plugins\Tia\Edges\InertiaEdges;
/**
* @internal
*/
final class Collectors
{
/** @var list<class-string> */
private const array COLLECTORS = [
BladeEdges::class,
TableTracker::class,
InertiaEdges::class,
];
public static function armAll(Recorder $recorder): void
{
foreach (self::COLLECTORS as $collector) {
$collector::arm($recorder);
}
}
}
+87
View File
@@ -0,0 +1,87 @@
<?php
declare(strict_types=1);
namespace Pest\Plugins\Tia;
use Pest\Support\Container;
/**
* @internal
*/
final class Configuration
{
/**
* @return $this
*/
public function always(): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->markEnabled();
return $this;
}
/**
* @return $this
*/
public function locally(): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->markEnabled();
$watchPatterns->markLocally();
return $this;
}
/**
* @return $this
*/
public function filtered(): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->markFiltered();
return $this;
}
/**
* @return $this
*/
public function baselined(): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->markBaselined();
return $this;
}
/**
* @return $this
*/
public function defaultBranch(string $branch): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->setDefaultBranch($branch);
return $this;
}
/**
* @param array<string, string> $patterns glob → project-relative test dir
* @return $this
*/
public function watch(array $patterns): self
{
/** @var WatchPatterns $watchPatterns */
$watchPatterns = Container::getInstance()->get(WatchPatterns::class);
$watchPatterns->add($patterns);
return $this;
}
}

Some files were not shown because too many files have changed in this diff Show More