Compare commits

..

21 Commits

Author SHA1 Message Date
nuno maduro db70017cb2 wip 2026-08-05 03:30:50 +01:00
nuno maduro 86adaedbbd fix: tia filtered 2026-08-04 23:30:03 +01:00
nuno maduro 92c7677c6e fix: don't allow invalid class names 2026-08-04 20:46:21 +01:00
nuno maduro 872f0a50c2 fix(tia): replayable scenarios running afterEach 2026-08-04 20:19:12 +01:00
nuno maduro 668809bc75 chore: adds missing test 2026-08-04 19:28:58 +01:00
nuno maduro 8d8f45c843 chore: bump deps 2026-08-04 19:28:52 +01:00
nuno maduro 086b3e9107 chore: ai instructions 2026-08-04 19:01:16 +01:00
nuno maduro 43fe26f324 chore: bumps deps 2026-08-04 19:01:08 +01:00
nuno maduro 865c1e5113 ci: snapshots 2026-08-04 18:51:04 +01:00
stephanbouman 19eed8d581 Fixed typos (#1819)
* Fixed typo in message

* Fixed typo in message
2026-08-03 11:29:03 -04:00
Danny de Wit ad1850b110 fix: keep TableExtractor table names as strings (#1794)
PHP coerces numeric-string array keys to ints, so collecting table names in
$tables[$name] and returning array_keys() can yield ints. That breaks the
declared list<string> and throws a TypeError in Recorder::linkTable(string).

Standard SQL like substring(x FROM 1 FOR 10) is enough to trigger it, since
the numeric operand matches the FROM pattern.

Closes #1793

Co-authored-by: Danny de Wit <d.dewit@fenceweb.com>
2026-08-02 22:50:47 -04:00
nuno maduro 585de259a2 chore: style 2026-08-02 22:26:34 -04:00
nuno maduro 953664dce0 release: v5.0.3 2026-08-02 22:24:32 -04:00
nuno maduro 1dd959848c chore: updates snapshots 2026-08-02 22:24:24 -04:00
Jeffrey van Hees 047753c836 fix: key Tia results per dataset row (#1799)
The result cache was keyed `Class::method` on both sides, so every row of a
`->with()` test shared one entry and the last writer won. On replay that single
status was handed to every row.

`TestMethod::id()` already appends `#dataSetName`, and `TestCase` reaches the
same value object through `valueObjectForEvents()`, so both sides can use it.
2026-08-02 21:43:15 -04:00
nuno maduro b63626a94d docs: adjusts readme
Signed-off-by: nuno maduro <enunomaduro@gmail.com>
2026-07-30 11:53:41 -04:00
nuno maduro ae080becd6 chore: style 2026-07-29 15:27:11 -04:00
nuno maduro aa86304794 release: v5.0.2 2026-07-29 14:59:13 -04:00
nuno maduro 75963ea973 chore: bumps deps 2026-07-29 14:59:03 -04:00
ohnotnow 5d0ceb1a5d Check for pcov being enabled for tia (#1779)
* Check for pcov being enabled for tia

* Delete tests/Features/TiaDriverGate.php

Signed-off-by: nuno maduro <enunomaduro@gmail.com>

---------

Signed-off-by: nuno maduro <enunomaduro@gmail.com>
Co-authored-by: Ohffs <ohffsnotnow@gmail.com>
Co-authored-by: nuno maduro <enunomaduro@gmail.com>
2026-07-29 14:28:38 -04:00
talaridisTh 6f0c0536a4 feat: document the tia options in pest --help (#1780) 2026-07-29 14:11:17 -04:00
65 changed files with 811 additions and 101 deletions
+18
View File
@@ -0,0 +1,18 @@
# CLAUDE.md
**Do not edit this file.** Agents must never add, remove, or reword anything here. If a change seems needed, say so and let a human do it.
## Ask before testing
When asked to review code or build a feature, do not run the test suite and do not write new tests. Make the change, report it, then ask the user whether tests should be added — describing the tests you have in mind — and wait for the user to confirm.
Two reasons this matters here: the suite takes minutes, and `tests/.snapshots/success.txt` plus the tally in `tests/Visual/Parallel.php` encode the whole suite's result, so a single added test breaks both.
Once the user confirms:
```bash
composer test:unit # fast, excludes the visual group
composer test:integration # visual and snapshot tests
composer test # everything CI runs, in CI's order
composer update:snapshots # only when a test was added or removed
```
+2 -1
View File
@@ -35,6 +35,7 @@ We cannot thank our sponsors enough for their incredible support in funding Pest
- **[SerpApi](https://serpapi.com/?ref=nunomaduro)**
- **[Typesense](https://typesense.org/?ref=nunomaduro)**
- **[Bento](https://bentonow.com/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/)**
- **[Pixel](https://wearepixel.com.au/?ref=nunomaduro)**
- **[Redberry](https://redberry.international/laravel-development/?ref=nunomaduro)**
Pest is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.
+6 -7
View File
@@ -18,19 +18,19 @@
],
"require": {
"php": "^8.4",
"brianium/paratest": "^7.23.0",
"brianium/paratest": "^7.23.1",
"nunomaduro/collision": "^8.9.5",
"nunomaduro/termwind": "^2.4.0",
"pestphp/pest-plugin": "^5.0.0",
"pestphp/pest-plugin-arch": "^5.0.0",
"pestphp/pest-plugin-mutate": "^5.0.0",
"pestphp/pest-plugin-mutate": "^5.0.1",
"pestphp/pest-plugin-profanity": "^5.0.0",
"phpunit/phpunit": "^13.2.4",
"phpunit/phpunit": "^13.2.6",
"symfony/process": "^8.1.0"
},
"conflict": {
"filp/whoops": "<2.18.3",
"phpunit/phpunit": ">13.2.4",
"phpunit/phpunit": ">13.2.6",
"sebastian/exporter": "<7.0.0",
"webmozart/assert": "<1.11.0"
},
@@ -58,12 +58,11 @@
]
},
"require-dev": {
"laravel/pao": "^1.1.2",
"pestphp/pest-dev-tools": "^5.0.0",
"pestphp/pest-plugin-browser": "^5.0.0",
"pestphp/pest-plugin-phpstan": "^5.0.0",
"pestphp/pest-plugin-rector": "^5.0.0",
"pestphp/pest-plugin-type-coverage": "^5.0.0",
"pestphp/pest-plugin-rector": "^5.0.2",
"pestphp/pest-plugin-type-coverage": "^5.0.2",
"psy/psysh": "^0.12.24"
},
"minimum-stability": "dev",
+2 -2
View File
@@ -90,11 +90,11 @@ final class TestSuiteLoader
(static function () use ($suiteClassFile) {
try {
include_once $suiteClassFile;
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
} catch (Throwable $e) {
Panic::with($e);
}
TestSuite::getInstance()->tests->makeIfNeeded($suiteClassFile);
})();
$loadedClasses = array_values(
+1 -1
View File
@@ -18,8 +18,8 @@
<directory suffix=".php">./tests</directory>
<directory suffix=".php">./tests-external</directory>
<exclude>./tests/.snapshots</exclude>
<exclude>./tests/.tests</exclude>
<exclude>./tests/Fixtures/Inheritance</exclude>
<exclude>./tests/Fixtures/Suites</exclude>
</testsuite>
</testsuites>
<source>
+5
View File
@@ -0,0 +1,5 @@
{
"exclude": [
"tests/Fixtures/Suites"
]
}
+3
View File
@@ -28,6 +28,9 @@ return RectorConfig::configure()
->withSkip([
__DIR__.'/src/Plugins/Parallel/Paratest/WrapperRunner.php',
__DIR__.'/tests/Fixtures/Arch',
// Fixture suites are pinned by the TeamCity / JUnit snapshots, down to
// the line numbers — rewriting their source would break them.
__DIR__.'/tests/Fixtures/Suites',
ReturnNeverTypeRector::class,
ArrowFunctionDelegatingCallToFirstClassCallableRector::class,
NarrowObjectReturnTypeRector::class,
+7 -2
View File
@@ -280,12 +280,17 @@ trait Testable
/** @var Tia $tia */
$tia = Container::getInstance()->get(Tia::class);
$status = $tia->getStatus(self::$__filename, $this::class.'::'.$this->name());
$status = $tia->getStatus(self::$__filename, $this->valueObjectForEvents()->id());
$replay = ReplayType::fromStatus($status);
if ($replay !== ReplayType::None) {
assert($status !== null);
// Marks the replay before the branches below throw, so `tearDown`
// short-circuits for every replayed result — the throwing branches
// never reach `parent::setUp`, so no user hook may run after them.
$this->__replay = $replay;
match ($replay) {
ReplayType::Pass, ReplayType::Risky => $this->__beginReplay($replay, $tia),
ReplayType::Skipped => $this->markTestSkipped($status->message()),
@@ -319,7 +324,7 @@ trait Testable
private function __beginReplay(ReplayType $replay, Tia $tia): void
{
$this->__replay = $replay;
$this->__replayAssertions = $tia->getAssertionCount($this::class.'::'.$this->name());
$this->__replayAssertions = $tia->getAssertionCount($this->valueObjectForEvents()->id());
$this->__ran = true;
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllAlreadyExist extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterAllWithinDescribe extends InvalidArgumentException implements E
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The afterAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class AfterBeforeTestFunction extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct('After method cannot be used with before the [test|it] functions in the filename `['.$filename.']`.');
parent::__construct('After method cannot be used with before the [test|it] functions in the filename ['.$filename.'].');
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllAlreadyExist extends InvalidArgumentException implements Ex
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll already exists in the filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll already exists in the filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class BeforeAllWithinDescribe extends InvalidArgumentException implements
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename `%s`.', $filename));
parent::__construct(sprintf('The beforeAll method can not be used within describe functions. Filename [%s].', $filename));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetAlreadyExists extends InvalidArgumentException implements Exc
*/
public function __construct(string $name, string $scope)
{
parent::__construct(sprintf('A dataset with the name `%s` already exists in scope [%s].', $name, $scope));
parent::__construct(sprintf('A dataset with the name [%s] already exists in scope [%s].', $name, $scope));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class DatasetDoesNotExist extends InvalidArgumentException implements Exce
*/
public function __construct(string $name)
{
parent::__construct(sprintf("A dataset with the name `%s` does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
parent::__construct(sprintf("A dataset with the name [%s] does not exist. You can create it using `dataset('%s', ['a', 'b']);`.", $name, $name));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class FileOrFolderNotFound extends InvalidArgumentException implements Exc
*/
public function __construct(string $filename)
{
parent::__construct(sprintf('The file or folder with the name `%s` could not be found.', $filename));
parent::__construct(sprintf('The file or folder with the name [%s] could not be found.', $filename));
}
}
+41
View File
@@ -0,0 +1,41 @@
<?php
declare(strict_types=1);
namespace Pest\Exceptions;
use InvalidArgumentException;
use NunoMaduro\Collision\Contracts\RenderlessEditor;
use NunoMaduro\Collision\Contracts\RenderlessTrace;
use Symfony\Component\Console\Exception\ExceptionInterface;
/**
* @internal
*/
final class InvalidTestClassName extends InvalidArgumentException implements ExceptionInterface, RenderlessEditor, RenderlessTrace
{
/**
* Creates a new Exception instance for the given class name.
*/
public static function fromClassName(string $filename, string $className): self
{
return new self(sprintf(
'The test file [%s] would create the class [%s], which is not a valid PHP class name. Please rename the test file.',
$filename,
$className,
));
}
/**
* Creates a new Exception instance for the given namespace.
*/
public static function fromNamespace(string $filename, string $namespace, string $part): self
{
return new self(sprintf(
'The test file [%s] would create the namespace [%s], which is not a valid PHP namespace, as [%s] may not be used as a namespace name. Please rename the folder in question.',
$filename,
$namespace,
$part,
));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestAlreadyExist extends InvalidArgumentException implements Excepti
*/
public function __construct(string $fileName, string $description)
{
parent::__construct(sprintf('A test with the description `%s` already exists in the filename `%s`.', $description, $fileName));
parent::__construct(sprintf('A test with the description [%s] already exists in the filename [%s].', $description, $fileName));
}
}
@@ -19,6 +19,6 @@ final class TestCaseClassOrTraitNotFound extends InvalidArgumentException implem
*/
public function __construct(string $testCaseClass)
{
parent::__construct(sprintf('The class `%s` was not found.', $testCaseClass));
parent::__construct(sprintf('The class [%s] was not found.', $testCaseClass));
}
}
+1 -1
View File
@@ -19,6 +19,6 @@ final class TestDescriptionMissing extends InvalidArgumentException implements E
*/
public function __construct(string $fileName)
{
parent::__construct(sprintf('Test description is missing in the filename `%s`.', $fileName));
parent::__construct(sprintf('Test description is missing in the filename [%s].', $fileName));
}
}
+11
View File
@@ -9,6 +9,7 @@ use Pest\Concerns;
use Pest\Contracts\HasPrintableTestCaseName;
use Pest\Evaluators\Attributes;
use Pest\Exceptions\DatasetMissing;
use Pest\Exceptions\InvalidTestClassName;
use Pest\Exceptions\ShouldNotHappen;
use Pest\Exceptions\TestAlreadyExist;
use Pest\Exceptions\TestClosureMustNotBeStatic;
@@ -138,6 +139,16 @@ final class TestCaseFactory
if (trim($className) === '') {
$className = 'InvalidTestName'.Str::random();
} elseif (! Str::isValidClassName($className)) {
throw InvalidTestClassName::fromClassName($this->filename, $className);
}
if ($this->namespace === null) {
foreach ($partsFQN as $partFQN) {
if (! Str::isValidIdentifier($partFQN)) {
throw InvalidTestClassName::fromNamespace($this->filename, $namespace, $partFQN);
}
}
}
$this->attributes = [
+1 -1
View File
@@ -6,7 +6,7 @@ namespace Pest;
function version(): string
{
return '5.0.0-beta.1';
return '5.0.3';
}
function testDirectory(string $file = ''): string
+35
View File
@@ -114,6 +114,41 @@ final readonly class Help implements HandlesArguments
],
];
$content['Tia'] = [
[
'arg' => '--tia',
'desc' => 'Re-run only the tests affected by your changes, replaying the rest from cache',
],
[
'arg' => '--no-tia',
'desc' => 'Disable test impact analysis for this run',
],
[
'arg' => '--tia --fresh',
'desc' => 'Discard the recorded dependency graph and record it again',
],
[
'arg' => '--tia --filtered',
'desc' => 'Narrow the run to the affected test files only',
],
[
'arg' => '--tia --locally',
'desc' => 'Enable test impact analysis on local machines only',
],
[
'arg' => '--tia --baselined',
'desc' => 'Fetch the shared dependency graph recorded by the CI baseline',
],
[
'arg' => '--tia --refetch',
'desc' => 'Force a fresh fetch of the shared dependency graph',
],
[
'arg' => '--baseline',
'desc' => 'Output to standard output the test impact analysis storage directory',
],
];
$content['Execution'] = [...[
[
'arg' => '--parallel',
+253 -7
View File
@@ -7,7 +7,9 @@ namespace Pest\Plugins;
use NunoMaduro\Collision\Adapters\Phpunit\Printers\DefaultPrinter;
use Pest\Contracts\Plugins\AddsOutput;
use Pest\Contracts\Plugins\HandlesArguments;
use Pest\Contracts\Plugins\HandlesOriginalArguments;
use Pest\Contracts\Plugins\Terminable;
use Pest\Exceptions\MissingDependency;
use Pest\Exceptions\NoAffectedTestsFound;
use Pest\Exceptions\TiaRequiresRepositoryRoot;
use Pest\Panic;
@@ -30,13 +32,14 @@ use Pest\Support\View;
use Pest\TestCaseFilters\TiaTestCaseFilter;
use Pest\TestSuite;
use PHPUnit\Framework\TestStatus\TestStatus;
use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Process\Process;
/**
* @internal
*/
final class Tia implements AddsOutput, HandlesArguments, Terminable
final class Tia implements AddsOutput, HandlesArguments, HandlesOriginalArguments, Terminable
{
use HandleArguments;
@@ -56,6 +59,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private const string BASELINE_PATH_OPTION = '--baseline';
/**
* Set by the mutation plugin on the subprocess running a single mutant,
* and nowhere else. Its own `--mutate` flag is popped before the argv is
* handed to that subprocess, so the flag cannot be matched instead.
*/
private const string ENV_MUTATION_TESTING = 'PEST_MUTATION_TESTING';
private const string ENV_TIA = 'PEST_TIA';
private const string ENV_FILTERED = 'PEST_TIA_FILTERED';
@@ -107,6 +117,34 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
'--compact', '--ci-build-id', '--min',
];
/**
* Flags that narrow this run to a subset of the suite.
*
* Only user-supplied, per-run narrowing belongs here. A filter that is
* always in force — `<groups>` in phpunit.xml, or a plugin registering a
* test case filter from `boot()` — applies equally to the runs that build
* the baseline, so it does not make this run narrower than the baseline
* and must not disable baseline writes.
*
* `--shard` is rewritten to `--filter` before this plugin sees the
* arguments, so it is covered here too. The `bin/pest`-only flags are
* stripped from the handled arguments, so they are matched against the
* original argv instead.
*
* Flags that cut a run short instead of narrowing it — `--bail`, `--retry`,
* `--stop-on-*` — do not belong here either. They only narrow the run when
* something actually fails, and that is not known until it is over, so they
* are handled by stoppedEarly() from addOutput().
*
* @var list<string>
*/
private const array PARTIAL_SELECTION_FLAGS = [
'--filter', '--exclude-filter', '--group', '--exclude-group',
'--covers', '--uses', '--testsuite', '--exclude-testsuite', '--test-suffix',
'--dirty', '--todo', '--todos', '--flaky', '--notes',
'--assignee', '--issue', '--ticket', '--pr', '--pull-request',
];
private bool $graphWritten = false;
private bool $replayRan = false;
@@ -122,8 +160,19 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private ?Graph $replayGraph = null;
/**
* The baseline this run reads from and writes to.
*
* `main` is only the fallback for a repository whose branch cannot be read
* — it is also the branch every other baseline falls back to reading, so
* writing there by accident corrupts the shared baseline. Resolved through
* resolveBranch() rather than at every use site, because the git call it
* needs is not free.
*/
private string $branch = 'main';
private bool $branchResolved = false;
/** @var array<string, true> */
private array $affectedFiles = [];
@@ -142,6 +191,27 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
private bool $filteredMode = false;
/**
* Bars this run from touching the graph at all, results included.
*
* Reserved for runs whose results describe something other than the code
* in the working tree, which is nothing the baseline can ever use.
*/
private bool $writesSuppressed = false;
/**
* Narrows this run's writes to the results of the tests it actually ran.
*
* A run that covered only part of the suite still learns something true
* about the tests it did reach. What it cannot do is speak for the rest:
* pruning results, advancing the recorded sha and replacing the edge map
* all claim the whole suite reported, so they stay behind a complete run.
*/
private bool $resultsOnlyWrites = false;
/** @var array<int, string> */
private array $originalArguments = [];
private ?string $driftLabel = null;
private ?string $driftDetails = null;
@@ -317,6 +387,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $this->cachedAssertionsByTestId[$testId] ?? 0;
}
/**
* {@inheritDoc}
*/
public function handleOriginalArguments(array $arguments): void
{
$this->originalArguments = $arguments;
}
/**
* {@inheritDoc}
*/
@@ -339,9 +417,21 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$cliEnabled = $this->hasArgument(self::OPTION, $arguments) || self::envFlagEnabled(self::ENV_TIA);
$alwaysEnabled = $watchPatterns->isEnabled()
&& (! $watchPatterns->isLocally() || Environment::name() === Environment::LOCAL);
$hasExplicitPath = $this->hasExplicitPathArgument($arguments);
$partial = ! $isWorker && ($hasExplicitPath || $this->hasPartialSelection($arguments));
$disabled = $disabled || $partial;
// A mutation subprocess runs the suite against source the mutation
// plugin has deliberately broken. Its failures describe the mutant, not
// the working tree, so unlike every other narrowed run there is nothing
// in its results worth keeping. The parent `--mutate` run is untouched
// by this: it runs the whole suite against real source.
if (getenv(self::ENV_MUTATION_TESTING) !== false) {
$this->writesSuppressed = true;
}
$enabled = ! $disabled && ($cliEnabled || $alwaysEnabled);
$this->filteredMode = ($this->hasArgument(self::FILTERED_OPTION, $arguments) || self::envFlagEnabled(self::ENV_FILTERED) || $watchPatterns->isFiltered())
&& ! $this->hasExplicitPathArgument($arguments)
&& ! $hasExplicitPath
&& ! $this->coverageReportActive();
$freshRequested = $this->hasArgument(self::FRESH_OPTION, $arguments);
$this->forceRefetch = $this->hasArgument(self::REFETCH_OPTION, $arguments);
@@ -355,6 +445,25 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$arguments = $this->popArgument(self::BASELINED_OPTION, $arguments);
if ($disabled) {
if ($partial) {
// TIA cannot choose what runs here — the user already did — but
// the tests they picked still report honestly, so their results
// are kept and everything that would speak for the excluded ones
// is not. `--no-tia` needs none of this: it still runs the whole
// suite, so it remains a complete run.
$this->resultsOnlyWrites = true;
// `$this->filteredMode` counts as asking for it: reaching here
// means the narrowing came from the command line while filtered
// mode came from the environment or the config, and a run that
// silently declines what the config asked for is the one most
// in need of the explanation.
if ($cliEnabled || $freshRequested || $this->forceRefetch || $this->filteredMode) {
$this->output->writeln('');
$this->renderChild('TIA does not apply to partial runs — running the selected tests directly.');
}
}
$this->forceRefetch = false;
$this->filteredMode = false;
$this->freshRebuild = false;
@@ -392,6 +501,18 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$this->flushWorkerReplay();
}
// Both only ever set for the parent — addOutput() returns early in
// workers, whose partials are ephemeral and only reach the baseline if
// the parent consumes them. Everything this method goes on to write is
// whole-suite by nature — the edge map above all — so a narrowed run
// stops here too, its results already persisted by addOutput().
if ($this->writesSuppressed || $this->resultsOnlyWrites) {
$this->recorder->reset();
$this->coverageCollector->reset();
return;
}
$recorder = $this->recorder;
if (! $this->recordingActive && ! $recorder->isActive()) {
@@ -481,12 +602,34 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $exitCode;
}
// `->only()` narrows the executed set exactly like `--filter` does, but
// is only knowable once the suite has been collected — too late to turn
// TIA off up front. Sampled in addOutput() because Only's lock file is
// already gone by the time terminate() runs (its plugin terminates
// first). Whether the run was cut short is likewise only knowable now.
if (Only::isEnabled() || $this->stoppedEarly()) {
$this->resultsOnlyWrites = true;
}
$this->reportMissingWorkerDrivers();
// Runs before the checks below: it is what fills the parent's result
// collector in parallel, and a worker that stopped early narrows the
// whole run.
if (Parallel::isEnabled()) {
$this->mergeWorkerReplayPartials();
}
if ($this->writesSuppressed) {
return $exitCode;
}
if ($this->resultsOnlyWrites) {
$this->snapshotTestResults(complete: false);
return $exitCode;
}
if ($this->replayRan) {
$this->bumpRecordedSha();
}
@@ -635,7 +778,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
Panic::with(new TiaRequiresRepositoryRoot($subdirectoryPrefix));
}
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
$this->resolveBranch($projectRoot);
$fingerprint = Fingerprint::compute($projectRoot);
$this->startFingerprint = $fingerprint;
@@ -698,7 +841,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
*/
private function handleWorker(array $arguments, string $projectRoot, bool $recordingGlobal, bool $replayingGlobal): array
{
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? 'main';
$this->resolveBranch($projectRoot);
if ($replayingGlobal) {
$this->installWorkerReplay($projectRoot);
@@ -1065,7 +1208,7 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
{
$this->output->writeln('');
$this->renderChild('Running in TIA mode, however TIA as skipped as it needs Needs ext-pcov or Xdebug.');
$this->renderChild('Running in TIA mode, however TIA is skipped as it needs ext-pcov or Xdebug.');
}
/**
@@ -1141,6 +1284,9 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
'replayed' => $this->replayedCount,
'affected' => $this->affectedCount,
'executed' => $this->executedCount,
// Only the worker knows it stopped early — the parent runs no tests
// of its own, so its own check would always come back clean.
'truncated' => $this->stoppedEarly(),
], JSON_UNESCAPED_SLASHES);
if ($json === false) {
@@ -1177,6 +1323,13 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
continue;
}
// One worker stopping early leaves the whole suite incomplete: the
// tests it never reached are missing from the merged result set just
// as if they had been filtered out.
if (($decoded['truncated'] ?? false) === true) {
$this->resultsOnlyWrites = true;
}
if (isset($decoded['replayed']) && is_int($decoded['replayed'])) {
$this->replayedCount += $decoded['replayed'];
}
@@ -1436,7 +1589,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$collector->reset();
}
private function snapshotTestResults(bool $markKnownTestFiles = false): void
/**
* Folds the run's results into the existing graph.
*
* An incomplete run passes `$complete: false`, which keeps the additive
* half — the results of the tests it did run — and drops the half that
* speaks for the suite as a whole.
*/
private function snapshotTestResults(bool $markKnownTestFiles = false, bool $complete = true): void
{
/** @var ResultCollector $collector */
$collector = Container::getInstance()->get(ResultCollector::class);
@@ -1455,6 +1615,14 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return;
}
try {
$this->resolveBranch($projectRoot);
} catch (MissingDependency) {
// This run never asked for TIA, so a missing git must not turn it
// into a failure the way it does on the TIA path. Writing to the
// fallback baseline is the lesser of the two evils.
}
$touchedFiles = [];
foreach ($results as $testId => $result) {
@@ -1468,6 +1636,16 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$touchedFiles[$file] = true;
}
// A result is only ever invalidated through the edges of the test
// that produced it, so one recorded for a test the graph has no
// edges for could never be invalidated again — it would be replayed
// as settled however far the code around it moved. Only a complete
// run records the edges that would close that gap, so until one
// does, the test stays unknown.
if (! $complete && (! is_string($file) || ! $graph->knowsTest($file))) {
continue;
}
$graph->setResult(
$this->branch,
$testId,
@@ -1483,7 +1661,11 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
$graph->markKnownTestFiles(array_keys($touchedFiles));
}
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
// Pruning reads the absence of a test from this run as the test being
// gone. That only holds if every test was invited to report.
if ($complete) {
$graph->pruneStaleResults($this->branch, array_keys($touchedFiles), array_keys($results));
}
$this->saveGraph($graph);
$collector->reset();
@@ -1528,6 +1710,70 @@ final class Tia implements AddsOutput, HandlesArguments, Terminable
return $coverage->coverage;
}
/**
* Whether a selection-narrowing flag was given, either among the arguments
* PHPUnit receives or — for the flags `bin/pest` consumes itself — among
* the original argv. Explicit path arguments and `->only()` are detected
* separately.
*
* @param array<int, string> $arguments
*/
private function hasPartialSelection(array $arguments): bool
{
foreach (self::PARTIAL_SELECTION_FLAGS as $flag) {
if ($this->hasArgument($flag, $arguments)) {
return true;
}
if ($this->hasArgument($flag, $this->originalArguments)) {
return true;
}
}
return false;
}
/**
* Whether the run stopped before reaching every test it had queued.
*
* Covers `--bail`, `--retry` and every `--stop-on-*` flag, the equivalent
* `phpunit.xml` attributes, and an interrupted run — none of which narrow
* the selection up front, so hasPartialSelection() cannot see them. The
* tests queued behind the defect that halted the run never reported, and
* folding what did report into the baseline prunes the cached results of
* their siblings in every file the run had already entered.
*
* Deliberately unguarded. Both callers run only once PHPUnit's
* configuration is registered — the kernel reads it unguarded itself just
* before dispatching addOutput(), and flushWorkerReplay() bails out unless
* the worker actually executed something. Swallowing a failure here would
* report every truncated run as complete, which is the corruption this
* guards against in the first place.
*/
private function stoppedEarly(): bool
{
return TestResultFacade::shouldStop();
}
/**
* Resolves the baseline this run reads from and writes to, once.
*
* Results are written on runs where TIA itself took no part, and those
* never reach handleParent(). Without this the default would stand and
* every such run would write its results to `main`, whatever branch it
* actually ran on.
*/
private function resolveBranch(string $projectRoot): void
{
if ($this->branchResolved) {
return;
}
$this->branchResolved = true;
$this->branch = new ChangedFiles($projectRoot)->currentBranch() ?? $this->branch;
}
/**
* @param array<int, string> $arguments
*/
+1 -1
View File
@@ -68,7 +68,7 @@ final class Recorder
public function driverAvailable(): bool
{
if (! $this->driverChecked) {
if (function_exists('pcov\\start')) {
if (function_exists('pcov\\start') && filter_var((string) ini_get('pcov.enabled'), FILTER_VALIDATE_BOOL)) {
$this->driver = 'pcov';
$this->driverAvailable = true;
} elseif (function_exists('xdebug_start_code_coverage') && function_exists('xdebug_info')) {
+2 -2
View File
@@ -56,7 +56,7 @@ final class TableExtractor
$tables[strtolower($name)] = true;
}
$out = array_keys($tables);
$out = array_map(strval(...), array_keys($tables));
sort($out);
return $out;
@@ -112,7 +112,7 @@ final class TableExtractor
}
}
$out = array_keys($tables);
$out = array_map(strval(...), array_keys($tables));
sort($out);
return $out;
@@ -22,7 +22,7 @@ final readonly class EnsureTiaAssertionsAreRecordedOnFinished implements Finishe
if ($test instanceof TestMethod) {
$this->collector->recordAssertions(
$test->className().'::'.$test->methodName(),
$test->id(),
$event->numberOfAssertionsPerformed(),
);
}
@@ -21,7 +21,7 @@ final readonly class EnsureTiaResultsAreCollected implements PreparationStartedS
$test = $event->test();
if ($test instanceof TestMethod) {
$this->collector->testPrepared($test->className().'::'.$test->methodName(), $test->file());
$this->collector->testPrepared($test->id(), $test->file());
}
}
}
+56
View File
@@ -17,6 +17,33 @@ final class Str
private const string PREFIX = '__pest_evaluable_';
/**
* The list of names PHP reserves, and therefore refuses, as class names.
*
* @see https://github.com/php/php-src/blob/master/Zend/zend_compile.c
*
* @var array<int, string>
*/
private const array RESERVED_CLASS_NAMES = [
'array',
'bool',
'callable',
'false',
'float',
'int',
'iterable',
'mixed',
'never',
'null',
'object',
'parent',
'self',
'static',
'string',
'true',
'void',
];
/**
* Create a (unsecure & non-cryptographically safe) random alpha-numeric
* string value.
@@ -64,6 +91,35 @@ final class Str
return (string) preg_replace('/[^a-zA-Z0-9_\x80-\xff]/', '_', $code);
}
/**
* Determine if the given name is a valid PHP identifier, and therefore may
* be used as a single namespace name.
*/
public static function isValidIdentifier(string $name): bool
{
return preg_match('/^[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*$/', $name) === 1;
}
/**
* Determine if the given name may be declared as a class name by an `eval`.
*/
public static function isValidClassName(string $name): bool
{
if (! self::isValidIdentifier($name)) {
return false;
}
if (in_array(strtolower($name), self::RESERVED_CLASS_NAMES, true)) {
return false;
}
$tokens = token_get_all(sprintf('<?php %s;', $name));
// Anything the lexer sees as a keyword, like `list` or `match`, may not
// be used as a class name.
return is_array($tokens[1] ?? null) && $tokens[1][0] === T_STRING;
}
/**
* Get the portion of a string before the last occurrence of a given value.
*/
@@ -1,5 +1,5 @@
Pest Testing Framework 5.0.0-beta.1.
Pest Testing Framework 5.0.3.
USAGE: pest <file> [options]
@@ -168,6 +168,16 @@
AI OPTIONS:
--ai ..... Run a code snippet as a fully scaffolded test for AI verification
TIA OPTIONS:
--tia Re-run only the tests affected by your changes, replaying the rest from cache
--no-tia ......................... Disable test impact analysis for this run
--tia --fresh .... Discard the recorded dependency graph and record it again
--tia --filtered ............ Narrow the run to the affected test files only
--tia --locally ......... Enable test impact analysis on local machines only
--tia --baselined Fetch the shared dependency graph recorded by the CI baseline
--tia --refetch ......... Force a fresh fetch of the shared dependency graph
--baseline Output to standard output the test impact analysis storage directory
MUTATION TESTING OPTIONS:
--mutate .... Runs mutation testing, to understand the quality of your tests
--mutate --parallel ...................... Runs mutation testing in parallel
@@ -1,3 +1,3 @@
Pest Testing Framework 5.0.0-beta.1.
Pest Testing Framework 5.0.3.
+13 -13
View File
@@ -1,27 +1,27 @@
##teamcity[testSuiteStarted name='Tests/tests/Failure' locationHint='pest_qn://tests/.tests/Failure.php' flowId='1234']
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/Failure' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php' flowId='1234']
##teamcity[testCount count='8' flowId='1234']
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/.tests/Failure.php::it can fail with comparison' flowId='1234']
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/.tests/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
##teamcity[testStarted name='it can fail with comparison' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail with comparison' flowId='1234']
##teamcity[testFailed name='it can fail with comparison' message='Failed asserting that true matches expected false.' details='at tests/Fixtures/Suites/Failure.php:6' type='comparisonFailure' actual='true' expected='false' flowId='1234']
##teamcity[testFinished name='it can fail with comparison' duration='100000' flowId='1234']
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because of no assertions' flowId='1234']
##teamcity[testStarted name='it can be ignored because of no assertions' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because of no assertions' flowId='1234']
##teamcity[testIgnored name='it can be ignored because of no assertions' message='This test did not perform any assertions' details='' flowId='1234']
##teamcity[testFinished name='it can be ignored because of no assertions' duration='100000' flowId='1234']
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/.tests/Failure.php::it can be ignored because it is skipped' flowId='1234']
##teamcity[testStarted name='it can be ignored because it is skipped' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can be ignored because it is skipped' flowId='1234']
##teamcity[testIgnored name='it can be ignored because it is skipped' message='This test was ignored.' details='' flowId='1234']
##teamcity[testFinished name='it can be ignored because it is skipped' duration='100000' flowId='1234']
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/.tests/Failure.php::it can fail' flowId='1234']
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/.tests/Failure.php:18' flowId='1234']
##teamcity[testStarted name='it can fail' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it can fail' flowId='1234']
##teamcity[testFailed name='it can fail' message='oh noo' details='at tests/Fixtures/Suites/Failure.php:18' flowId='1234']
##teamcity[testFinished name='it can fail' duration='100000' flowId='1234']
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/.tests/Failure.php::it throws exception' flowId='1234']
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/.tests/Failure.php:22' flowId='1234']
##teamcity[testStarted name='it throws exception' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it throws exception' flowId='1234']
##teamcity[testFailed name='it throws exception' message='Exception: test error' details='at tests/Fixtures/Suites/Failure.php:22' flowId='1234']
##teamcity[testFinished name='it throws exception' duration='100000' flowId='1234']
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/.tests/Failure.php::it is not done yet' flowId='1234']
##teamcity[testStarted name='it is not done yet' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is not done yet' flowId='1234']
##teamcity[testFinished name='it is not done yet' duration='100000' flowId='1234']
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/.tests/Failure.php::build this one.' flowId='1234']
##teamcity[testStarted name='build this one.' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::build this one.' flowId='1234']
##teamcity[testFinished name='build this one.' duration='100000' flowId='1234']
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/.tests/Failure.php::it is passing' flowId='1234']
##teamcity[testStarted name='it is passing' locationHint='pest_qn://tests/Fixtures/Suites/Failure.php::it is passing' flowId='1234']
##teamcity[testFinished name='it is passing' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='Tests/tests/Failure' flowId='1234']
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/Failure' flowId='1234']
Tests: 3 failed, 1 risky, 2 todos, 1 skipped, 1 passed (3 assertions)
Duration: 1.00s
+8 -8
View File
@@ -1,18 +1,18 @@
##teamcity[testSuiteStarted name='Tests/tests/SuccessOnly' locationHint='pest_qn://tests/.tests/SuccessOnly.php' flowId='1234']
##teamcity[testSuiteStarted name='Tests/Fixtures/Suites/SuccessOnly' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php' flowId='1234']
##teamcity[testCount count='4' flowId='1234']
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/.tests/SuccessOnly.php::it can pass with comparison' flowId='1234']
##teamcity[testStarted name='it can pass with comparison' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison' flowId='1234']
##teamcity[testFinished name='it can pass with comparison' duration='100000' flowId='1234']
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can also pass' flowId='1234']
##teamcity[testStarted name='can also pass' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can also pass' flowId='1234']
##teamcity[testFinished name='can also pass' duration='100000' flowId='1234']
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset' flowId='1234']
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
##teamcity[testSuiteStarted name='can pass with dataset' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset' flowId='1234']
##teamcity[testStarted name='can pass with dataset with data set "(true)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::can pass with dataset with data set "(true)"' flowId='1234']
##teamcity[testFinished name='can pass with dataset with data set "(true)"' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='can pass with dataset' flowId='1234']
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/.tests/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
##teamcity[testSuiteStarted name='`block` → can pass with dataset in describe block' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testStarted name='`block` → can pass with dataset in describe block with data set "(1)"' locationHint='pest_qn://tests/Fixtures/Suites/SuccessOnly.php::`block` → can pass with dataset in describe block with data set "(1)"' flowId='1234']
##teamcity[testFinished name='`block` → can pass with dataset in describe block with data set "(1)"' duration='100000' flowId='1234']
##teamcity[testSuiteFinished name='`block` → can pass with dataset in describe block' flowId='1234']
##teamcity[testSuiteFinished name='Tests/tests/SuccessOnly' flowId='1234']
##teamcity[testSuiteFinished name='Tests/Fixtures/Suites/SuccessOnly' flowId='1234']
Tests: 4 passed (4 assertions)
Duration: 1.00s
+1 -14
View File
@@ -1,17 +1,4 @@
WARN Tests\Fixtures\CollisionTest
- error
- success
PASS Tests\Fixtures\DirectoryWithTests\ExampleTest
✓ it example 1
PASS Tests\Fixtures\ExampleTest
✓ it example 2
WARN Tests\Fixtures\Inheritance\Base\ExampleTest
- example
PASS Tests\Fixtures\Inheritance\ExampleTest
✓ example
Tests: 3 skipped, 3 passed (3 assertions)
Tests: 1 passed (1 assertions)
+12 -1
View File
@@ -1565,6 +1565,9 @@
✓ it allows performing no expectations without being risky
✓ a "describe" group of tests → it allows performing no expectations without being risky
PASS Tests\Features\Tia
✓ it does not run user hooks when replaying cached skipped and incomplete results
PASS Tests\Features\Ticket
✓ it may be associated with an ticket #1, #2
✓ nested → it may be associated with an ticket #1, #4, #5, #6, #3
@@ -1908,6 +1911,11 @@
✓ activateLinkTracking() → it tracks linked sources across consecutive tests
✓ activateLinkTracking() → it records nothing while inactive
PASS Tests\Unit\Plugins\Tia\ResultKey
✓ it keys a result per dataset row rather than per method
✓ it records assertions against the same per-dataset key
✓ it leaves a test without a dataset keyed by class and method
PASS Tests\Unit\Plugins\Tia\TableExtractor
✓ fromSql() → it extracts tables from plain DML
✓ fromSql() → it extracts tables from joins
@@ -1916,10 +1924,12 @@
✓ fromSql() → it records the table, not the schema, for qualified identifiers
✓ fromSql() → it handles quoted identifiers
✓ fromSql() → it ignores schema metadata tables
✓ fromSql() → it does not leak int keys for numeric identifiers
✓ fromSql() → it returns nothing for non-DML statements
✓ fromMigrationSource() → it extracts tables from Schema builder calls
✓ fromMigrationSource() → it extracts tables from raw DDL statements
✓ fromMigrationSource() → it records the table, not the schema, in qualified DDL and DML
✓ fromMigrationSource() → it does not leak int keys for numeric table names
✓ fromMigrationSource() → it extracts tables from DB::table calls
PASS Tests\Unit\Plugins\Tia\TestPaths
@@ -2153,6 +2163,7 @@
✓ a parallel test can extend another test with same name
✓ parallel reports invalid datasets as failures
✓ parallel can have multiple exclude-groups
✓ parallel can have multiple groups
PASS Tests\Visual\ParallelNestedDatasets
✓ parallel loads nested datasets from nested directories
@@ -2186,4 +2197,4 @@
✓ pass with dataset with ('my-datas-set-value')
✓ within describe → pass with dataset with ('my-datas-set-value')
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1542 passed (3375 assertions)
Tests: 1 deprecated, 4 warnings, 5 incomplete, 2 notices, 40 todos, 35 skipped, 1549 passed (3389 assertions)
+5 -5
View File
@@ -18,27 +18,27 @@ $run = function (string $target): array {
};
test('reports missing datasets as errors for a single file run', function () use ($run): void {
$result = $run('tests/.tests/IssueOnly.php');
$result = $run('tests/Fixtures/Suites/IssueOnly.php');
expect($result['output'])
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain('FAILED')
->toContain('Tests: 1 failed')
->and($result['code'])->not->toBe(0);
})->skipOnWindows();
test('reports missing datasets as errors alongside passing tests', function () use ($run): void {
$result = $run('tests/.tests/IssueWithPassing.php');
$result = $run('tests/Fixtures/Suites/IssueWithPassing.php');
expect($result['output'])
->toContain("A dataset with the name `missing` does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain("A dataset with the name [missing] does not exist. You can create it using `dataset('missing', ['a', 'b']);`.")
->toContain('1 passed')
->toContain('1 failed')
->and($result['code'])->not->toBe(0);
})->skipOnWindows();
test('reports dataset closure exceptions as errors', function () use ($run): void {
$result = $run('tests/.tests/DatasetClosureThrows.php');
$result = $run('tests/Fixtures/Suites/DatasetClosureThrows.php');
expect($result['output'])
->toContain('boom from dataset')
+2 -2
View File
@@ -10,12 +10,12 @@ beforeEach(function (): void {
});
it('throws exception if dataset does not exist', function (): void {
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name `first` does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
expect(fn () => DatasetsRepository::resolve(['first'], __FILE__))->toThrow(DatasetDoesNotExist::class, "A dataset with the name [first] does not exist. You can create it using `dataset('first', ['a', 'b']);`.");
});
it('throws exception if dataset already exist', function (): void {
DatasetsRepository::set('second', [[]], __DIR__);
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name `second` already exists in scope ['.__DIR__.'].');
expect(fn () => DatasetsRepository::set('second', [[]], __DIR__))->toThrow(DatasetAlreadyExists::class, 'A dataset with the name [second] already exists in scope ['.__DIR__.'].');
});
it('sets closures', function (): void {
+1 -1
View File
@@ -38,7 +38,7 @@ class State
{
public array $runCount = [];
public array $appliedCount = [];
public array $appliedCount;
public function __construct()
{
+1 -1
View File
@@ -142,7 +142,7 @@ it('works as higher order test')
it('fails after exhausting all retries', function (): void {
$process = new Process(
['php', 'bin/pest', 'tests/.tests/FlakyFailure.php'],
['php', 'bin/pest', 'tests/Fixtures/Suites/FlakyFailure.php'],
dirname(__DIR__, 2),
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1'],
);
+84
View File
@@ -0,0 +1,84 @@
<?php
use Pest\Plugins\Tia;
use Pest\Plugins\Tia\ChangedFiles;
use Pest\Plugins\Tia\FileState;
use Pest\Plugins\Tia\Fingerprint;
use Pest\Plugins\Tia\Graph;
use Pest\Plugins\Tia\Storage;
use Pest\Support\Str;
use Symfony\Component\Process\Process;
it('does not run user hooks when replaying cached skipped and incomplete results', function (): void {
$projectRoot = dirname(__DIR__, 2);
$home = sys_get_temp_dir().'/pest-tia-'.bin2hex(random_bytes(8));
$fixture = 'tests/Fixtures/Suites/TiaReplayHooks.php';
mkdir($home, 0755, true);
try {
$changedFiles = new ChangedFiles($projectRoot);
$branch = $changedFiles->currentBranch() ?? 'main';
$sha = $changedFiles->currentSha();
$id = fn (string $description): string => 'P\Tests\Fixtures\Suites\TiaReplayHooks::'.Str::evaluable($description);
$graph = new Graph($projectRoot);
$graph->setFingerprint(Fingerprint::compute($projectRoot));
$graph->setRecordedAtSha($branch, $sha);
// Hashes the working tree as it stands, so the replay sees nothing as changed.
$graph->setLastRunTree($branch, $changedFiles->snapshotTree($changedFiles->since($sha) ?? []));
$graph->markKnownTestFiles([$fixture]);
$graph->setResult($branch, $id('replayed pass'), 0, '', 0.01, 1, $fixture);
$graph->setResult($branch, $id('replayed skip'), 1, 'cached skip', 0.01, 0, $fixture);
$graph->setResult($branch, $id('replayed incomplete'), 2, 'cached incomplete', 0.01, 0, $fixture);
$json = $graph->encode();
expect($json)->not->toBeNull();
$originalHome = getenv('HOME');
putenv('HOME='.$home);
try {
$storage = new FileState(Storage::tempDir($projectRoot));
} finally {
putenv($originalHome === false ? 'HOME' : 'HOME='.$originalHome);
}
expect($storage->write(Tia::KEY_GRAPH, (string) $json))->toBeTrue();
$process = new Process(
['php', 'bin/pest', $fixture, '--tia'],
$projectRoot,
[
'COLLISION_PRINTER' => 'DefaultPrinter',
'COLLISION_IGNORE_DURATION' => 'true',
'PARATEST' => 0,
'PAO_DISABLE' => '1',
'HOME' => $home,
],
);
$process->run();
$output = removeAnsiEscapeSequences($process->getOutput().$process->getErrorOutput());
// Both hooks throw, so the run stays green only if neither one ran.
expect($output)->toContain('3 replayed')
->and($output)->not->toContain('must not run for replayed tests')
->and($output)->toContain('1 incomplete, 1 skipped, 1 passed')
->and($process->getExitCode())->toBe(0);
} finally {
$paths = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($home, FilesystemIterator::SKIP_DOTS),
RecursiveIteratorIterator::CHILD_FIRST,
);
foreach ($paths as $path) {
$path->isDir() ? @rmdir($path->getPathname()) : @unlink($path->getPathname());
}
@rmdir($home);
}
})->skipOnWindows();
@@ -0,0 +1,5 @@
<?php
// The folder name creates the namespace segment `2fa`, which starts with a number.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `list`, which is a PHP keyword.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `int`, which is a name PHP reserves.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
// The file name creates the class `2fa`, which starts with a number.
it('never runs')->assertTrue(true);
@@ -0,0 +1,5 @@
<?php
it('belongs to group one', function () {
expect(true)->toBeTrue();
})->group('one');
@@ -0,0 +1,5 @@
<?php
it('belongs to group three', function () {
expect(true)->toBeTrue();
})->group('three');
@@ -0,0 +1,5 @@
<?php
it('belongs to group two', function () {
expect(true)->toBeTrue();
})->group('two');
+24
View File
@@ -0,0 +1,24 @@
<?php
// Only ever run through `tests/Features/Tia.php`, against a seeded TIA graph.
// Both hooks throw, so a replayed test that wrongly runs one fails the run.
beforeEach(function (): void {
throw new RuntimeException('The beforeEach hook must not run for replayed tests.');
});
afterEach(function (): void {
throw new RuntimeException('The afterEach hook must not run for replayed tests.');
});
test('replayed pass', function (): void {
expect(true)->toBeTrue();
});
test('replayed skip', function (): void {
expect(true)->toBeTrue();
});
test('replayed incomplete', function (): void {
expect(true)->toBeTrue();
});
+114
View File
@@ -0,0 +1,114 @@
<?php
declare(strict_types=1);
use Pest\Plugins\Tia\ResultCollector;
use Pest\Subscribers\EnsureTiaAssertionsAreRecordedOnFinished;
use Pest\Subscribers\EnsureTiaResultsAreCollected;
use PHPUnit\Event\Code\TestDox;
use PHPUnit\Event\Code\TestMethod;
use PHPUnit\Event\Telemetry\Duration;
use PHPUnit\Event\Telemetry\Info;
use PHPUnit\Event\Telemetry\MemoryUsage;
use PHPUnit\Event\Telemetry\System;
use PHPUnit\Event\Telemetry\SystemCpuTimeMeter;
use PHPUnit\Event\Telemetry\SystemGarbageCollectorStatusProvider;
use PHPUnit\Event\Telemetry\SystemMemoryMeter;
use PHPUnit\Event\Telemetry\SystemStopWatch;
use PHPUnit\Event\Test\Finished;
use PHPUnit\Event\Test\PreparationStarted;
use PHPUnit\Event\TestData\DataFromDataProvider;
use PHPUnit\Event\TestData\TestDataCollection;
use PHPUnit\Metadata\MetadataCollection;
function tiaResultKeyTestMethod(?string $dataSetName): TestMethod
{
$testData = $dataSetName === null
? TestDataCollection::fromArray([])
: TestDataCollection::fromArray([DataFromDataProvider::from($dataSetName, '', '')]);
return new TestMethod(
'Tests\Feature\OrderTest',
'it prices an order',
'/project/tests/Feature/OrderTest.php',
1,
new TestDox('Order', 'it prices an order', 'it prices an order'),
MetadataCollection::fromArray([]),
$testData,
);
}
function tiaResultKeyTelemetryInfo(): Info
{
$system = new System(
new SystemStopWatch,
new SystemMemoryMeter,
new SystemGarbageCollectorStatusProvider,
new SystemCpuTimeMeter,
);
$zeroDuration = Duration::fromSecondsAndNanoseconds(0, 0);
$zeroMemory = MemoryUsage::fromBytes(0);
$zeroCpuTime = $system->snapshot()->userCpuTime();
return new Info(
$system->snapshot(),
$zeroDuration,
$zeroMemory,
$zeroDuration,
$zeroMemory,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
$zeroCpuTime,
);
}
it('keys a result per dataset row rather than per method', function (): void {
$collector = new ResultCollector;
$subscriber = new EnsureTiaResultsAreCollected($collector);
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')));
$collector->testPassed();
$collector->finishTest();
$subscriber->notify(new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('fake')));
$collector->testSkipped('the fake driver does not report balances');
$collector->finishTest();
// Without the dataset in the key both rows write to `Class::method`, so the
// second one overwrites the first and a replay hands every row the same
// status — a passing row reported as skipped, or a failing one as passed.
expect(array_keys($collector->all()))->toBe([
'Tests\Feature\OrderTest::it prices an order#opp',
'Tests\Feature\OrderTest::it prices an order#fake',
]);
});
it('records assertions against the same per-dataset key', function (): void {
$collector = new ResultCollector;
new EnsureTiaResultsAreCollected($collector)->notify(
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp')),
);
$collector->testPassed();
new EnsureTiaAssertionsAreRecordedOnFinished($collector)->notify(
new Finished(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod('opp'), 7),
);
expect($collector->all()['Tests\Feature\OrderTest::it prices an order#opp']['assertions'])->toBe(7);
});
it('leaves a test without a dataset keyed by class and method', function (): void {
$collector = new ResultCollector;
new EnsureTiaResultsAreCollected($collector)->notify(
new PreparationStarted(tiaResultKeyTelemetryInfo(), tiaResultKeyTestMethod(null)),
);
$collector->testPassed();
expect(array_keys($collector->all()))->toBe(['Tests\Feature\OrderTest::it prices an order']);
});
+17
View File
@@ -47,6 +47,15 @@ describe('fromSql()', function (): void {
->and(TableExtractor::fromSql('select * from information_schema.tables'))->toBeEmpty();
});
it('does not leak int keys for numeric identifiers', function (): void {
// `substring(x FROM 1 FOR 3)` is standard SQL, and the `1` matches the
// FROM pattern. Collecting names as array keys makes PHP coerce the
// numeric string to an int, which then violates the declared
// list<string> and blows up Recorder::linkTable(string).
expect(TableExtractor::fromSql('select substring(name from 1 for 3) from users'))
->each->toBeString();
});
it('returns nothing for non-DML statements', function (): void {
expect(TableExtractor::fromSql('PRAGMA foreign_keys = ON'))->toBeEmpty()
->and(TableExtractor::fromSql(''))->toBeEmpty()
@@ -89,6 +98,14 @@ describe('fromMigrationSource()', function (): void {
->toBe(['audits', 'events', 'sessions', 'settings', 'users']);
});
it('does not leak int keys for numeric table names', function (): void {
// A table named `123` is a legal quoted identifier. Collecting names as
// array keys makes PHP coerce it to an int, breaking the declared
// list<string>, so it must survive as a string rather than be dropped.
expect(TableExtractor::fromMigrationSource("DB::table('123')->insert([]);"))
->toBe(['123']);
});
it('extracts tables from DB::table calls', function (): void {
expect(TableExtractor::fromMigrationSource("DB::table('permissions')->insert([]);"))
->toBe(['permissions']);
+1 -1
View File
@@ -15,7 +15,7 @@ it('does not allow to add the same test description twice', function (): void {
$testSuite->tests->set($method);
})->throws(
TestAlreadyExist::class,
sprintf('A test with the description `%s` already exists in the filename `%s`.', 'bar', 'foo'),
sprintf('A test with the description [%s] already exists in the filename [%s].', 'bar', 'foo'),
);
it('does not allow static closures', function (): void {
+8 -8
View File
@@ -29,30 +29,30 @@ $run = function () {
$normalizedPath = (fn (string $path): string => str_replace('/', DIRECTORY_SEPARATOR, $path));
test('junit output', function () use ($normalizedPath, $run): void {
$result = $run('tests/.tests/SuccessOnly.php');
$result = $run('tests/Fixtures/Suites/SuccessOnly.php');
expect($result['testsuite']['@attributes'])
->name->toBe('Tests\tests\SuccessOnly')
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
->tests->toBe('4')
->assertions->toBe('4')
->errors->toBe('0')
->failures->toBe('0')
->skipped->toBe('0')
->and($result['testsuite']['testcase'])->toHaveCount(2)
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
->and($result['testsuite']['testcase'][0]['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
});
test('junit with parallel', function () use ($normalizedPath, $run): void {
$result = $run('tests/.tests/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
$result = $run('tests/Fixtures/Suites/SuccessOnly.php', '--parallel', '--processes=1', '--filter', 'can pass with comparison');
expect($result['testsuite']['@attributes'])
->name->toBe('Tests\tests\SuccessOnly')
->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php'))
->name->toBe('Tests\Fixtures\Suites\SuccessOnly')
->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php'))
->tests->toBe('1')
->assertions->toBe('1')
->errors->toBe('0')
->failures->toBe('0')
->skipped->toBe('0')
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/.tests/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\tests\SuccessOnly')->classname->toBe('Tests.tests.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
->and($result['testsuite']['testcase']['@attributes'])->name->toBe('it can pass with comparison')->file->toBe($normalizedPath('tests/Fixtures/Suites/SuccessOnly.php::it can pass with comparison'))->class->toBe('Tests\Fixtures\Suites\SuccessOnly')->classname->toBe('Tests.Fixtures.Suites.SuccessOnly')->assertions->toBe('1')->time->toStartWith('0.0');
});
+12 -4
View File
@@ -24,13 +24,13 @@ test('parallel', function () use ($run): void {
$file = file_get_contents(__FILE__);
$file = preg_replace(
'/\$expected = \'.*?\';/',
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';",
"\$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';",
$file,
);
file_put_contents(__FILE__, $file);
}
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1525 passed (3322 assertions)';
$expected = '1 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1531 passed (3334 assertions)';
expect($output)
->toContain("Tests: {$expected}")
@@ -42,8 +42,8 @@ test('a parallel test can extend another test with same name', function () use (
})->skipOnWindows();
test('parallel reports invalid datasets as failures', function () use ($run): void {
expect($run('tests/.tests/ParallelInvalidDataset'))
->toContain("A dataset with the name `missing.dataset` does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
expect($run('tests/Fixtures/Suites/ParallelInvalidDataset'))
->toContain("A dataset with the name [missing.dataset] does not exist. You can create it using `dataset('missing.dataset', ['a', 'b']);`.")
->toContain('Tests: 1 failed, 1 passed (1 assertions)')
->toContain('Parallel: 3 processes');
})->skipOnWindows();
@@ -58,3 +58,11 @@ test('parallel can have multiple exclude-groups', function () use ($run): void {
expect((int) $doubleMatch[1])->toBeLessThan((int) $singleMatch[1])
->and($doubleExclude)->toContain('Parallel: 3 processes');
})->skipOnWindows();
test('parallel can have multiple groups', function () use ($run): void {
$output = $run('tests/Fixtures/Suites/MultipleGroups', '--group=one', '--group=two');
expect($output)
->toContain('Tests: 2 passed (2 assertions)')
->toContain('Parallel: 3 processes');
})->skipOnWindows();
+1 -1
View File
@@ -27,7 +27,7 @@ test('allows to run a single test', function () use ($run, $snapshot): void {
})->skipOnWindows();
test('allows to run a directory', function () use ($run, $snapshot): void {
expect($run('tests/Fixtures'))->toContain($snapshot('allows-to-run-a-directory'));
expect($run('tests/Fixtures/DirectoryWithTests'))->toContain($snapshot('allows-to-run-a-directory'));
})->skipOnWindows();
it('disable decorating printer when colors is set to never', function () use ($snapshot): void {
+1 -1
View File
@@ -10,7 +10,7 @@ function normalize_windows_os_output(string $text): string
}
test('visual snapshot of team city', function (string $testFile): void {
$testsPath = dirname(__DIR__)."/.tests/$testFile";
$testsPath = dirname(__DIR__)."/Fixtures/Suites/$testFile";
$snapshot = implode(DIRECTORY_SEPARATOR, [
dirname(__DIR__),
+2 -2
View File
@@ -8,7 +8,7 @@ test('filter works with unicode characters in filename', function (): void {
$process = new Process([
'php',
'bin/pest',
'tests/.tests/StraßenTest.php',
'tests/Fixtures/Suites/StraßenTest.php',
'--colors=never',
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);
@@ -26,7 +26,7 @@ test('filter with unicode regex matches unicode filename', function (): void {
'php',
'bin/pest',
'--filter=.*Straß.*',
'tests/.tests/',
'tests/Fixtures/Suites/',
'--colors=never',
], dirname(__DIR__, 2), ['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true', 'PAO_DISABLE' => '1']);