mirror of
https://github.com/pestphp/pest.git
synced 2026-09-07 15:23:34 +02:00
nuno maduro
75 lines
5.2 KiB
Markdown
75 lines
5.2 KiB
Markdown
---
|
|
name: snapshots-update
|
|
description: "Write each snapshot of this project again, then run the integration suite until it passes. Use when the user asks to update the snapshots, when a test of `tests/Visual/` fails, or when `composer test:integration` fails after a change of the number of the tests."
|
|
disable-model-invocation: true
|
|
---
|
|
|
|
# Snapshots
|
|
|
|
Write each snapshot of this project again, then run `composer test:integration` until it passes.
|
|
|
|
`tests/Pest.php` puts each test of `tests/Visual/` in the group `integration`, and each of those tests asserts the output of a full run of the suite. Thus a change of the number of the tests, of the number of the assertions, or of one line of the output of `bin/pest` breaks the integration suite, and this skill repairs it.
|
|
|
|
Obey the order of the sections. Section 3 writes the number of the tests of a clean run, and section 4 writes that number into `tests/.snapshots/success.txt`. The reverse order writes the failure of the test `parallel` into that snapshot, and the integration suite then fails on the test `visual snapshot of test suite on success`.
|
|
|
|
## 1. Read the working tree
|
|
|
|
Run `git status --short`. Name each file that carries a change, then continue with that change in place, because the user starts this skill in the middle of a change.
|
|
|
|
Write no commit and no tag during this work. Leave each change in the working tree.
|
|
|
|
## 2. Turn off the coverage driver
|
|
|
|
Put `XDEBUG_MODE=off` in front of each command of this skill. `.github/workflows/tests.yml` gives `coverage: none` to each job, thus a run of this project on GitHub collects no coverage, and `tests/.snapshots/success.txt` holds the line `it adds coverage if --coverage exist → Coverage is not available`. A run with a coverage driver writes one test more as passed, and the snapshot then fails on GitHub.
|
|
|
|
Run `php -m | grep -iE "xdebug|pcov"` and read the name of each coverage driver of this computer. `XDEBUG_MODE=off` turns off `xdebug`. Add `-d pcov.enabled=0` to `php` for `pcov`, and run the command of the composer script `update:snapshots` directly for that flag.
|
|
|
|
## 3. Write the summary of the parallel run
|
|
|
|
Run this command with a timeout of 900 seconds:
|
|
|
|
```
|
|
XDEBUG_MODE=off COLLISION_PRINTER=DefaultPrinter COLLISION_IGNORE_DURATION=true PAO_DISABLE=1 php bin/pest --parallel --processes=3 --exclude-group=integration
|
|
```
|
|
|
|
Read the line that starts with `Tests:`. Take the text after `Tests:` up to and with `(<number> assertions)`, such as `2 deprecated, 4 warnings, 5 incomplete, 3 notices, 40 todos, 27 skipped, 1578 passed (3426 assertions)`. That text is the summary.
|
|
|
|
Run `grep -n "assertions)';" tests/Visual/Parallel.php`. The command gives one line, and that line holds the value of `$expected` of the test `parallel`. Write the summary in that line when the two texts differ, and change no other line of the file.
|
|
|
|
## 4. Write each snapshot again
|
|
|
|
Run `XDEBUG_MODE=off composer update:snapshots` with a timeout of 1800 seconds. The command runs `REBUILD_SNAPSHOTS=true php bin/pest --update-snapshots --exclude-group=tia`, and it writes `tests/.snapshots/success.txt` and each file under `tests/.pest/snapshots/`. The command starts a full suite inside a test of `tests/Visual/`, thus the command needs some minutes.
|
|
|
|
The command exits with the code 2 after a correct run. Read the output and accept these three results:
|
|
|
|
- a `FAILED` test of `tests/Visual/`, because the test reads the old snapshot and writes the new snapshot in the same run.
|
|
- a `FAILED` test of `tests/Features/Flaky.php`, because `--update-snapshots` writes each snapshot and a flaky test stops the retry when a snapshot changes.
|
|
- a `RISKY` test that performs no assertion, because a test that only writes a snapshot asserts nothing.
|
|
|
|
Run `XDEBUG_MODE=off composer test:unit` for a `FAILED` test outside `tests/Visual/` and outside `tests/Features/Flaky.php`. That command writes no snapshot, thus it separates a regression of the code from an effect of `--update-snapshots`. Stop when the test fails again, give the report of section 6, and write no snapshot again.
|
|
|
|
Run `grep -c "Coverage is not available" tests/.snapshots/success.txt`. The count is 1. Return to section 2 for the count 0, because a coverage driver of this computer stays on.
|
|
|
|
## 5. Run the integration suite
|
|
|
|
Run `XDEBUG_MODE=off composer test:integration` with a timeout of 3600 seconds.
|
|
|
|
Read each test that failed, then take one action.
|
|
|
|
| Test that failed | Action |
|
|
| --- | --- |
|
|
| the test `parallel` of `tests/Visual/Parallel.php` | Return to section 3, because the numbers of the suite moved after the last measure. |
|
|
| a different test of `tests/Visual/` | Return to section 3, because `tests/.snapshots/success.txt` holds the numbers of section 3. |
|
|
| a test outside `tests/Visual/` | Stop. Give the report of section 6, and write no snapshot again. |
|
|
|
|
Return to section 3 two times at the most. Give the report of section 6 after the last run, and name each test that failed in that run.
|
|
|
|
## 6. Report
|
|
|
|
Give four items:
|
|
|
|
- each file that this work changed, from `git status --short`.
|
|
- the summary that `tests/Visual/Parallel.php` holds now, and the summary that the file held before this work.
|
|
- the result of the last run of `composer test:integration`, and the name of each test that failed in that run.
|
|
- each section that you repeated, and the reason.
|