mirror of
https://github.com/Respect/Validation.git
synced 2026-03-23 02:04:43 +01:00
9862963d06
A new workflow, continuous-integration-perf.yml was introduced. It: - Checks out the `benchmarks` branch locally. - Runs the benchmarks, accounting for non-existant baselines and target (main/PR). - Stores the .phpbench storage folder and a human-readable report in the `benchmarks` branch. - Does not make a PR fail, and never reports a failure when merging to main. - Allows workflow_dispatch for quick re-runs, and has an option to reset the baseline in case something changes (GitHub runner setup gets faster/slower, major refactors, etc). Thus, it keeps a historical record of all benchmark results. These results can be viewed by exploring GitHub via the web interface and seeing the changes in `latest.md` (the human file commited). Additionally, one can clone the `benchmarks` branch and run `phpbench log` to explore the history in more detail. Some adjustments to previously added benchmarks were made: - Assertions were included in order to track time and memory tresholds. - The benchmarks are now more surgical, and address the concrete validators instead of the whole chain validate. These changes were made to make benchmarks more isolated, with the intention of adding chain-related benchmarks separately in the future.
88 lines
2.9 KiB
YAML
88 lines
2.9 KiB
YAML
name: Continuous Integration (perf)
|
|
|
|
on:
|
|
push:
|
|
paths-ignore:
|
|
- 'bin/**'
|
|
- 'docs/**'
|
|
pull_request:
|
|
paths-ignore:
|
|
- 'bin/**'
|
|
- 'docs/**'
|
|
workflow_dispatch:
|
|
inputs:
|
|
baseline:
|
|
description: 'Baseline mode. latest: compare against latest benchmarks; rebaseline: store new baseline.'
|
|
type: choice
|
|
default: 'latest'
|
|
options:
|
|
- 'latest'
|
|
- 'rebaseline'
|
|
|
|
jobs:
|
|
tests:
|
|
name: Benchmarks
|
|
runs-on: ubuntu-latest
|
|
continue-on-error: true # This job is experimental
|
|
permissions:
|
|
contents: write
|
|
pull-requests: write
|
|
|
|
strategy:
|
|
matrix:
|
|
php-version:
|
|
- "8.5"
|
|
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v6
|
|
with:
|
|
persist-credentials: true
|
|
|
|
- name: Install PHP
|
|
uses: shivammathur/setup-php@v2
|
|
with:
|
|
php-version: ${{ matrix.php-version }}
|
|
extensions: xdebug
|
|
|
|
- name: Install Dependencies
|
|
run: composer install --prefer-dist ${{ matrix.composer-extra-arguments }}
|
|
|
|
- name: Fetch Benchmarks
|
|
run: |
|
|
git fetch origin benchmarks
|
|
mkdir -p .phpbench
|
|
git checkout origin/benchmarks -- .phpbench || echo "No previous benchmarks found"
|
|
|
|
- name: Run Benchmarks
|
|
run: |
|
|
# Baseline does not exist or rebaseline requested. Generate it.
|
|
if [ -z "$(ls -A .phpbench)" ] || [ "${{ github.event.inputs.baseline || 'latest' }}" = "rebaseline" ]; then
|
|
vendor/bin/phpbench run --report=aggregate --progress=plain --store --tag=${GITHUB_SHA}
|
|
|
|
# On main branch push, update baseline with tolerance for failures.
|
|
elif [ "${GITHUB_REF}" = "refs/heads/main" ] && [ "${GITHUB_EVENT_NAME}" = "push" ]; then
|
|
vendor/bin/phpbench run --report=aggregate --progress=plain --store --tag=${GITHUB_SHA} --ref=latest --tolerate-failure
|
|
|
|
# On other branches, compare against latest baseline, fails if worse.
|
|
else
|
|
vendor/bin/phpbench run --report=aggregate --progress=plain --store --tag=${GITHUB_SHA} --ref=latest
|
|
fi
|
|
|
|
# Generate report for human consumption
|
|
vendor/bin/phpbench report --report=aggregate --ref=latest |
|
|
tail -n+2 | head -n-2 | tr '+' '|' > report.md
|
|
|
|
cat report.md
|
|
|
|
- name: Commit Benchmark Results
|
|
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
|
run: |
|
|
set -euo pipefail
|
|
git config user.name "github-actions[bot]"
|
|
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
git checkout benchmarks
|
|
mv -f report.md latest.md
|
|
git add .phpbench latest.md
|
|
git commit -m "Store benchmark results [skip ci]" || echo "No changes to commit"
|
|
git push origin benchmarks
|