Remove AppVeyor, run E2E tests during CI #1365
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,11 +16,12 @@ jobs: | |
|
|
||
| strategy: | ||
| matrix: | ||
| operating-system: [ubuntu-latest] | ||
| php-version: ['7.4'] | ||
| dependencies: [''] | ||
| coverage-driver: [pcov, xdebug] | ||
| include: | ||
| - { operating-system: 'windows-latest', php-version: '7.4', dependencies: '', coverage-driver: 'xdebug' } | ||
| - { operating-system: 'ubuntu-latest', php-version: '7.4', dependencies: '--prefer-lowest', coverage-driver: 'pcov' } | ||
| - { operating-system: 'ubuntu-latest', php-version: '8.0', dependencies: '--ignore-platform-req=php', coverage-driver: 'pcov' } | ||
|
|
||
|
|
@@ -36,6 +37,7 @@ jobs: | |
| with: | ||
| php-version: ${{ matrix.php-version }} | ||
| coverage: ${{ matrix.coverage-driver }} | ||
| ini-values: memory_limit=512M | ||
| tools: composer:v2 | ||
|
|
||
| - name: Get composer cache directory | ||
|
|
@@ -46,9 +48,9 @@ jobs: | |
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ steps.composer-cache.outputs.dir }} | ||
| key: composer-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('composer.*') }}-${{ matrix.dependencies }} | ||
| restore-keys: | | ||
| composer-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('composer.*') }}- | ||
| composer-${{ runner.os }}-${{ matrix.php-version }}- | ||
| composer-${{ runner.os }}- | ||
| composer- | ||
|
|
@@ -57,16 +59,48 @@ jobs: | |
| run: | | ||
| composer update --no-interaction --prefer-dist --no-progress ${{ matrix.dependencies }} | ||
|
|
||
| - name: Run tests | ||
| shell: bash | ||
| run: | | ||
| make test-unit | ||
|
|
||
| - name: Run integration tests | ||
| shell: bash | ||
| run: | | ||
| make test-unit PHPUNIT_GROUP=integration | ||
|
|
||
| - name: Cache E2E tests dependencies | ||
| if: runner.os == 'Windows' | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: tests/e2e/*/vendor | ||
| key: e2e-vendor-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('tests/e2e/*/composer.json') }}-${{ matrix.dependencies }} | ||
| restore-keys: | | ||
| e2e-vendor-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('tests/e2e/*/composer.json') }}- | ||
| e2e-vendor-${{ runner.os }}-${{ matrix.php-version }}- | ||
| e2e-vendor-${{ runner.os }}- | ||
| composer- | ||
|
There was a problem hiding this comment. Installing anything with Composer is insanely slow in Windows, just compare these 4 seconds on Linux with these 34 seconds on Windows. The solution is to cache the dependencies, but not only archives, the whole This might be a mistake, but let's see. |
||
|
|
||
| - name: Install Composer dependencies for E2E tests | ||
| shell: bash | ||
| run: | | ||
| ls tests/e2e/*/composer.json | xargs dirname | | ||
| xargs -I{} composer --working-dir={} update --no-interaction --prefer-dist --no-progress ${{ matrix.dependencies }} | ||
|
There was a problem hiding this comment. Surprise! This works on Windows too! |
||
|
|
||
| - name: Run a subset of E2E tests | ||
| if: runner.os == 'Windows' | ||
| shell: bash | ||
| run: | | ||
| make test-e2e-phpunit PHP='phpdbg -qrr' E2E_PHPUNIT_GROUP=e2e BENCHMARK_SOURCES= | ||
|
There was a problem hiding this comment. It just wouldn't work with Xdebug. It seems installed, but then it is not. |
||
|
|
||
| - name: Install Linux dependencies for E2E tests | ||
| if: runner.os != 'Windows' | ||
| run: | | ||
| sudo apt-get install -y --no-install-recommends expect | ||
|
There was a problem hiding this comment. Getting Expect on Windows, is, well, complicated. This could be worked around with WSL, but it'll make Windows VM "unpure": we can't expect everyone using Infection with Windows with WSL. |
||
|
|
||
| - name: Run the whole set of E2E tests | ||
| if: runner.os != 'Windows' | ||
| env: | ||
| TERM: xterm-256color | ||
| run: | | ||
| make test-e2e E2E_PHPUNIT_GROUP=e2e INFECTION=bin/infection BENCHMARK_SOURCES= | ||
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # yamllint disable rule:line-length | ||
| # yamllint disable rule:braces | ||
|
|
||
| name: Mutation Testing | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - main | ||
| - master | ||
|
|
||
| env: | ||
| MIN_MSI: 68.00 | ||
| MIN_COVERED_MSI: 80.00 | ||
|
|
||
| jobs: | ||
| tests: | ||
| runs-on: ${{ matrix.operating-system }} | ||
|
|
||
| strategy: | ||
| matrix: | ||
| operating-system: [ubuntu-latest, windows-latest] | ||
| php-version: ['7.4'] | ||
| dependencies: [''] | ||
| coverage-driver: [pcov] | ||
|
|
||
| name: Mutation testing on ${{ matrix.operating-system }} with PHP ${{ matrix.php-version }}, using ${{ matrix.coverage-driver }} | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Setup PHP | ||
| uses: shivammathur/setup-php@v2 | ||
| with: | ||
| php-version: ${{ matrix.php-version }} | ||
| coverage: ${{ matrix.coverage-driver }} | ||
| tools: composer:v2 | ||
|
|
||
| - name: Get composer cache directory | ||
| id: composer-cache | ||
| run: echo "::set-output name=dir::$(composer config cache-files-dir)" | ||
|
|
||
| - name: Cache dependencies | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ steps.composer-cache.outputs.dir }} | ||
| key: composer-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('composer.*') }}-${{ matrix.dependencies }} | ||
| restore-keys: | | ||
| composer-${{ runner.os }}-${{ matrix.php-version }}-${{ hashFiles('composer.*') }}- | ||
| composer-${{ runner.os }}-${{ matrix.php-version }}- | ||
| composer-${{ runner.os }}- | ||
| composer- | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| composer update --no-interaction --prefer-dist --no-progress ${{ matrix.dependencies }} | ||
|
|
||
| - name: Collect coverage report | ||
| run: | | ||
| php vendor/phpunit/phpunit/phpunit --stop-on-failure --coverage-xml=build/logs/coverage-xml --log-junit=build/logs/junit.xml | ||
|
|
||
| - name: Run Infection | ||
| run: | | ||
| php bin/infection -j2 --skip-initial-tests --min-msi=$MIN_MSI --min-covered-msi=$MIN_COVERED_MSI --coverage=build/logs --log-verbosity=none --skip-initial-tests --no-interaction --no-progress |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,55 +1,2 @@ | ||
| build: false | ||
| clone_depth: 1 | ||
|
There was a problem hiding this comment. why do we need to keep this file? There was a problem hiding this comment. AppVeyor will fail the build without it. There was a problem hiding this comment. so will we remove once 0.18 is not maintained anymore? There was a problem hiding this comment. Yes, that's the idea. Remove it once we don't need it anywhere. |
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,7 +31,7 @@ | |
|
|
||
| <groups> | ||
| <exclude> | ||
| <group>e2e</group> | ||
| </exclude> | ||
| </groups> | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| if [ "$DRIVER" = "pcov" ] | ||
| then | ||
| # `pcov` requires at least PHPUnit 8.0 (used by symfony/phpunit-bridge) | ||
| export SYMFONY_PHPUNIT_VERSION="8.0" | ||
| fi | ||
|
|
||
| set -e | ||
|
|
||
| run () { | ||
| local INFECTION=${1} | ||
| local PHPARGS=${2} | ||
|
|
||
| if [ "$DRIVER" = "phpdbg" ] | ||
| then | ||
| phpdbg $PHPARGS -qrr $INFECTION | ||
| else | ||
| php $PHPARGS $INFECTION | ||
| fi | ||
|
|
||
| diff -u -w expected-output.txt infection.log | ||
| } | ||
|
There was a problem hiding this comment. SymfonyFlex needs a special environment variable, and it uses old version of PHPUnit which is not compatible with the one we use, so we can't really run this test from |
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only need to consider two files here, no others in
tests/e2eor otherwise.