Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: run ci build related tests in parallel #1907

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

ci: run ci build related tests in parallel #1907

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
550610d
test: run ci tests in parallel
mattcosta7 Dec 4, 2023
ecb9393
ci: cache lib and pnpm
mattcosta7 Dec 4, 2023
ce154c0
ci: cache lib and pnpm
mattcosta7 Dec 4, 2023
4a314ca
ci: use setup-node to cache pnpm
mattcosta7 Dec 4, 2023
2160d87
ci: use setup-node to cache pnpm
mattcosta7 Dec 4, 2023
cc3f3f0
ci: playwright
mattcosta7 Dec 4, 2023
216997d
Merge branch 'main' into run-test-variants-in-parallel
kettanaito Dec 16, 2023
d402ddb
Update ci.yml
mattcosta7 Dec 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
142 changes: 134 additions & 8 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,45 +7,171 @@ on:
workflow_dispatch:

jobs:
# Builds the library and persists it as an artifact.
# Later jobs can then reuse that build artifact, skipping
# the build step.
build:
name: Build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Install dependencies
run: pnpm install

- name: Build
run: pnpm build

- name: Cache build output
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

test_unit:
mattcosta7 marked this conversation as resolved.
Show resolved Hide resolved
name: Unit tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Unit tests
run: pnpm test:unit

- name: Build
run: pnpm build
test_node:
name: Node tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Node.js tests
run: pnpm test:node

test_browser:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth noting that ~85% of the test runtime is occupied by Playwright tests. We have around 200 tests there, and GitHub machines only has 2 cores, which means we are running all of them sequentially. This is the main bottleneck of CI performance for us right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, totally.
In some cases though, being able to re-run the playwright tests a few times would be nice even if that was 7minutes each, without needing to rebuild, and run the other suites first. 3 minutes each x 5 runs is 15 minutes saved wall clock

name: Browser tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Playwright install
run: pnpm exec playwright install --with-deps chromium

- name: Browser tests
run: pnpm test:browser

- name: Native tests
run: pnpm test:native

- name: Upload test artifacts
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: test/browser/test-results

test_native:
name: Native tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Native tests
run: pnpm test:native