Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Galata in JupyterLab #10796

Merged
merged 10 commits into from Aug 28, 2021
Merged

Add Galata in JupyterLab #10796

merged 10 commits into from Aug 28, 2021

Conversation

fcollonval
Copy link
Member

@fcollonval fcollonval commented Aug 11, 2021 •
edited

References

Fixes jupyterlab/galata#67
Fixes #10480

Features:

  • mock state (user can set an initial state)
  • mock settings (user can set initial values)
  • create unique folder for each test on the server
  • automatically go to JupyterLab page, hook Galata Inpage code and wait for the app to be started
  • Stop terminals and sessions created during a test
  • Playwright HTML report:

playwright_html_report

Code changes

⚠️ @types/node was frozen to ^14.6.1 in package.json to resolve typedoc error related to jest types

This imports galata as a first party package in JupyterLab core to ensure the helpers are in sync with JupyterLab packages.

This also convert galata from a tool to a set of fixtures and helpers on top of Playwright Test Runner - this will allow to catch up quickly all the nice features already and coming into Playwright.

It changes also the logic to fit test framework aka isolation of test functions to be able to select a file or a function only as proposed by Playwright Test Runner. To allow that the state and the settings are mocked in memory to avoid changes on the hard drive. The sessions and terminals created during a test are automatically deleted. A unique temporary path can be created for a test to be use on the server to ensure files are not shared between tests.

For now the HTML report has been removed, it will be restored at the next Playwright release that will include a default HTML report. HTML report is available in 1.14.0. This turn that feature on (although it is not documented - so probably not considered as stable; I encountered crash by time to time).

For now using multiple workers does not work (I did not get why but when using multiple workers some tests are even not started?!). It seems than when using multiple workers, if a test failed it will result in additional test failures.

Tasks:

User-facing changes

Packages using galata will need to update their tests to use Playwright Test Runner directly

Backwards-incompatible changes

Galata is no more a tool but a set of helpers and fixtures for Playwright Test Runner.

@jupyterlab-dev-mode
Copy link

Thanks for making a pull request to JupyterLab!

To try out this branch on binder, follow this link: Binder

@github-actions github-actions bot added Design System CSS pkg:extensionmanager tag:CSS For general CSS related issues and pecadilloes tag:Examples labels Aug 11, 2021
@blink1073 blink1073 added this to the 4.0 milestone Aug 11, 2021
@afshin afshin mentioned this pull request Aug 11, 2021
Closed
@fcollonval fcollonval mentioned this pull request Aug 12, 2021
Closed
@mbektas
Copy link
Member

mbektas commented Aug 18, 2021

thanks @fcollonval! looks very interesting! I will take a detailed look.

@fcollonval fcollonval mentioned this pull request Aug 18, 2021
Closed
@blink1073
Copy link
Member

Huge thanks for working on this @fcollonval, super exciting!

@mbektas
Copy link
Member

mbektas commented Aug 21, 2021

@fcollonval I had chance to take a deeper look. The enhancements you implemented such as mocking state & settings and proper cleanup along with overall refactoring are really great and exciting!

I checked also from the point of backward compatibility. With these changes, CLI is going away, as far as I understand, along with some features such as connecting to remote browser instances, debugging helpers like --no-headless, --skip-visual-regression. Should we consider bringing back CLI and implement the configuration options provided by those?

@fcollonval
Copy link
Member Author

fcollonval commented Aug 24, 2021 •
edited

With these changes, CLI is going away, as far as I understand,

Yes

along with some features such as

Feature Status Question
connecting to remote browser instances No idea but it is not gonna be easy
Look like it could be possible with fixture
--no-headless Playwright --headed option
--skip-visual-regression What is the user story for this?

Should we consider bringing back CLI and implement the configuration options provided by those?

I would prefer as much as possible to use the playwright CLI as they are moving forward faster than us. So it will be easier if we don't proxy it; especially as they have other tools like playwright codegen or playwright open.

@fcollonval fcollonval marked this pull request as ready for review August 25, 2021 14:40
@fcollonval
Initial import of galata
7298e68 
Homogeneous version

Fix tests

WIP drop jest

WIP playwright

First working version

WIP mock state and settings

Use node-fetch instead of axios

Add temporary server test folder fixture

Remove html report assets

Track sessions and terminals related to a test

Finalize notebook tests

Set up CI

Don't use a single worker by default

Delete data.csv

Increase timeout to wait for JLab in dev-mode

Build the galata assets before the tests

Install chromium

Upload galata artifacts

Bump playwright test runner

enhance settings fixture

Correct tests

Fix integrity

Split assests into two artifacts

Fix regex and settings setting

Fix mockSettings test

WIP Convert contextmenu tests

galata refactoring

All contents helpers use the REST API

Correct linter ignore files

Correct contextmenu test

Migrate debugger tests

Start converting general tests

Convert general tests

convert notebook-run

Improve temporary server folder name

Convert toc tests

Convert notebook create

Convert notebook-edit

Convert notebook-toolbar

Improve test:report
Open browser
Don't expose local port

Fix galata tests

Improve test partitioning for sessions and terminals

Use Playwright project feat. to merge galata and ui-tests

Update test using project structure

Fix jupyterlab/galata#69

Upgrade newly added tests

Add install

Fix build inpage

Fix tests

More fix

Try to fix typedoc

Update reference screenshots

Use galata.newContentsHelper

More robust `page.waitIsReady`

Fix types for typedoc

Revert change for integrity testing

Auto review
@fcollonval
Don't use docker to speed up tests
89f1cf8
@jtpio
Copy link
Member

jtpio commented Aug 26, 2021 •
edited

This should also fix #10480

@fcollonval
Fix linter
24f5b62 
More robust test
@jtpio
Update galata/package.json
746b308 
Co-authored-by: Jeremy Tuloup <jeremy.tuloup@gmail.com>
jtpio
jtpio approved these changes Aug 28, 2021
View reviewed changes
Copy link
Member

@jtpio jtpio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@blink1073 blink1073 merged commit 0b6f1d7 into jupyterlab:master Aug 28, 2021
@blink1073 blink1073 mentioned this pull request Aug 28, 2021
Closed
@fcollonval fcollonval deleted the ft/add-galata branch August 29, 2021 07:36
This was referenced Aug 29, 2021
Merged
Merged
@krassowski krassowski mentioned this pull request Aug 29, 2021
Closed
3 tasks
@fcollonval fcollonval modified the milestones: 4.0, 3.2.x Aug 30, 2021
This was referenced Aug 30, 2021
Closed
Closed
Closed
Closed
Merged
@mbektas mbektas mentioned this pull request Sep 1, 2021
Closed
@jtpio jtpio mentioned this pull request Sep 2, 2021
Closed
This was referenced Sep 9, 2021
Closed
Closed
@blink1073
Copy link
Member

@meeseeksdev please backport to 3.2.x

@lumberbot-app
Copy link

lumberbot-app bot commented Sep 23, 2021

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
$ git checkout 3.2.x
$ git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
$ git cherry-pick -m1 0b6f1d76e551e29dc02235da4335914648d35e52
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
$ git commit -am 'Backport PR #10796: Add Galata in JupyterLab'
  1. Push to a named branch :
git push YOURFORK 3.2.x:auto-backport-of-pr-10796-on-3.2.x
  1. Create a PR against branch 3.2.x, I would have named this PR:

"Backport PR #10796 on branch 3.2.x (Add Galata in JupyterLab)"

And apply the correct labels and milestones.

Congratulation you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove Still Needs Manual Backport label once the PR gets merged.

If these instruction are inaccurate, feel free to suggest an improvement.

@blink1073
Copy link
Member

@fcollonval I'll leave the Galata backporting to you ;)

fcollonval added a commit to fcollonval/jupyterlab that referenced this pull request Sep 29, 2021
@fcollonval
Backport PR jupyterlab#10796: Add Galata in JupyterLab
a4c8547
@fcollonval
Copy link
Member Author

Backport PR #11179 has been opened

fcollonval added a commit to fcollonval/jupyterlab that referenced this pull request Sep 29, 2021
@fcollonval
Backport PR jupyterlab#10796: Add Galata in JupyterLab
0420ad2
blink1073 pushed a commit that referenced this pull request Sep 30, 2021
@fcollonval
Backport PR #10796 to 3.2.x: Add Galata in JupyterLab (#11179)
18f953f 
* Backport PR #10796: Add Galata in JupyterLab

* Update @jupyterlab/galata version

* Correct galata repository urls

* Undo changes to notebook example

* Address CI errors

* Fix `vega-lite` error

* Upgrade vega-lite to v5

* Upgrade screenshots

* Fix check release
@github-actions github-actions bot added the status:resolved-locked Closed issues are locked after 30 days inactivity. Please open a new issue for related discussion. label Mar 29, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jtpio jtpio jtpio approved these changes

@blink1073 blink1073 blink1073 approved these changes

@mbektas mbektas Awaiting requested review from mbektas

Assignees

@fcollonval fcollonval

Labels
Design System CSS maintenance pkg:celltags pkg:coreutils pkg:extensionmanager pkg:services status:resolved-locked Closed issues are locked after 30 days inactivity. Please open a new issue for related discussion. tag:CSS For general CSS related issues and pecadilloes tag:Performance tag:Testing
Projects
None yet
Milestone
3.2.x
Development

Successfully merging this pull request may close these issues.

From galata using playwright to playwright with galata Move Galata to the jupyterlab repo?
5 participants
@fcollonval @mbektas @blink1073 @jtpio @afshin