Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests with Selenium worker again #1402

Open
borekb opened this issue Mar 16, 2019 · 3 comments
Open

Run tests with Selenium worker again #1402

borekb opened this issue Mar 16, 2019 · 3 comments
Assignees
Labels
help wanted Good issue for community contribution – no need for deep knowledge of VP internals noteworthy Significant issue or PR, to be highlighted in release notes scope: tests Testing code. For infrastructure (CI, etc.), use "dev-infrastructure".
Milestone

Comments

@borekb
Copy link
Member

borekb commented Mar 16, 2019

We currently only run end2end tests with the WP-CLI driver, see here, and the other Selenium tests have been disabled in dc531ed (PR #1329) as it has been challenging to make them work. It has been postponed for post-beta work in #1284 but it's worth a separate issue so I'm creating this one.

The PR #1329 mentions this around Selenium tests:

Make Selenium tests work again. I spent some time on them in this PR but they will require more love. Quick thoughts at this point:

  • The actual test code (navigating URLs, clicking on things) still looks fine to me but the setup code is slightly suspicious, e.g., here or here.
  • facebook/webdriver seems to be more popular than giorgiosironi/phpunit-selenium (Selenium2TestCase), I've also heard good things about Codeception. As long-term maintainability of our Selenium tests is an important goal, considering a new framework is something we should do.

Also related is the ability to switch WP-CLI and Selenium workers more dynamically than via test-config.yml – we don't have a separate issue for that yet but it's something I'd like to do as part of #1389 or shortly after that.

@borekb borekb added the scope: tests Testing code. For infrastructure (CI, etc.), use "dev-infrastructure". label Mar 16, 2019
@borekb borekb added this to the 4.0 milestone Mar 16, 2019
@borekb borekb self-assigned this Mar 16, 2019
@borekb borekb added the noteworthy Significant issue or PR, to be highlighted in release notes label Mar 27, 2019
@borekb
Copy link
Member Author

borekb commented Apr 23, 2019

There's an interesting project called wp-browser that we could consider using.

@borekb borekb added the help wanted Good issue for community contribution – no need for deep knowledge of VP internals label Apr 27, 2019
@borekb
Copy link
Member Author

borekb commented May 19, 2019

I'm now convinced that we should probably re-work our Selenium tests to Codeception + wp-browser:

My other thought was to drop Selenium tests entirely but there are scenarios where we still need it, mostly:

  • All the tests/Selenium tests, for example, testing that we install .htaccess rules correctly.
  • Certain End2End test scenarios cannot be tested via WP-CLI – search for PHPUnit_Framework_SkippedTestError inside *WpCliWorker.php files.

@borekb
Copy link
Member Author

borekb commented Jun 27, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Good issue for community contribution – no need for deep knowledge of VP internals noteworthy Significant issue or PR, to be highlighted in release notes scope: tests Testing code. For infrastructure (CI, etc.), use "dev-infrastructure".
Projects
4.0-beta2 → 4.0-beta3
  
Awaiting triage
Development

No branches or pull requests

1 participant