Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move execution reordering to main iterator #4302

Open
6 of 19 tasks
epdenouden opened this issue Jun 17, 2020 · 0 comments
Open
6 of 19 tasks

Move execution reordering to main iterator #4302

epdenouden opened this issue Jun 17, 2020 · 0 comments
Assignees
Labels
feature/test-runner CLI test runner type/refactoring A refactoring that should be applied to make the code easier to understand and maintain

Comments

@epdenouden
Copy link
Contributor

epdenouden commented Jun 17, 2020

Move all existing test execution reordering functionality to the code that answers the question what test shall we run next?

This refactoring is a continuation of the work for the recent @depends improvements and does not change anything for the end-users. The deliverable is a small pull request with @internal clean up.

Why

Folding the existing reordering mechanism into the core iterator and cleaning up the supporting code has tangible benefits:

  • Reordering currently only happens on startup, giving the sorter a limited view. This is closely related to how @dataProvider tests are loaded.
  • The TestSuiteSorter sorts the internal tests graph which can have side-effects. In contrast the extended iterator will simply loop over the existing tree in a different order. This makes it more flexible and robust.
  • Lay the groundwork for a clean implementation of on-demand @dataProvider tests with proper rewind/next hooks.
  • The refreshed code is easier to understand and maintain. The TestSuiteSorter has too much knowledge of the runner internals.

To do

  • create an additional minimal interface for tests that implement the requirements for the PHPUnit @depends-resolver
    • implement ReorderableTest (name suggestions welcome!)
    • implement getInternalId() for all PHPUnit test objects
    • further improve provides-requires dependency resolver for all reordarable test types
  • use TestSuiteIterator for all test iteration
    • extend iterator to keep to-do/done lists instead of just looping over a list
    • sort a TestSuite during Iterator::rewind()
    • add dependency checks to Iterator::next()
    • inject the result cache (see also: --filter mechanism)
    • inject test ordering options
  • retire TestSuiteSorter as the main reordering mechanism
    • keep TestSuiteSorter comparators for sorting individual TestSuites
    • move test graph traversal for calculation original execution order utility function fits here
    • retire internal getSorterUID in favour of new interface
    • give unit tests for TestSuite[Iterator|Sorter] a good wash
  • update internal test naming scheme in result cache
  • get code to LTS release quality
    • full feature coverage
    • project coverage, especially remove obsolete reordering guards
@epdenouden epdenouden added type/refactoring A refactoring that should be applied to make the code easier to understand and maintain status/waiting-for-feedback Waiting for feedback from original reporter feature/test-runner CLI test runner labels Jun 17, 2020
@epdenouden epdenouden added this to the PHPUnit 9.3 milestone Jun 17, 2020
@epdenouden epdenouden self-assigned this Jun 17, 2020
@epdenouden epdenouden removed the status/waiting-for-feedback Waiting for feedback from original reporter label Jun 18, 2020
@sebastianbergmann sebastianbergmann removed this from the PHPUnit 9.4 milestone Sep 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/test-runner CLI test runner type/refactoring A refactoring that should be applied to make the code easier to understand and maintain
Projects
None yet
Development

No branches or pull requests

2 participants