Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests runorder #492

Closed
wants to merge 3 commits into from
Closed

Conversation

aslakhellesoy
Copy link

Following this checklist to help us incorporate your
contribution quickly and easily:

  • Make sure there is a JIRA issue filed
    for the change (usually before you start working on it). Trivial changes like typos do not
    require a JIRA issue. Your pull request should address just this issue, without
    pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
    where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
    is to use the JIRA issue title in the pull request title and in the first line of the
    commit message.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean install to make sure basic checks pass. A more thorough check will
    be performed on your pull request automatically.
  • You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.


This is a proof of concept of an alternative implementation to #348 (currently only for JUnit 4).

I considered the suggestions by @Tibor17 to do serialization/deserialization of the run config, but from what I can tell this is already handled. I can access the -Dtest= value in testResolver.getIncludedPatterns().

I've added two tests. The one that is ignored illustrates what I would like to achieve: Run test methods in a specific order without having to group the executions by class. Any suggestions how we might achieve that?

A bit of context - I'm working on a test case prioritization tool that may suggest ordering test methods in any order. I believe you're also working on something like this @winglam?

@@ -270,18 +276,57 @@ private void executeTestSet( Class<?> clazz, RunListener reporter, Notifier noti
}
}

private static int indexOf( List<ResolvedTest> tests, Description description )
Copy link
Contributor

@Tibor17 Tibor17 Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do something like this, you are on wrong way.
The problem is that you wanted to get to the end within one day, and that's the problem.
You should follow those 5 steps, you would not get to the end after you have implemented first 4 steps but would would do 85% of your work without removing anything.
If you are not sure in the beginning, it is still right way to write an integration test because it is always needed and it is the vision you want to emphase to everyone.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will close this PR and send another one. I've also written up a JIRA issue explaining a bit more about the motivation: https://issues.apache.org/jira/browse/SUREFIRE-2041

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see #495 for my take II on this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants