Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SUREFIRE-2055] Always show random seed #505

Merged
merged 2 commits into from May 4, 2022
Merged

[SUREFIRE-2055] Always show random seed #505

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1e9db1f
[SUREFIRE-2055] Always show random seed
delanym Apr 3, 2022
efebb73
Avoid NPE in DefaultRunOrderCalculator
slawekjaranowski Apr 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 5 additions & 2 deletions ...-surefire-common/src/main/java/org/apache/maven/plugin/surefire/AbstractSurefireMojo.java
View file
Open in desktop
Expand Up @@ -3114,9 +3114,12 @@ protected void warnIfIllegalFailOnFlakeCount() throws MojoFailureException

private void printDefaultSeedIfNecessary()
{
if ( getRunOrderRandomSeed() == null && getRunOrder().equals( RunOrder.RANDOM.name() ) )
if ( getRunOrder().equals( RunOrder.RANDOM.name() ) )
{
setRunOrderRandomSeed( System.nanoTime() );
slawekjaranowski marked this conversation as resolved.
Show resolved Hide resolved
if ( getRunOrderRandomSeed() == null )
{
setRunOrderRandomSeed( System.nanoTime() );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I would expect some warning that the seed is missing.
If random, then yes, we should use random and prevent from any cohesion between the neighbouring runs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see the implementation, because the impl has much better opportunities to use real random number if this parameter is NULL.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runOrderRandomSeed is never NULLm if is not provided it will be set to nanoTime.

Currently is printed only when is not given in configuration, change introduce to print always - also if is given by configuration.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slawekjaranowski runOrder can be other than random and so DefaultRunOrderCalculator may obtain null seed.
@delanym see my comment #505 (comment)

}
getConsoleLogger().info(
"Tests will run in random order. To reproduce ordering use flag -D"
+ getPluginName() + ".runOrder.random.seed=" + getRunOrderRandomSeed() );
Expand Down
2 changes: 1 addition & 1 deletion surefire-api/src/main/java/org/apache/maven/surefire/api/util/DefaultRunOrderCalculator.java
View file
Open in desktop
Expand Up @@ -55,7 +55,7 @@ public DefaultRunOrderCalculator( RunOrderParameters runOrderParameters, int thr
this.runOrder = runOrderParameters.getRunOrder();
this.sortOrder = this.runOrder.length > 0 ? getSortOrderComparator( this.runOrder[0] ) : null;
Long runOrderRandomSeed = runOrderParameters.getRunOrderRandomSeed();
this.random = new Random( runOrderRandomSeed == null ? System.nanoTime() : runOrderRandomSeed );
random = new Random( runOrderRandomSeed == null ? System.nanoTime() : runOrderRandomSeed );
}

@Override
Expand Down
26 changes: 25 additions & 1 deletion surefire-its/src/test/java/org/apache/maven/surefire/its/RunOrderIT.java
View file
Open in desktop
Expand Up @@ -97,7 +97,21 @@ public void testRandomJUnit4SameSeed()
}
}
}


@Test
public void testRandomJUnit4PrintSeedWithGivenSeed()
{
OutputValidator validator = executeWithRandomOrder( "junit4", 0L );
validator.verifyTextInLog( "To reproduce ordering use flag" );
}

@Test
public void testRandomJUnit4PrintSeedWithNoGivenSeed()
{
OutputValidator validator = executeWithRandomOrder( "junit4" );
validator.verifyTextInLog( "To reproduce ordering use flag" );
}

@Test
public void testReverseAlphabeticalJUnit4()
throws Exception
Expand Down Expand Up @@ -187,6 +201,16 @@ private OutputValidator executeWithRunOrder( String runOrder, String profile )
.verifyErrorFree( 3 );
}

private OutputValidator executeWithRandomOrder( String profile )
{
return unpack()
.activateProfile( profile )
.forkMode( getForkMode() )
.runOrder( "random" )
.executeTest()
.verifyErrorFree( 3 );
}

private OutputValidator executeWithRandomOrder( String profile, long seed )
{
return unpack()
Expand Down