Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polish documentation, particularly Stopwatch #643

Merged
merged 8 commits into from May 27, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/default-locale-timezone.adoc
Expand Up @@ -42,7 +42,7 @@ NOTE: A class-level configuration means that the specified locale is set before

== `@DefaultTimeZone`

The default `TimeZone` is specified according to the https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html#getTimeZone(java.lang.String)[TimeZone.getTimeZone(String)] method.
The default `TimeZone` is specified according to the https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html#getTimeZone-java.lang.String-[TimeZone.getTimeZone(String)] method.

[source,java,indent=0]
----
Expand Down
4 changes: 4 additions & 0 deletions docs/disable-if-test-fails.adoc
Expand Up @@ -62,3 +62,7 @@ That means if...
* a test in `SpecificTests` fails

... then, only remaining tests in `SpecificTests` are disabled and other implementations of `Tests` remain unaffected, i.e. their tests are not disabled.

== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution], but no guarantee can be given how soon after a failed test, other tests will be disabled, i.e. how soon other threads observe that "their" tests are supposed to be disabled.
6 changes: 4 additions & 2 deletions docs/disable-parameterized-tests.adoc
Expand Up @@ -9,7 +9,6 @@ These are as follows:
- DisableIfDisplayName
- DisableIfArgument


== DisableIfDisplayName

The `@DisableIfDisplayName` annotation can be used to selectively disable parameterized tests based on their display names, which are dynamically registered on runtime.
Expand Down Expand Up @@ -39,7 +38,6 @@ include::{demo}[tag=disable_parameterized_regex]
NOTE: Since JUnit Pioneer 1.5.0, using both `matches` and `contains` in a single annotation is no longer permitted.
The reason is that it's not clear from reading the annotation whether it's *and* or *or* semantics, i.e. whether the display name needs to both match the regex *and* contain the substring to be disabled or whether fulfilling one criterion suffices.


== DisableIfArgument

This extension can be used to selectively disable parameterized tests based on their arguments (converted with `toString()`).
Expand Down Expand Up @@ -177,3 +175,7 @@ Just like with `contains`, if any argument matches any expression from `matches`
NOTE: While the documentation uses `String` values for demonstration purposes, you can use it to disable tests with other parameter types.
However, the arguments will be converted to `String` with `Object#toString()` before evaluation.
Make sure that your parameter types have a meaningful `toString` method.

== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
6 changes: 4 additions & 2 deletions docs/disabled-until.adoc
Expand Up @@ -3,8 +3,6 @@
:xp-demo-dir: ../src/demo/java
:demo: {xp-demo-dir}/org/junitpioneer/jupiter/DisableUntilExtensionDemo.java

== Introduction

It's sometimes useful to disable a test.
One can imagine many reasons for doing so, maybe a remote dependency of the test is broken or not yet deployed.
Maybe the test is still in development and unstable.
Expand Down Expand Up @@ -47,3 +45,7 @@ The `@DisabledUntil` annotation can only be used once per class or method.

The test will be skipped only if the date specified by `date` is the future.
If `date` is today or in the past, the test will be executed normally but a warning entry will be published to the https://junit-pioneer.org/docs/report-entries[test report].

== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
6 changes: 4 additions & 2 deletions docs/issue.adoc
Expand Up @@ -3,8 +3,6 @@
:xp-demo-dir: ../src/demo/java
:demo: {xp-demo-dir}/org/junitpioneer/jupiter/IssueExtensionDemo.java

== Introduction

The main reason developers write tests is to ensure the functionality of a requirement or to avoid technical problems.
The `@Issue` annotation allows marking tests with a `String`, referencing a related issue (like a requirement, or a bugfix) of an issue-tracker (like Jira or Redmine).

Expand Down Expand Up @@ -51,3 +49,7 @@ include::{demo}[tag=issue_processor_sample]

NOTE: The implementing class must be registered to the Java ecosystem as a service.
For further information about registering a service, please see the Java https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html[`ServiceLoader` documentation].

== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
1 change: 0 additions & 1 deletion docs/json-argument-source.adoc
Expand Up @@ -197,7 +197,6 @@ That means (1) it must be on the module path and (2) it must be resolved.
The steps above ensure that your build tool knows about the parser and should accomplish (1), but if no other module depends on the parser (directly or indirectly), (2) requires additional work.
In that case, you need to manually resolve the module by applying the command line option `--add-modules=com.fasterxml.jackson.databind` to the Java process that executes the tests.


== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
2 changes: 0 additions & 2 deletions docs/project-page.adoc
Expand Up @@ -9,7 +9,6 @@ excerpt: 'JUnit 5 extension pack, pushing the frontiers on Jupiter.<br/> <small>
JUnit Pioneer is an https://nipafx.dev/junit-5-extension-model/[extension pack] for https://junit.org/junit5/[JUnit 5] or, https://nipafx.dev/junit-5-architecture-jupiter/[to be more precise], for the Jupiter engine.
It offers https://junit-pioneer.org/docs/[a wide variety of extensions] and is continuously released with more.


== Quick start

Determine the latest version (e.g. on https://search.maven.org/artifact/org.junit-pioneer/junit-pioneer[Maven Central]) and add Pioneer as a test dependency.
Expand All @@ -35,7 +34,6 @@ testCompile group: 'org.junit-pioneer', name: 'junit-pioneer', version: /* ... *

Done, you're good to go! 👍


== Contributing

There are various ways to help us improve JUnit Pioneer if you're interested:
Expand Down
1 change: 0 additions & 1 deletion docs/report-entries.adoc
Expand Up @@ -108,7 +108,6 @@ Then the extension will publish `"21 - Hello - 21"` and `"42 - World - 42"`.

Accessing test parameters in the key of the report entry is unsupported.


== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
7 changes: 3 additions & 4 deletions docs/standard-input-output.adoc
Expand Up @@ -13,12 +13,13 @@ This becomes particularly important when running tests in parallel, where other
The extension consists of two parts:

* The annotation `@StdIo`. It allows defining input that is read from `System.in` without having to wait for user input.
* Parameters `StdIn` and `StdOut`, which you can have injected into your test. Their `capturedLines()` methods allow you to access lines read from `System.in` or written to `System.out`, so you can verify them with common assertions.
* Parameters `StdIn` and `StdOut`, which you can have injected into your test.
Their `capturedLines()` methods allow you to access lines read from `System.in` or written to `System.out`, so you can verify them with common assertions.

For example, after calling `System.out.println("Hello")` and `System.out.println("World")`, the `StdOut::capturedLines` method would return an array ["Hello", "World"].
With `System.out.print("Hello")` and `System.out.println("World")` (note that the first method does not print a line break), it would return `["HelloWorld"]`.

Here's which combinations of this annotation (with or without values for the read lines) and parameters are valid:
Here are the valid combinations of the annotation (with or without values for the read lines) and parameters:

`@StdIo("...")`::
In this case `System.in` gets replaced and the code under test will read the specified lines (in the snippet, that's just the line `"..."`).
Expand Down Expand Up @@ -62,7 +63,6 @@ include::{demo}[tag=stdio_both_replaced_and_verify]

The remaining combinations of the annotation, its values, and `StdIn`/`StdOut` are considered misconfigurations and lead to exceptions.


== Thread-Safety

Since `System.in` and `System.out` are global state, reading and writing them during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution] can lead to unpredictable results and flaky tests.
Expand All @@ -77,7 +77,6 @@ Tests that cover code that reads or writes `System.in` or `System.out` need to b

Tests annotated in this way will never execute in parallel with tests annotated with `@StdIo`.


== Edge cases and unexpected behavior

=== Empty input (with or without `StdIn`)
Expand Down
36 changes: 29 additions & 7 deletions docs/stopwatch.adoc
Expand Up @@ -3,18 +3,40 @@
:xp-demo-dir: ../src/demo/java
:demo: {xp-demo-dir}/org/junitpioneer/jupiter/StopwatchExtensionDemo.java

== Introduction

Test shall be fast.
To get a feeling how fast or slow a specific test is, the `@Stopwatch` annotation can be used.
It will measure elapsed time and pass the result to the `https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/TestReporter.html[TestReporter]`, which will publish it.
Annotating a test with `@Stopwatch` will measure the time it takes to execute and report the result to the https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/TestReporter.html[`TestReporter`].
How that information is displayed depends on the tool used to run the tests and how it processes test report entries.

== Usage

To measure elapsed time just add the `@Stopwatch` annotation to a class or, like in the following example, to a method:
`@Stopwatch` can be applied to a method to report its execution time:

[source,java,indent=0]
----
include::{demo}[tag=method]
----

It can also be applied to a class, in which case it reports the execution times of each test as if it were applied to each method individually:

[source,java,indent=0]
----
include::{demo}[tag=stopwatch_demo]
include::{demo}[tag=class]
----

It can also be applied to a class and a method therein but since a class-level annotation already works as if each method was annotated, the method-level annotations would be redundant.

== Output

This is how IntelliJ displays a report entry (in the Run/Debug panel):

----
timestamp = 2022-05-26T12:16:14.021646, StopwatchExtension = Execution of 'test()' took [11] ms.
----

Other tools may or may not print report entries.

The most reliable way to gather entries is to create a https://junit.org/junit5/docs/current/user-guide/#launcher-api-listeners-custom[`TestExecutionListener`] and register it with the launcher.
https://junit.org/junit5/docs/current/api/org.junit.platform.engine/org/junit/platform/engine/reporting/ReportEntry.html[Report entries] from the stopwatch extension can be identified by checking their https://junit.org/junit5/docs/current/api/org.junit.platform.engine/org/junit/platform/engine/reporting/ReportEntry.html#getKeyValuePairs()[key-value pairs] - at least one key will start with the string `"StopwatchExtension"`.

== Thread-Safety

This extension is safe to use during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
1 change: 0 additions & 1 deletion docs/temp-directory.adoc
Expand Up @@ -58,4 +58,3 @@ class MyTests {
== Thread-Safety

This extension's thread-safety is undetermined, we recommend to be cautious when using it during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution].
(We're working on it.)
4 changes: 2 additions & 2 deletions docs/vintage-test.adoc
Expand Up @@ -6,7 +6,7 @@
The annotation `org.junitpioneer.vintage.@Test` is a drop-in replacement for https://junit.org/junit4/javadoc/4.12/org/junit/Test.html[JUnit 4's `org.junit.@Test` annotation], but marks the method as a regular JUnit Jupiter test.

You can make use of it when migrating tests from JUnit 4 to Jupiter by doing a fulltext search/replace of `import org.junit.Test` with `import org.junitpioneer.vintage.Test`.
That means you do not need to run JUnit 5's Vintage engine to execute such tests--Jupiter suffices.
That means you do not need to run JUnit 5's Vintage engine to execute such tests - Jupiter suffices.

This should be seen as an intermediate step towards a full migration and should be followed up by gradual/manual replacement of `org.junitpioneer.vintage.Test` with Jupiter's `org.junit.jupiter.api.Test`.
To emphasize its character as a temporary solution and to reduce https://github.com/junit-pioneer/junit-pioneer/issues/137[risk of accidental use], it's marked as deprecated.
Expand Down Expand Up @@ -55,7 +55,7 @@ include::{demo}[tag=vintage_test_timeout]
----

Like in JUnit 4, `timeout` aborts long-running tests.
The following tests does not run indefinitely--it failes after 100 milliseconds:
The following test does not run indefinitely - it fails after 100 milliseconds:

[source,java,indent=0]
----
Expand Down
Expand Up @@ -15,16 +15,16 @@
public class DisableUntilExtensionDemo {

// tag::disable_until_simple[]
@DisabledUntil(date = "2022-01-01")
@Test
@DisabledUntil(date = "2022-01-01")
void test() {
// Test will be skipped if it's 2021-12-31 or earlier
}
// end::disable_until_simple[]

// tag::disable_until_with_reason[]
@DisabledUntil(reason = "The remote server won't be ready until next year", date = "2022-01-01")
@Test
@DisabledUntil(reason = "The remote server won't be ready until next year", date = "2022-01-01")
void testWithReason() {
// Test will be skipped if it's 2021-12-31 or earlier
}
Expand Down
Expand Up @@ -17,8 +17,8 @@
public class IssueExtensionDemo {

// tag::issue_simple[]
@Issue("REQ-123")
@Test
@Issue("REQ-123")
void test() {
// One of the tests for the issue with the id "REQ-123"
}
Expand Down
25 changes: 21 additions & 4 deletions src/demo/java/org/junitpioneer/jupiter/StopwatchExtensionDemo.java
Expand Up @@ -14,12 +14,29 @@

public class StopwatchExtensionDemo {

// tag::stopwatch_demo[]
@Stopwatch
// tag::method[]
@Test
@Stopwatch
void test() {
// Some test
// execution time will be reported
}
// end::method[]

// tag::class[]
@Stopwatch
class TestCases {

@Test
void test_1() {
// execution time will be reported
}

@Test
void test_s() {
// execution time will be reported
}

}
// end::class[]

// end::stopwatch_demo[]
}
Expand Up @@ -40,7 +40,7 @@ class DisableIfTestFailsExtension implements TestExecutionExceptionHandler, Exec
*
* Setting the information needs to be thread safe, so only positive results (i.e. tests must be disabled)
* will be set. The easiest way to do that is to simply use absence/presence of a key in the store
* as indicator, which means the specific value doesn't.
* as indicator, which means the specific value doesn't matter.
*/

private static final Namespace NAMESPACE = Namespace.create(DisableIfTestFailsExtension.class);
Expand Down Expand Up @@ -82,8 +82,8 @@ private static Stream<Configuration> findConfigurations(ExtensionContext context
.map(DisableIfTestFailsExtension::findConfigurations)
.orElse(Stream.empty());

List<Configuration> stream = Stream.concat(onClassConfig, onParentClassConfigs).collect(toList());
return stream.stream();
List<Configuration> configurations = Stream.concat(onClassConfig, onParentClassConfigs).collect(toList());
return configurations.stream();
}

private static Stream<Configuration> createConfigurationFor(ExtensionContext context,
Expand Down
1 change: 1 addition & 0 deletions src/main/java/org/junitpioneer/jupiter/DisabledUntil.java
Expand Up @@ -28,6 +28,7 @@
* <p>{@code @DisabledUntil} can be used on the method and class level. It can only be used once per method or class,
* but is inherited from higher-level containers.</p>
*
* @since 1.6.0
* @see org.junit.jupiter.api.Disabled
*/
@Retention(RetentionPolicy.RUNTIME)
Expand Down
21 changes: 3 additions & 18 deletions src/main/java/org/junitpioneer/jupiter/StopwatchExtension.java
Expand Up @@ -36,30 +36,15 @@ public void afterTestExecution(ExtensionContext context) {
calculateAndReportElapsedTime(context);
}

/**
* Stores the current time for the method in the execution context.
*
* @param context Extension context of the class
*/
void storeNowAsLaunchTime(ExtensionContext context) {
private void storeNowAsLaunchTime(ExtensionContext context) {
context.getStore(NAMESPACE).put(context.getUniqueId(), clock.instant().toEpochMilli());
}

/**
* Loads the stored time for method from the execution context.
*
* @param context Extension context of the class
*/
long loadLaunchTime(ExtensionContext context) {
private long loadLaunchTime(ExtensionContext context) {
return context.getStore(NAMESPACE).get(context.getUniqueId(), long.class);
}

/**
* Calculates the elapsed time method and publishes it to the execution context.
*
* @param context Extension context of the class
*/
void calculateAndReportElapsedTime(ExtensionContext context) {
private void calculateAndReportElapsedTime(ExtensionContext context) {
long launchTime = loadLaunchTime(context);
long elapsedTime = clock.instant().toEpochMilli() - launchTime;

Expand Down
Expand Up @@ -62,6 +62,7 @@ public void reportingEntryPublished(TestIdentifier testIdentifier, ReportEntry e

if (messages.containsKey(REPORT_ENTRY_KEY)) {
String issueId = messages.get(REPORT_ENTRY_KEY);
// because test IDs are unique, there's no risk of overriding previously entered information
testCases.put(testId, new IssueTestCaseBuilder(testId).setIssueId(issueId));
}
}
Expand Down