Rename extension to ExpectedToFail

README.md

@@ -120,6 +120,7 @@ The least we can do is to thank them and list some of their accomplishments here
 #### 2022
 
 * [Filip Hrisafov](https://github.com/filiphr) contributed the [JSON Argument Source](https://junit-pioneer.org/docs/json-argument-source/) support (#101 / #492)
+* [Marcono1234](https://github.com/Marcono1234) contributed the [`@ExpectedToFail` extension](https://junit-pioneer.org/docs/expected-to-fail-tests/) (#551 / #676)
 * [Mathieu Fortin](https://github.com/mathieufortin01) contributed the `suspendForMs` attribute in [retrying tests](https://junit-pioneer.org/docs/retrying-test/) (#407 / #604)
 * [Pankaj Kumar](https://github.com/p1729) contributed towards improving GitHub actions (#587 / #611)
 * [Rob Spoor](https://github.com/robtimus) enabled non-static factory methods for `@CartesianTest.MethodFactory` (#628)
@@ -134,7 +135,6 @@ The least we can do is to thank them and list some of their accomplishments here
 * [Scott Leberknight](https://github.com/sleberknight) resolved a javadoc issue (#547 / #548)
 * [Slawomir Jaranowski](https://github.com/slawekjaranowski) Migrate to new Shipkit plugins (#410 / #419)
 * [Stefano Cordio](https://github.com/scordio) contributed [the Cartesian Enum source](https://junit-pioneer.org/docs/cartesian-product/#cartesianenumsource) (#379 / #409 and #414 / #453)
-* [Marcono1234](https://github.com/Marcono1234) contributed the [`@NotWorking` extension](https://junit-pioneer.org/docs/not-working-tests/) (#546)
 
 #### 2020
 

docs/docs-nav.yml

@@ -16,12 +16,12 @@
         url: /docs/disable-if-test-fails/
       - title: "Disable Parameterized Tests"
         url: /docs/disable-parameterized-tests/
+      - title: "Expected to Fail Tests"
+        url: /docs/expected-to-fail-tests/
       - title: "Issue information"
         url: /docs/issue/
       - title: "JSON Argument Source"
         url: /docs/json-argument-source
-      - title: "Not Working Tests"
-        url: /docs/not-working-tests/
       - title: "Publishing Report Entries"
         url: /docs/report-entries/
       - title: "Range Sources"

docs/not-working-tests.adoc → docs/expected-to-fail-tests.adoc

@@ -1,5 +1,5 @@
-:page-title: Not Working Tests
-:page-description: Extends JUnit Jupiter with `@NotWorking`, which marks a test method as 'not working'
+:page-title: Expected to Fail Tests
+:page-description: Extends JUnit Jupiter with `@ExpectedToFail`, which marks a test method as temporarily 'expected to fail'
 
 Often tests fail due to a bug in the tested application or in used dependencies.
 Traditionally such a test method would be annotated with JUnit's `@Disabled`.
@@ -8,10 +8,10 @@ However, this has the following disadvantages when the bug causing the test fail
 * the developer might not notice the existing test method and create a new one
 * the existing test method might not be noticed and remains disabled for a long time after the bug has been fixed, adding no value for the project
 
-`@NotWorking` tries to solve these issues.
+`@ExpectedToFail` tries to solve these issues.
 Unlike `@Disabled` it still executes the annotated test method but aborts the test if a test failure or error occurs.
-However, if the test is executed successfully the `@NotWorking` annotation will cause a test failure because the test _is working_.
-This lets the developer know that they have fixed a bug (possibly by accident) and that they can now remove the `@NotWorking` annotation from the test method.
+However, if the test is executed successfully the `@ExpectedToFail` annotation will cause a test failure because the test _is working_.
+This lets the developer know that they have fixed the bug (possibly by accident) and that they can now remove the `@ExpectedToFail` annotation from the test method.
 
 The annotation can only be used on methods and as meta-annotation on other annotation types.
 Similar to `@Disabled`, it has to be used in addition to a "testable" annotation, such as `@Test`.
@@ -22,26 +22,18 @@ Such test methods should use https://junit.org/junit5/docs/current/api/org.junit
 
 == Basic Use
 
-The test is aborted because the tested method `brokenMethod()` causes a test failure.
+The test is aborted because the tested method `brokenMethod()` returns an incorrect result.
 
-[source,java]
+[source,java,indent=0]
 ----
-@NotWorking
-@Test
-void doSomething() {
-	brokenMethod();
-}
+include::{demo}[tag=expected_to_fail]
 ----
 
-A custom message can be provided, explaining why the test is not working.
+A custom message can be provided, explaining why the tested code is not working as intended at the moment.
 
-[source,java]
+[source,java,indent=0]
 ----
-@NotWorking("Implementation bug in brokenMethod()")
-@Test
-void doSomething() {
-	brokenMethod();
-}
+include::{demo}[tag=expected_to_fail_message]
 ----
 
 == Thread-Safety

src/demo/java/org/junitpioneer/jupiter/ExpectedToFailExtensionDemo.java

@@ -0,0 +1,41 @@
+/*
+ * Copyright 2016-2022 the original author or authors.
+ *
+ * All rights reserved. This program and the accompanying materials are
+ * made available under the terms of the Eclipse Public License v2.0 which
+ * accompanies this distribution and is available at
+ *
+ * http://www.eclipse.org/legal/epl-v20.html
+ */
+
+package org.junitpioneer.jupiter;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+import org.junit.jupiter.api.Test;
+
+public class ExpectedToFailExtensionDemo {
+
+	// tag::expected_to_fail[]
+	@ExpectedToFail
+	@Test
+	void test() {
+		int actual = brokenMethod();
+		assertEquals(10, actual);
+	}
+	// end::expected_to_fail[]
+
+	// tag::expected_to_fail_message[]
+	@ExpectedToFail("Implementation bug in brokenMethod()")
+	@Test
+	void doSomething() {
+		int actual = brokenMethod();
+		assertEquals(10, actual);
+	}
+	// end::expected_to_fail_message[]
+
+	private int brokenMethod() {
+		return 0;
+	}
+
+}

src/main/java/org/junitpioneer/jupiter/NotWorking.java → src/main/java/org/junitpioneer/jupiter/ExpectedToFail.java

@@ -21,11 +21,12 @@
 import org.junit.jupiter.api.extension.ExtendWith;
 
 /**
- * {@code @NotWorking} is a JUnit Jupiter extension to mark test methods as 'not working'.
+ * {@code @ExpectedToFail} is a JUnit Jupiter extension to mark test methods as temporarily
+ * 'expected to fail'.
  * Such test methods will still be executed but when they result in a test failure or error
  * the test will be aborted.
  * However, if the test method unexpectedly executes successfully, it is marked as failure
- * to let the developer know that the test is now successful and that the {@code @NotWorking}
+ * to let the developer know that the test is now successful and that the {@code @ExpectedToFail}
  * annotation can be removed.
  *
  * <p>The big difference compared to JUnit's {@link org.junit.jupiter.api.Disabled @Disabled}
@@ -44,7 +45,7 @@
  * specific action.
  *
  * <p>For more details and examples, see
- * <a href="https://junit-pioneer.org/docs/not-working-tests/" target="_top">the documentation on <code>@NotWorking</code></a>.
+ * <a href="https://junit-pioneer.org/docs/expected-to-fail-tests/" target="_top">the documentation on <code>@ExpectedToFail</code></a>.
  * </p>
  *
  * @see org.junit.jupiter.api.Disabled
@@ -53,18 +54,19 @@
  * Implementation note:
  * Only supports METHOD and ANNOTATION_TYPE as targets but not test classes because there
  * it is not clear what the 'correct' behavior would be when only a few test methods
- * execute successfully. Would the developer then have to remove the @NotWorking annotation
+ * execute successfully. Would the developer then have to remove the @ExpectedToFail annotation
  * from the test class and annotate methods individually?
  */
 @Documented
 @Retention(RUNTIME)
 @Target({ METHOD, ANNOTATION_TYPE })
-@ExtendWith(NotWorkingExtension.class)
-public @interface NotWorking {
+@ExtendWith(ExpectedToFailExtension.class)
+public @interface ExpectedToFail {
 
 	/**
-	 * Defines the message to show when a test is aborted because it is 'not working'.
-	 * This can be used for example to briefly explain why a test is not working.
+	 * Defines the message to show when a test is aborted because it is failing.
+	 * This can be used for example to briefly explain why the tested code is not working
+	 * as intended at the moment.
 	 * An empty string (the default) causes a generic default message to be used.
 	 */
 	String value() default "";

src/main/java/org/junitpioneer/jupiter/NotWorkingExtension.java → src/main/java/org/junitpioneer/jupiter/ExpectedToFailExtension.java

@@ -21,24 +21,24 @@
 import org.junit.platform.commons.support.AnnotationSupport;
 import org.opentest4j.TestAbortedException;
 
-class NotWorkingExtension implements Extension, InvocationInterceptor {
+class ExpectedToFailExtension implements Extension, InvocationInterceptor {
 
 	/**
 	 * No-arg constructor for JUnit to be able to create an instance.
 	 */
-	public NotWorkingExtension() {
+	public ExpectedToFailExtension() {
 	}
 
-	private static NotWorking getNotWorkingAnnotation(ExtensionContext context) {
+	private static ExpectedToFail getExpectedToFailAnnotation(ExtensionContext context) {
 		return AnnotationSupport
-				.findAnnotation(context.getRequiredTestMethod(), NotWorking.class)
-				.orElseThrow(() -> new IllegalStateException("@NotWorking is missing."));
+				.findAnnotation(context.getRequiredTestMethod(), ExpectedToFail.class)
+				.orElseThrow(() -> new IllegalStateException("@ExpectedToFail is missing."));
 
 	}
 
 	/**
 	 * Returns whether the exception should be preserved and reported as is instead
-	 * of considering it an expected 'not working' exception.
+	 * of considering it an 'expected to fail' exception.
 	 *
 	 * <p>This method is used for exceptions which abort test execution and should
 	 * have higher precedence than aborted exceptions thrown by this extension.
@@ -59,17 +59,17 @@ private <T> T invokeAndInvertResult(Invocation<T> invocation, ExtensionContext e
 				throw t;
 			}
 
-			NotWorking annotation = getNotWorkingAnnotation(extensionContext);
+			ExpectedToFail annotation = getExpectedToFailAnnotation(extensionContext);
 
 			String message = annotation.value();
 			if (message.isEmpty()) {
-				message = "Test marked as 'not working' failed as expected";
+				message = "Test marked as temporarily 'expected to fail' failed as expected";
 			}
 
 			throw new TestAbortedException(message, t);
 		}
 
-		return fail("Test marked as 'not working' succeeded; remove @NotWorking from it");
+		return fail("Test marked as 'expected to fail' succeeded; remove @ExpectedToFail from it");
 	}
 
 	@Override

src/main/java/org/junitpioneer/jupiter/package-info.java

@@ -8,7 +8,7 @@
  *     <li>{@link org.junitpioneer.jupiter.DefaultLocale} and {@link org.junitpioneer.jupiter.DefaultTimeZone}</li>
  *     <li>{@link org.junitpioneer.jupiter.DisabledUntil}</li>
  *     <li>{@link org.junitpioneer.jupiter.DisableIfTestFails}</li>
- *     <li>{@link org.junitpioneer.jupiter.NotWorking}</li>
+ *     <li>{@link org.junitpioneer.jupiter.ExpectedToFail}</li>
  *     <li>{@link org.junitpioneer.jupiter.ReportEntry}</li>
  *     <li>{@link org.junitpioneer.jupiter.RetryingTest}</li>
  *     <li>{@link org.junitpioneer.jupiter.StdIo}</li>