Provide pioneer own AssertJ assertions

[Bukama]

When checking ExcecutionResults inside a test class for Throwables you have to go through the results and filter them. While doing this, already assertions are made that there are such results. Furthermore to avoid repeated codes some assertions were moved into a method inside the test class and were so hided when only looking at the acutal test case.

We also have some private assertThat... methods in some extension test classes which can be used in other tests so. Moreover there are no good assertThat(results) methods in the default AssertJ API. So in stream the idea came up to provide some own AssertJ assertion methods, to do something like the following.

//   TODO: assertThat(results).hasSingleTestFailedWith(ExtensionConfigurationException.class);
//   TODON'T: result.hasThrowable().isInstanceOf().hasMessage()?
     results.assertTestFailedWithExtensionConfigurationException();

Another one:

PioneerAnnotationUtilsTests:

		private void assertFailedTestHasMessage(ExecutionResults results, String... messages) {
			assertThat(results.numberOfFailedTests()).isEqualTo(1);
			assertThat(results.firstFailuresThrowableMessage()).isEqualTo(String.join(",", messages));
		}

Examples can be seen at this tutorial

[Michael1993]

I've been thinking about writing an entire separate part for single test assertions (because we do a LOT of single test assertions)
So instead of saying:

assertThat(results).hasSingleStartedTest();
assertThat(results).hasNumberOfFailedTests(0);
assertThat(results).hasSingleSuccessfulTest();

We could write:

assertThat(results).hasSingleTest().thatStarted().thatSucceeded();

Or maybe even:

assertThat(results).hasSingleTest().thatStarted().thatSucceeded().butDidNot().fail();

But that seems slightly overkill to me.

[aepfli]

Generally speaking, and from an API point of perspective. i would recommend to think what would a user need, and not just what is sufficient for our usecase. Hence that i would actually prefer a way where i can request a specific amount of tests like

assertThat(result).hasNumberOfTests(int amount)
assertThat(result).hasNumberOfTestsBetween(int start, int end)

and based on that we could further test for

....executed
....successful
....failed // or what ever naming

so we can use the same asserts on one or 2 or n test results. if we than create a wrapper method which is name .hasSingleTest() and is internally calling .hasNumberOfTests(1) is not the big magic after all. But in the future we could use this for more than just the single testcase, with imho little implementation effort. or do i miss a crucial part here?

[Michael1993]

What I meant was:
Are there enough assertions that assert a single test to create a whole different interface for those assertions? You don't really answer that question in your answer (only tangentially).

Edit of shame: I misunderstood what you meant. Of course we could also have the same interface for multiple tests:

assertThat(results).hasNumberOfTests(2).thatStarted().thatFailed();

Did you mean it like that? If so, you are absolutely right.

[Michael1993]

Where do we stop, though? You could even do something like this:

assertThat(results).hasSingleTest().that.wasStarted().and.hasFailed();

But that seems like a lot of effort for something that's meant to be used internally only.
I mean, I'd love to do it, but it does kind of look insane (and not really practical).

[aepfli]

well the initial goal is to use it internally, but what if your code and the usage is so beautiful and intuitive that this might even be used in other junit5 extension for assertions :) who knows.