Unbuffered TestDox printer option

[epdenouden]

As a developer of slower integration tests
I want to see prettified tests results as soon as tests are completed
so that my workflow is both agile and æsthetic

Background

The buffering testdox printer and its friend the spinner are breaking some workflows that people used to good effect.

The example from #3560 is using Testdox to get readable results while running. The actual test order is not that important, set by configuration or implied by test collection structure. My implementation of the buffering printer breaks this specific use case.

Solution

Implement explicit "do not buffer result ouput" option, along the lines of #3513:

• extend buffering mechanism to explicitly support always-flush
• add CLI flag
• add end-to-end tests voor --enable-buffer and --disable-buffer flag
• add XML-configuration option
• add unit test coverage for unbuffered Testdox

[rpkamp]

This would be a lot simpler if the buffering were done in a decorator.

By default you apply the decorator when the tests are shuffled, unless the CLI/config indicates it shouldn't.

This has the added benefit that the decorator could decorate any result printer, not just the TestDox printer. This may not be interesting for PHPUnit itself, but it might be for third party printers.

I wouldn't mind taking a stab at this if you'd like.

[BafS]

Any new about this issue @epdenouden ?

[epdenouden]

Yes, this issue is on hold until the new event system is ready.

Back then I did some experiments and it ended up about as hacky as the TestDox internals.

[N4M3Z]

I see the issue is still marked as blocked, are there any news about the new event system?

[epdenouden]

@m4rw3r The event system is still under development. @localheinz and @theseer are working on the central parts. As soon as those are (pre-release) stable I can migrate the logging and execution reordering.

[epdenouden]

Also, this is under active development again as I want more logging options while working on #4302 and adjacent refactorings.

[epdenouden]

[BafS]

@sebastianbergmann this PR didn't get merged #4346, this issue is still valid.

[sebastianbergmann]

The existing TestDox implementation in PHPUnit 8.5 and PHPUnit 9.5 will not be changed. PHPUnit 10 will have an implementation of TestDox that does not have this problem.

[BafS]

Okay thanks for the feedback!