TAP support?

[BytewaveMLP]

The Test Anything Protocol is yet another way some testing frameworks might output test suite information for reporting purposes. I'm looking to integrate some sort of test reporting to a Ruby project, and existing reporting formats for RSpec leave a lot to be desired. I understand JUnit is an option, but the most popular available formatter that outputs JUnit XML groups all tests into one testsuite tag per runner, rather than grouping actual test suites together, which is far from optimal. The RSpec TAP formatter, on the other hand, outputs group data as expected.

The downside of TAP is that it would require a bespoke parser, rather than coming as a standard format eg JSON, XML. I'm also not sure how trivial it would be to implement in a project like this; it seems there are separate modules for different suite reporting schemes, which would be a start I suppose.

[EnricoMi]

I looked into supporting TAP and wasn't very delighted by the set of information provided on test failure. What this action needs to be useful (except for merely provide success and failure counts) is the test file name, test name, line number, and some message. The only information TAP provides is some test name string and message. Looks like TAP 14 provides a structured YAML block with some of those information.

I like to provide some example tap (TAP 14 spec) files that include failures and errors, I could take a look again.

[BytewaveMLP]

Hm. So upon closer investigation, the community-made TAP reporter for RSpec that I wanted to use doesn't actually support TAP 14, but rather an extended version of TAP 13. However, I might give a go at extending it to use TAP 14, as it doesn't seem excessively difficult. If I manage that, I'll provide you with a sampling of output.