Marking .trx unit tests as retried?

[AndrewCraswell]

Description

Apologies if this is not the correct place for this question. I am working on a end-to-end testing plugin for Playwright to allow test results to be exported as VSTest .trx files to be uploaded the Azure Pipelines test report. Allowing us to use this format would give options that aren't available to alternative test result formats like JUnit. Namely, we want to support attachments (images, video, diagnostics) which JUnit doesn't support. We've got this functionality built out but are having a couple problems understanding the .trx file schema.

I want to be able to identify when a specific test failed and was retried. Playwright allows us to configure how many retry attempts should be made, but we aren't seeing that detail when opening the file in Visual Studio Test Explorer or viewing the report in Azure Pipelines. I want to be able to identify flaky tests, but not sure if that can be done by using just the .trx schema. I couldn't find the actual schema definition for .trx files, so am not sure how to proceed.

[Evangelink]

The trx file format is quite old and inherited but sadly we don't have its specification/schema. Because of the many tools parsing it, we usually avoid any change to it.

@nohwnd Any extra info we could provide here?

[nohwnd]

I don't think retries are supported in trx because the format was always driven by the capabilities of testhost, which technically does support retrying a test, but there is nothing explicit on TestObject, so you can just express them in test properties, and then your UI has to handle them.

AzDO supports retries, but I think it retries on the whole run level, so it will produce a whole new TRX.