Show both input and output in generated snapshot similar to CombinationApprovals

[Anchita]

Hi, I am looking for an equivalent of VerifyAllCombination as was available in ApprovalTests CombinationApprovals.

In essence, I would like to test with various combination of data and verify the results as well as share them with stakeholders. However, the when I use Verify, I only get the output in the file without the input corresponding to it. Is there an option that I am missing? If not could we please add support for this as it will be a handy little feature.

Thanks
Anchita

[SimonCropp]

hey Anchita

i usually recommenced people use Parameterised Tests as an alternative to CombinationApprovals

but in your case i dont think that would help, since Parameterised Tests result in each test case having a corresponding verified file.

can you share an example of you inputs and what you want to appear in the file? perhaps i can suggest an alternative approach

[Anchita]

Hi Simon,

Thank you for your response but as you say parameterised tests won't solve my testing needs.

As a example, we have a calculation engine responsible for generating probability of a result in a sporting event (eg a football match). The calculation will be based on a formula that would require various inputs about real time data, eg. which teams are playing, how many goals have been scored already, etc and it will output the probability of a win or loss for a team. The scenarios will get even more complicated to test when the calculation formula is used to generate more outputs rather than just match result. eg. It would take some data in request body and generate a response which contains probabilities for various events such as match result, first team to score, number of corners, number of penalties and so on.

To cover off all edge cases, I would like to test the calculation engine with various inputs generated through property based tests. If the VerfiyTests outputs a single file showing all the inputs and the corresponding outputs then I can share the same with the stakeholders to approve as the verified file. This will help us identify any odd behaviours in the formula use. I can then configure the property based tests to always run with the same input values to ensure that the received file matches every time.

I've had bugs in the formula that were missed by the small number of parameterised tests. If I can leverage the power of VerifyTests in combination with Property tests then that will help me increase stakeholders confidence significantly.

A hard example to explain but hope that makes sense? I can explain further if you have any questions.

Many Thanks,
Anchita

[SimonCropp]

would building up an in memory representation, and verifying that work? ie you could build up a list or dictionary of all the data and pass that to verify

[Anchita]

Thank you but I'm not sure how would that help. Test data generation and passing it to Verify is not an issue. Its the fact that the received file only contains the result of the calculations and not the inputs. The filename does contain the input data but if there was a way to have it in the file contents then that would be useful. Also, if running a 100 property based tests to test one calculation formula then it would be good to have the ability to generate a single received file containing a list of inputs and corresponding outputs. That way, there won't be a 100 verified files to maintain. Just a single one where you can see all the permutations and combinations tested and the resulting output.

[SimonCropp]

you can pass the inputs to the Verify call as well

[Anchita]

Ah, I see. Thanks, I'll give that a go.