Support interleaving property checks

[dmurvihill]

🚀 Feature Request

I would appreciate a driver that takes a function that closes over multiple property definitions and tests each given property in a round robin, allowing side effects from each property check to bleed into the next one.

Motivation

Allows us to re-use values from slow generators and re-use intermediate values that take a long time to generate. This would primarily be useful in integration tests.

Example

Suppose we are designing a REST API for a to-do list app (or, we could be testing its data access layer). We want to test the following properties of the task resource:

• GET after POST returns the same field values from the request body (or DTO) (2 requests)
• POST creates a unique task on every request. (4 requests)
• PUT is idempotent. (5 requests)
• GET after DELETE results in a miss. (3 requests)

To check even 10 examples of each test layer will require 140(!) requests to the backend.

With this proposed feature, on the other hand:

the_new_feature(() => {
  genFirstPostRequest = arbitraryCreateTaskDto();
  let firstPostRequest;
  let firstTaskId;
  let firstTask;

  property('GET after POST returns same field values', [genFirstPostRequest], async (firstRequest) => {
    firstPostRequest = firstRequest;
    firstTaskId = await myApi.createTask(firstRequest);
    firstTask = await myApi.getPost(firstTaskId);
    expect(firstTask).toContainEqual(firstPostRequest);
    // 3 requests
  });

  property('POST creates a unique task on every request', async () => {
    const secondTaskId = await myAPI.createTask(firstPostRequest);
    await myAPI.getTask(secondTaskId);
    expect(firstTask.id).not.toEqual(secondTask.id);
    // 2 requests
  });

  property('PUT is idempotent', [arbitraryPutTaskDto({ id: firstTaskId })], async (putRequestData) => {
    // Yes, this would seem to require chaining the POST data arbitrary into the PUT data arbitrary.
    // I think there is a way around that but I'll leave it alone for now.
    await myApi.updateTask(putRequestData);
    const updatedTask = await myApi.getTask(firstTaskId);
    await myApi.updateTask(putRequestData);
    const reUpdatedTask = await myApi.getTask(firstTaskId);
    expect(reUpdatedTask).toEqual(updatedTask);
    // 4 requests
  });

  property('GET after DELETE causes a miss', async () => {
    await myApi.deleteTask(firstTaskId);
    expect(await myApi.getTask(firstTaskId)).toBeNull());
    // 2 requests
  });
});

(I apologize for any syntax or semantic errors, I haven't checked this code but I think it is good enough to get the idea across)

Now we are testing the same four properties in 110 requests, an improvement of about 20% -- and that was just from re-using a single request!

We also save time generating the test data, which can make a significant difference in large enterprises with complex data types, as you discovered when profiling #2650.

[dubzzz]

I might be able to provide some helpers to deal with this one. Not sure yet, I'll need to test some stuff and experiment various API.