Full End-to-End Testing with TestContainers

[jshook]

We have many integrated examples of workloads across a variety of drivers. These are shipped along with the main artifacts.
They have not all been thoroughly tested for each release.

In keeping with our plans to bring these workloads forward into the docs as living examples, we need a way to verify that they all function correctly before we ship a new release.

We can use https://www.testcontainers.org/ to stand up appropriate targets for most or all of our driver types. We can also use the built-in discovery mechanisms like --list-workloads to find all examples and run them with a small cycle count.

The goal will be to run all examples at least once before promoting any major release.

[jshook]

@MikeYaacoubStax Do you have any updates or questions to add to this conversation after looking into this further?

[MikeYaacoubStax]

Updates on the state of it so far:

• We are running the internal NBCLIScenarioParser to list the workloads
• We are filtering the workloads to test only those that have a scenario called "basic_check"
• We created a "basic_check" scenario with a low cycle count for all CQL driven workloads
• We are grouping the filtered workloads by driver type used, each group will invoke its own type of docker image container for the target system.
• For now, only CQL workloads targeting Cassandra are being tested, and they all pass
• Should be good for a pull request tomorrow, then we can add containers for other drivers and try them as we go along

[jshook]

Since this effort has been started and we have a mechanism for doing this in main, I'll retire this conversation. We can treat any further improvements as specific requests.