Support for multiple simultaneous tests

[xendren]

We have the k6-operator running in a k8s cluster. If TeamA creates a configMap with their load test script and TeamB creates a configMap for their test scripts, can they both apply their k6 resource at the same time to trigger the startup of both sets of load tests? We are trying to start up another load test while one is running, but the 2nd one is not trying to start. Looking in the operator log files, it appears that it is still waiting for all of the first test pods to finish. Is that by design?

[yorugac]

Hi @xendren, thanks for opening the issue. Yes, it is currently by design. You'll need separate deployments of the operator to support different teams at once.

This feature may sound simple at first glance but it is rather non-trivial to implement in a reliable manner, e.g. see #108. Also, the roadmap for the operator is in flux so we cannot promise when this feature could be added.

[xendren]

Hi @xendren, thanks for opening the issue. Yes, it is currently by design. You'll need separate deployments of the operator to support different teams at once.

This feature may sound simple at first glance but it is rather non-trivial to implement in a reliable manner, e.g. see #108. Also, the roadmap for the operator is in flux so we cannot promise when this feature could be added.

Hey @yorugac, thank you for the response. I believe operators must be unique per cluster, so that would mean only one test can be ran per cluster at a time. That is definitely a major limitation or us.
Thanks again.

[mugioka]

This feature may sound simple at first glance but it is rather non-trivial to implement in a reliable manner

If it is difficult to implement, i want namespace mode k6-operator.
Namespace mode k6-operator is only able to edit k6 CR that is exists in same namespace.

[yorugac]

@mugioka you can execute 2+ test runs in different namespaces even now. They just won't be simultaneous.

[mugioka]

@yorugac

They just won't be simultaneous.

In my environment, I would like to build a multi-tenancy load testing environment. In that case, there may be cases where TeamA and TeamB want to run load tests at the same time, so I thought it would be better to have a single k6 operator that can handle all k6 CRs, or a k6 operator with namespace mode.

[MohanedSaad]

Hi @yorugac , we have used the operator in the past and we were able to run tests simultaneously. What has changed to cause this?

[yorugac]

Hi @MohanedSaad,
In v0.0.7 we added more complex state processing than before: now operator has initialization and cleanup and also watches for when the test run will finish. It allows new features but obviously, has its drawbacks.
Basically, improving this requires more flexible state processing while maintaining all the features. There's some ongoing work for this so that is in foreseeable future, hopefully. Some related issues: #108, #128

[kneemaa]

+1 i would like to only need one operator and allow my teams or CICD to automate running of tests

[cmergenthaler]

Is there any update regarding this? We really need to enable multiple simultaneous tests in our loadtest environment

[yorugac]

@cmergenthaler, the main issue is currently #138 - it can be considered a blocker here.

Thanks for commenting: it's important to know for our internal prioritization.

[cdanmar]

I wanted to add that simultaneous tests are one of the reasons k6 load testing has been so successful in our organization. The ability to quickly stress test all important services at once enables us to simulate large scale traffic events across the organization while keeping the testing window small. It also enables our developers to self service their load tests and launch them whenever they want against our environment.

I look forward to this issue being resolved, as we are stuck on the old version since July and I might have to start looking for alternatives soon.

[yorugac]

Hi @cdanmar, thank you for feedback. FYI, we've finally prioritized the idempotency problem for this quarter. So the aim is to have the solution included into the next major release of the operator. Part of the solution will be added with the PR #169 - that should allow simultaneous tests case at least for OSS tests (i.e. without cloud output). ATM I think I'll be making a RC right after this PR is ready and merged - so that it can be used straight-away.
Apologies for such long delays!

[yorugac]

Support for simultaneous test runs with cloud output will be added with PR #213