Releases: testground/testground
Release v0.6.0 - EKS
Introduction
As an outcome of the hardwork done in 2022 with revitalising Testground, we can run large scale tests in k8s with a cluster that auto-scales up and down once idle. Please read more about achievements in this blog post:
https://blog.ipfs.tech/testground-highlights-in-2022/
In order to use cluster:k8s
with EKS, please install the latest release of testground/infra
What's Changed
- fix: 'testground describe' does not have args by @hacdias in #1124
- fix: local_exec path injection by @adlrocha in #1126
- add flag to control cgo by @nonsense in #1132
- fix regression in default client URL by @raulk in #1134
- feat: supervisor and async tasks by @hacdias in #1125
- initial notification to slack on completed task ; basic html pages for tasks and logs by @nonsense in #1140
- exec:go: build with no optimisations nor inlining. by @raulk in #1101
- add goreportcard badge by @nonsense in #1164
- prepend pod name to logs by @nonsense in #1152
- disable github actions by @nonsense in #1167
- add route to gateway, so that we dont blackhole it by @nonsense in #1171
- feat: builder docker/node and sdk-js by @hacdias in #1158
- simplify network test plan by @nonsense in #1177
- Fixed exec:go replace by @adlrocha in #1176
- metrics autodiscover / automated dashboards for testplans by @nonsense in #1173
- feat: support env vars in composition files by @hacdias in #1200
- testplan benchmarks: remove
all
testcase by @nonsense in #1206 - add support for capturing profiles. by @raulk in #1209
- fix: redis port collision check by @hacdias in #1210
- add support for temp directories to local:exec and local:docker runners. by @raulk in #1212
- use new SDK in quickstart template by @coryschwartz in #1217
- local:docker should not expose 6060 twice by @nonsense in #1207
- build(deps): bump lodash from 4.17.20 to 4.17.21 in /plans/example-js by @dependabot in #1228
- build(deps): bump hosted-git-info from 2.8.8 to 2.8.9 in /plans/example-js by @dependabot in #1227
- chore: upgrade to Go 1.16 and change defaults by @hacdias in #1233
- feat: emit periodic goroutine dumps by @hacdias in #1234
- fix: macOS CI by @hacdias in #1246
- build(deps): bump glob-parent from 5.1.1 to 5.1.2 in /plans/example-js by @dependabot in #1243
- fix: return error instead of panicking by @hacdias in #1245
- Revert "feat: emit periodic goroutine dumps" by @hacdias in #1242
- fix: enable Redis on k8s test by @hacdias in #1250
- fix: handle k8s .initPool error by @hacdias in #1247
- chore: update CI machine to newer LTS Ubuntu by @hacdias in #1261
- fix: extra_sources builder reference by @daonb in #1259
- chore: update go dependencies (excl. docker) by @hacdias in #1258
- fix: do not change read-only non-namespaced sysctl by @hacdias in #1260
- feat: http sync service by @hacdias in #1187
- fix: k8s dependencies by @hacdias in #1268
- fix: delete another nf_conntrack_max by @Rorical in #1269
- feat: optionally disable metrics batching by @hacdias in #1256
- chore: add node.js examples by @hacdias in #1270
- fix: handle http error on client by @hacdias in #1276
- feat: update default node image by @hacdias in #1277
- chore: update to xcode 13.2.0 by @hacdias in #1285
- plans/example-rust: Add minimal Rust example by @mxinden in #1287
- plans/example-js: fix test using new default client by @laurentsenta in #1306
- build(deps): bump async from 3.2.0 to 3.2.3 in /plans/example-js by @dependabot in #1318
- build(deps): bump ansi-regex from 4.1.0 to 4.1.1 in /plans/example-js by @dependabot in #1319
- build(deps): bump minimist from 1.2.5 to 1.2.6 in /plans/example-js by @dependabot in #1301
- Fix task unmarshaling by @brdji in #1330
- Add tmpl_dir to daemon env. by @brdji in #1339
- feat: Add additional parameters to groups in composition files by @laurentsenta in #1338
- feat: print explicit outcome by @laurentsenta in #1345
- build(deps): bump crossbeam-utils from 0.8.6 to 0.8.8 in /plans/example-rust by @dependabot in #1348
- plan/example-rust: fix the test with new sdk and optimize build by @laurentsenta in #1327
- fix: update test & ci to use the sync service by @laurentsenta in #1353
- fix: testground run with --wait exits with code = 0 on some failures by @laurentsenta in #1347
- fix: build config bugfixes by @laurentsenta in #1360
- Do not specify GOARCH when building to support arm64 by @lalexgap in #1372
- fix: show task outcome in cli by @laurentsenta in #1369
- fix: builder result parsing in web server by @laurentsenta in #1321
- interop: add builder and path options in compositions' groups by @laurentsenta in #1367
- Update env-example.toml by @dektech in #1375
- Fix storm test by @brdji in #1393
- chores: simplify labels by @laurentsenta in #1395
- fix: in docker:go builder, path & modfile options can't be mixed by @laurentsenta in #1413
- fix: testground instability due to race condition on outcomes by @laurentsenta in #1407
- Add support for marking runners disabled ( #1236) by @brdji in #1405
- Limit runs per branch by @brdji in #1399
- feat: various usability & stabiliy improvements for interop by @laurentsenta in #1421
- ci: docker publish edge version by @laurentsenta in #1426
- Force --wait when caller passes --collect by @wdbaruni in #1436
- cmd/build: fix
write-artifacts
to correctly truncate & close the output by @laurentsenta in #1444 - Add skipped integration tests to Makefile by @brdji in #1467
- integration_test: fix interop test 19 by @laurentsenta in #1480
- Fix SKIP_LOG_PARSING condition by @kerzhner in #1479
- Exchange IPs in pingpong test by @brdji in #1462
- feat: introduce changelog by @laurentsenta in #1445
- Feat/1386 browser hello world test by @GlenDC in #1474
- InfluxDB: enable Flux by @ackintosh in #1473
- fix: go dependencies rewrite in exec:go by @adlrocha in #1469
- Allow connecting to additional containers on the control network + integration test by @kerzhner in #1481
- ROADMAP.md: Create our Roadmap by @laurentsenta in #1484
- ROADMAP.md: refine by @laurentsenta in #1497
- ci: disable k8s tests in circleci by @galargh in https://g...
Release v0.5.3
📡 Traffic Routing Policies
We now support traffic routing policies, i.e., you can decide whether or not your tests can have access to the external Internet. This feature is available on the local:docker
and cluster:k8s
runners and it's defined via a new option on the network.Config
struct of our Go SDK:
cfg := &network.Config{
Network: "default",
RoutingPolicy: network.AllowAll // or network.DenyAll
}
By default, DenyAll
is applied.
👷♀️ Builders
docker:go
suffered a few improvements since last version:
- You can now configure the base image to build from (#1081).
- We added an option to skip the runtime image (#1078).
- The builder is now terminatable via
testground terminate --builder docker:go
(#1079). - It is now possible to specify cross-repository replacements for dependencies (#1099).
- We now support Go dependency caching in order to speed up builds (#1115).
For all builders, you can now ship additional files with the test plan source code (#1085), using a extra_sources
map, consisting of builder => []paths
.
🔐 Basic Authentication
The daemon now supports basic authentication through the tokens
array in the configuration file. The client should, in turn, provide a single token
on its configuration to connect to the daemon (#1113).
💧 Trickle default configs to all groups
Compositions can now specify [build]
and [run]
subtables in the [global]
table. Build and run settings in groups will inherit the default configuration (1aaab).
🐞 Fixes and small improvements
- Use gzip on all runners when collecting outputs.
- Fixed misleading default values on some commands, such as
testground run single help
(#1072). - Fixed
healthcheck
deadlock oncluster:k8s
(#1074). - Increase build timeout (#1075).
- User can now defined custom exposed ports (#1093).
- Port numbers are now injected as env variables (#1095).
- Detect Redis port collisions (#1108).
- Copy sources for each unique build to avoid sharing directories (#1117).
📦 Upgrading?
A few breaking changes were made since the last release. When upgrading please take this into consideration:
- You should update your
.env.toml
client.endpoint
to include a scheme - for examplehttp
(#1118). See sample.env.toml
for details. - By default,
DenyAll
is applied for the Traffic Routing Policies. On previous versions,AllowAll
was applied tocluster:k8s
andDenyAll
tolocal:docker
(#1082, #1084, #1107 #1119).
Release v0.5.2
This is a smaller release, but there are a few bug fixes which are important.
Bugfixes
- Build on macOS works again.
- Improved error reporting about configuration errors.
Improved testing
- Sidecar runs with pprof enabled by default.
Laying the groundwork for TAAS
- Testground users will not notice yet, but Testground-as-a-service is coming. This release includes some groundwork that will come in handy for asynchronous task-based execution, coming soon in a future release.
Upgrading from v0.5.1
Note that a few configuration parameters have been changed. Consult .env.toml and update your environment configuration if necessary.
Release v0.5.1
Testground v0.5.1 is a maintenance release, but also includes a few new features.
🌐 Network partitions
Testground now includes functionality for introducing network partitions. Users can now include network rules when configuring networks with the sidecar
, specifying if they want to allow, drop or reject certain packets.
🧪 Integration tests
We improved the test coverage of Testground and now run a lot more tests on every PR and commit to the master branch, such as integration tests for all our runners
👷 Builders
docker:go
builder extensions - we've included a few simple hooks to allow users to modify the default Dockerfile build steps, enabling functionality such as:
- adding an extra binary to the final image.
- adding static libraries that need to be present at runtime.
- adding debugging tools.
- copying input assets (e.g. test vectors).
docker:generic
builder - users can now include their own Dockerfile along with their test plan.
Additional improvements
- We use deterministic Docker image tags, which results in much faster test plan builds, runs and feedback loops when running a given test plan.
- Users now see additional auxiliary outputs from the
sidecar
when running test plans with thelocal:docker
runner. This enables faster debugging of more complex plans that take advantage of the network shaping functionality in Testground.
For a full list of every change in this release, click here.
If you are compiling Testground on macOS, use commit 277364c160882e6ac3c83106d17ce438dfdefdd9
, rather than tag v0.5.1
. Due to adding tests to the sidecar
(which works only under linux
), we forgot to add a build directive, which results into failed compilation on macOS for v0.5.1
.
Thanks 🙌
Huge shout out to all contributors and users of Testground. Your help is invaluable!
Release v0.5.0 (the launch release)
>>> Read the announcement blog post! <<<
The v0.5.0 release of Testground comes packed with a complete overhaul of the codebase and lots of features, and stability, hardening and usability improvements!
🎓 This release also marks our graduation from the warm and friendly IPFS community into a top-level GitHub organisation of our own. 🚀
The ambitious mission of building bulletproof, unstoppable networks and systems requires rock-solid testing platforms. We hope Testground brings about a quantum leap in the way we—as a community—engineer p2p and distributed systems for the decentralized future.
Read on to discover all the features, improvements, and changes that this release brings about!
👯♀️ SDK: The sync service is now turbocharged and nicer to use (v2)
- The API has been revamped almost entirely for ease of use, removal of friction, and performance.
- Subscriptions no longer create a blocking connection each.
- Barriers now use
MGET
, and are governed by a single Ticker. - No more painful pointer/value confusion when subscribing and publishing. The
Topic
can use pointer or value types.- Similarly,
Publish()
andSubscribe()
are intelligent enough to recognise and convert between pointer and value types of the underlying type.
- Similarly,
- Usage of Redis TTLs has been dropped. The previous implementation was unreliable, as it made assumptions about test participant lifetime and duration.
- Lots of API sugar with
Must*
variants that panic on error (useful for usage withrunner.Invoke[Map]
), and composite operations (e.g.SignalAndWait
,PublishSubscribe
)
☎️ SDK: new network
package to expose network operations such as traffic shaping
network.NewClient
creates a network client, with which you can wait for the network to be initialized, apply traffic shaping, and obtain your data network IP easily.- Check out the full docs at pkg.go.dev.
📊 Observability pipeline rearchitected, migrated to InfluxDB, with tight SDK integration
- The Prometheus + Pushgateway setup was brittle and unstable.
- Test plans now emit:
- Life-cycle events: facilitate real-time progress monitoring, either by a human, or by the forthcoming watchtower/supervisor service.
runenv.RecordStart()
,runenv.RecordFailure()
,runenv.RecordSuccess()
, etc.
- Diagnostics: track insights about the test instance execution itself in real-time, e.g. sync service stats, stages entry/exit, network API events, runtime stats (e.g. go runtime metrics).
- Results: observations about the subsystems and components under test. Conceptually speaking, results are a part of the test output. Results are the end goal of running a test plan, and feed comparative series over runs of a test plan, along time, across dependency sets.
runenv.R()
- Results are batch-uploaded to InfluxDB when the test concludes.
- Life-cycle events: facilitate real-time progress monitoring, either by a human, or by the forthcoming watchtower/supervisor service.
- See architecture diagram below for a complete view of the circuitry.
- Healthchecks now instantiate InfluxDB.
🧩 Daemon no longer knows about test plans
- We brought sanity to the client/daemon separation. The daemon is now a building and execution engine with no knowledge of concrete test plans.
- Test plans can now live anywhere (public/private Git repos, or filesystem); they do not need to be co-located with the testground source.
- Plans are managed on the client side, and we’ve introduced the
testground plan
command group to streamline management. More on this below.
🏠 $TESTGROUND_HOME
directory is the homebase of everything
- It hosts plans, SDKs, and data and work directories. The layout is as follows:
$TESTGROUND_HOME
|
|__ plans >>> [c] contains test plans, can be git checkouts, symlinks to local dirs, or the source itself
| |__ suite-a >>> test plans can be grouped in suites (which in turn can be nested); this enables you to host many test plans in a single repo / directory.
| | |__ plan-1 >>> source of a test plan identified by suite-a/plan-1 (relative to $TESTGROUND_HOME/plans)
| | |__ plan-2
| |__ plan-3 >>> source of a test plan identified by plan-3 (relative to $TESTGROUND_HOME/plans)
|
|__ sdks >>> [c] hosts the test development SDKs that the client knows about, so they can be used with the --link-sdk option.
| |__ sdk-go
|
|__ data >>> [d] data directory
|__ outputs
|__ work
[c] = used client-side // [d] = used mostly daemon-side.
- If not set explicitly, we default to
$HOME/testground
, and the daemon creates the skeleton on first start.
🐣 Dedicated GitHub organisation + repo refactor
- Testground now lives on its own GitHub organisation, and has multiple repos.
- testground/testground is the platform repo and main entry point.
- testground/sdk-go contains the Go SDK. It is now versioned separately from the platform.
- testground/infra contains the infrastructure scripts to create and manage a Kubernetes cluster.
- testground/docs is our docs site, powered by GitBook.
- testground/pm contains project management artifacts.
- Additionally, project-specific test plans have been extracted from the
testground
repository into third party repositories:
📒 New Testground documentation website
- Our documentation website is now live at https://docs.testground.ai/
- We have added a lot of articles documenting many parts of Testground, and how to use it.
💎 cluster:k8s
runner and local:docker
runner and infrastructure hardening
- CPU limits on test plan pods have been removed due to throttling when using the
cluster:k8s
runner. - The
control
network has been blackholed - communicating between test plans over thecontrol
network is no longer possible. - Testground daemon is now deployed remotely on Kubernetes when using the cluster:k8s runner.
- Kernel parameters have been adjusted to support test plan runs with up to 10k libp2p nodes.
🗂 New testground plan
CLI command group + CLI cleanup
testground plan
command group:testground plan import --git --from <git url>
testground plan create
to quickly create a templated test plan. Templates are picked from https://github.com/testground/plan-templates.testground plan rm
removes a test plan from$TESTGROUND_HOME/plans
.testground plan list [--testcases]
enumerates test plans and (optionally) test cases known to the client.
- General CLI cleanup to reduce surprise factor and friction.
🚀 Testground CI has moved from Travis to CircleCI
- Builds are much faster now, with quicker feedback cycles. Down from ~15 minutes to ~4 minutes!
- CI builds now benefit from Docker layer caching.
Thanks 🙌
HUGE, HUGE thanks to everyone who made it possible to get Testground to this stage. Your continued code contributions, patience and debugging energy, organisational skills, technical writing abilities, comms kung-fu, etc. were all indispensable to hit this incredible milestone.
❤️ @raulk, @nonsense, @coryschwartz, @Robmat05, @hacdias, @daviddias, @momack2, @yusefnapora, @vyzo, @dirkmc, @aschmahmann, @Stebalien, @jimpick, @petar. ❤️
Release v0.4.0
Testground v0.4.0 is a bugfix and infrastructure hardening release.
Highlighted v0.4.0 changes/features
🌊 Testground daemon outputs stream
- Testground now streams all output for the
testground daemon
to thetestground client
, making it easier to monitor ongoing testplan runs.
👯 Deduplication of testplan builds
- Testground now runs only unique builds for a given Testground composition.
🏥 Healthchecks for various runners (local:docker
, cluster:k8s
, etc.) have been improved
💪 Kubernetes cluster environment hardening
-
Testground dependencies, such as Redis and Prometheus, are now running on separate host machines than testplan instances, which provides better isolation and resource management.
-
Resource requirements and limits are now enforced for all services running on Testground infrastructure.
-
Kubernetes setup scripts have been improved and are now more robust.
-
Kernel parameters, such as Netfilter's connection tracking limits, and file descriptor limits, have been adjusted to support testplan runs with up to 10k libp2p nodes.
We have merged a lot of bug fixes and improvements to all Testground components. For a full list of every change, click here.
Thanks 🙌
Huge thanks to the early adopters and users of Testground: @yusefnapora, @vyzo, @dirkmc, @aschmahmann, @Stebalien, @jimpick, @petar . Your help is invaluable in making Testground a great platform for distributed testing at scale.
Thanks to all the contributors that made this milestone happen: @coryschwartz, @raulk, @Robmat05, @nonsense, @yusefnapora, @vyzo, @dirkmc, @aschmahmann, @petar, @Stebalien, @jimpick, @daviddias, @hacdias and others. ❤️
Release v0.3.0
Observability and robustness are the key themes for this Testground release! Note: we have upgraded to Go 1.14, so make sure you have too if you plan on building from source.
Highlighted v0.3.0 changes/features
👁 Automatic exposition of pprof and Prometheus endpoints in test instances
- All test instances automatically expose their pprof and Prometheus endpoint.
- Each instance prints out the URL of its HTTP instrumentation endpoint (pprof/Prometheus) on startup.
- In
local:exec
, instances run as system processes, and they use host ports. We start with 6060, and fall back to port 0 (random) on bind failure. Browse to the printed out address directly. - In
local:docker
, instances bind to port 6060, which is exposed and published on a random port on the host. Rundocker ps
ordocker port <container_id>
to find which host port to access. - In
cluster:k8s
, instances bind to port 6060, and users are encouraged to usekubectl port-forward
to set up a local endpoint that forward to the pod's port.
- In
💪 Stability/maturity/evolutionary improvements
- Add a set of baseline benchmarks to acquire accurate readings of the performance of infrastructural components.
- Harden Redis and and the sync service on various fronts.
- Refactor sidecar to make it more maintainable.
- Rate-limit container start and deletion in the
local:docker
runner, to avoid hammering poor, old Docker and making it trip over. - Improve terminal output of Docker messages.
- Made output paths consistent.
💾 cluster:k8s
runner: switched from S3 to EFS
- For better performance.
testground collect --runner cluster:k8s
has been adapted to fetch outputs from EFS.- If you're running a cluster, make sure to destroy it and re-create it. See Kubernetes README for more instructions.
💊 Prometheus and pushgateway logic in healthchecks + fixes
- The healthchecks in the
local:docker
andcluster:k8s
runner now start and expose Prometheus and pushgateway automatically. - We expect to merge the analogous checks for the
local:exec
runner by the next release.
📊 Preview: Grafana dashboard manager
- Read
dashboards/README.md
. - Ready-made Grafana dashboards for infrastructural components: Weave and Redis.
Everything else
Check everything else that went into into release v0.3.0 here: https://github.com/ipfs/testground/milestone/5?closed=1.
Thanks!
BIG THANK YOU to all the contributors that persevered while the outside world is taking an unprecedented, uncertain, bold turn to combat the pandemic that affects us all. We hope you found a little refuge in Testground to escape the mental loops, despair and boredom that self-isolation and other very necessary measures impose on our human condition. Keep it coming! We're moving boldly towards an epic v0.4.0 release!
❤️ @coryschwartz @nonsense @aschmahmann @Robmat05 @dirkmc @yusefnapora @petar @raulk @daviddias @hacdias ❤️
Release v0.2.0
This is another action-packed release from the Testground team. A solid step in our path towards creating a robust and delightful platform for testing distributed systems at all scales! Read on to learn what's new.
Highlighted v0.2.0 features
🏥 FEATURE: Runner healthchecks + automatic fixing.
- Testground now supports the
testground healthcheck --runner <runner_id> [--fix]
command, which automates the verification that all preconditions for the runner to operate properly are met. - This includes things like Redis processes/containers, sidecar containers, directories, etc.
- The
--fix
switch will attempt automatic healing of healthchecks that fail. testground run
implicitly performs a healthcheck on the target runner, before scheduling the run.- For now, the
local:exec
andlocal:docker
runners support healthchecks, withcluster:k8s
joining the party soon. - Builders will also support healthchecks in the near future.
💣 FEATURE: Runner termination.
- Testground now supports the
testground terminate --runner <runner_id>
command, which destroys the environment of a runner, including all started test jobs/containers, as well as precondition containers (Redis, sidecar, etc.) - In the future, this command will be more flexible, allowing the user to indicate the scope of the destruction (#611).
- Supported runners:
local:docker
,cluster:k8s
.
🧩 FEATURE: Manual build selectors.
- Testground composition files can now specify selectors to be applied to each build. These translate to build tags in Go builds, and can be used to construct shims for funnel-shaped wildcard test plans, such that a single test plan can target a variety of upstream dependency versions with changing APIs.
- Read more in the docs/EVOLVING_APIs.md design doc.
- In the future, we will introduce automatic build selectors, in the manner described in the above doc.
📦 FEATURE: --collect
and --collect-into
flags.
- The
testground run
command now supports the--collect
and--collect-into
flags that automatically perform output collection (i.e.testground collect
) into an archive with the run ID name (--collect
), or a user-specified file (--collect-into
).
📡 FEATURE: Increased observability + Prometheus + pushgateway in cluster:k8s.
- The Kubernetes run environment now bundles Prometheus and pushgateway support, for test plans and other infrastructural components to be able to push metrics proactively.
- pprof enabled on sidecar.
- More improvements coming in this area soon.
Fixes and improvements
We have merged TONS of bug fixes and improvements in the sidecar, sync service, Redis scaling, cluster:k8s runner, k8s networking setup, S3, etc.
- IMPROVEMENTS (observability): measure when testground is ready and testplans are running (#560), sdk runtime metrics (#545), expose pprof port on sidecar (#558), use shared ssh key for kops hosts ; command to extract kube context (#551).
- FIXES (sidecar): listen for more Docker events, not just
stop
(#580), free netlink handle (#575), sidecar errors: check for context canceled (#526), fix: remove a runenv hack in the sidecar (#503). - FIXES (sync service): increase redis max clients (#578), cancel all subscriptions when closing the watcher (#576), use a shared redis client (#574), test redis address resolution, resolve redis once, abort early when we get too many results, improve context abort error, don't panic on error if we're canceling anyways, add pprof port to pods (#553), adjust redis config (#595), avoid logging spurious errors when we shutdown the watcher, feat: use contexts for the sync service (#456), buffer barrier channel (#516).
- FIXES (runtime): runenv: flush logger on close. (#518), feat: distinguish between runenv and run params.
- FIXES (AWS): aws ecr repo must be unique across regions (#536), fix pushing image to remote (aws ecr) (#493), extract S3_ENDPOINT in var, so that we can change region for bucket (#541), fix some minor issues with the S3 collection logic (#533).
- FIXES (k8s networking): wait for flannel initContainer (#563).
- FIXES (exec:go builder): don't hide output from go commands.
- FIXES (cluster:k8s runner): handle canceled context in cluster_k8s.go (#510), configurable pod resource requirements (#513), max allowed pods check in cluster_k8s.go (#509), extract outputs configs to toml configuration (#494).
- FIXES (docker:go builder): docker builder volume (#591).
- DOC: Proposal: dealing with upstream API evolution in test plans (#565).
Release v0.1.0
Highlighted v0.1.0 features
- Ability to run test plans ranging from 1-1000 nodes locally (executables and Docker), and/or on a cluster (Kubernetes).
- Ability to compile test plans into Docker images (compatible with the local:docker and cluster:k8s runners), and into executables (local:exec, for rapid prototyping and iteration).
- Compiling test plans against specific upstream dependencies (e.g. kad-dht v0.3, or commit 1a2b3c).
- Network traffic shaping: simulating latencies, bandwidths, and connectedness.
- Redis-backed sync API to coordinate and choreograph distributed test workloads.
- Compositions: executing test runs with groups of instances built against different upstreams (e.g. 500 instances against go-ipfs v0.4.23, 250 instances against master, 250 against commit 42a0b1, with params X=1, Y=2).
- Storage of test outputs and metrics locally, and on S3.
- With a single command, collect all outputs from all instances from a given run, no matter the runner (local:exec, local:docker, cluster:k8s), into a ZIP file for analysis.
- One-click bootstrapping of your own experimental Kubernetes cluster on AWS.
Thanks 🙌
Thanks to all the contributors that made this huge milestone happen: @raulk, @nonsense, @daviddias, @hacdias, @Stebalien, @dirkmc, @aschmahmann, @jimpick, as well as all the new folks who started helping out recently: @coryschwartz, @gmasgras, @yusefnapora, and others. ❤️