test: add 'test-services' system for running tests locally with requisite services in Docker #2214
Conversation
…site services in Docker
tl;dr:
npm run test-services:start # starts services in Docker
npm run test:with-test-services # runs 'npm test' with envvars from test/test-services.env
npm run test-services:stop # stops Docker containers
Closes: open-telemetry#2038
github-actions
bot
assigned blumamir, dyladan, legendecas, naseemkullah, osherv and pichlermarc
github-actions
bot
added
pkg:instrumentation-ioredis
pkg:instrumentation-mysql
pkg:instrumentation-mysql2
pkg:resource-detector-alibaba-cloud
pkg:resource-detector-aws
pkg:resource-detector-gcp
pkg:resource-detector-instana
pkg:propagation-utils
pkg:instrumentation-amqplib
pkg:instrumentation-lru-memoizer
labels
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2214 +/- ##
==========================================
- Coverage 90.97% 90.37% -0.61%
==========================================
Files 146 147 +1
Lines 7492 7502 +10
Branches 1502 1571 +69
==========================================
- Hits 6816 6780 -36
- Misses 676 722 +46 |
There was a problem hiding this comment.
First pass reviewing this I like how this is laid out, and the fact that the services are still optional to run as a separate test script.
I think both mongo and redis tests are in here because they had the original test setup, but other packages should probably be done in separate PRs to limit scope.
| npm test | ||
| ``` | ||
|
|
||
| However, some instrumentations require some test-services to be running (e.g. a MongoDB server to test the `instrumentation-mongodb` package). Use the `test-services`-related npm scripts to start test services in Docker and to run the tests with the appropriate configuration to use them: |
There was a problem hiding this comment.
We should add Docker into the Tools used section above, noting that it's optional.
| "test-services:start": "cd ../../.. && npm run test-services:start redis", | ||
| "test-services:stop": "cd ../../.. && npm run test-services:stop redis", | ||
| "test:with-test-services": "NODE_OPTIONS='-r dotenv/config' DOTENV_CONFIG_PATH=../../../test/test-services.env npm test", | ||
| "test:debug": "NODE_OPTIONS='-r dotenv/config' DOTENV_CONFIG_PATH=../../../test/test-services.env ts-mocha --inspect-brk --no-timeouts -p tsconfig.json 'test/redis.test.ts'", |
There was a problem hiding this comment.
We could use a .mocharc.cjs file for config options instead of specifying it in the CLI command.
There was a problem hiding this comment.
Wouldn't that then get used for normal test runs as well? My understanding of the point of the test:debug -- which existed before this change -- was to run with options other than a default npm run test.
There was a problem hiding this comment.
I believe mocha config files are like eslint config files where you can have multiple with custom names and pass them in wherever you need them. https://mochajs.org/#custom-locations
There was a problem hiding this comment.
Given that the diff for this part of this PR is not changing how ts-mocha is called:
- "test:debug": "cross-env RUN_REDIS_TESTS_LOCAL=true ts-mocha --inspect-brk --no-timeouts -p tsconfig.json 'test/redis.test.ts'",
+ "test:debug": "NODE_OPTIONS='-r dotenv/config' DOTENV_CONFIG_PATH=../../../test/test-services.env ts-mocha --inspect-brk --no-timeouts -p tsconfig.json 'test/redis.test.ts'",
I think changing to .mocharc.cjs is out of scope for this change and would distract from the test-services-related changes being made. Thoughts?
There was a problem hiding this comment.
Yep that's fair. I was mostly pointing out because of the long npm scripts in general that you mentioned, but otherwise agree it doesn't need to happen here.
🤔 Do we have a strict requirement that tests must be able to run on Windows natively? Could we have a suggestion to use WSL, and then write these up in a separate file to keep this cleaner? Personally I'm a fan of Makefiles but I suspect that will be too big of a change here 😅 |
Same, but ...
vs the current and long: but I don't know modern Windows dev well enough to know if that is just a middle finger to Windows devs. (Aside: If only |
|
Note that I'd considered However, it installs 24MB worth of stuff -- grant some of those deps we already have in this repo -- so I balked. ( |
I don't care strongly either way. My bias would have been to do them all in a oner... unless the changes started getting big. |
tl;dr:
Closes: #2038
current status
This is a proposal for #2038. It is incomplete and still in draft so we can discuss it.
I've only handled redis and mongo services so far, and usage by instr-redis, instr-redis-4, instr-ioredis, instr-mongodb, instr-mongoose.
If we agree that this looks reasonable, I can do the rest of the instrumentations and services listed at #2038.
/cc @pichlermarc @blumamir
overview
test-services-related npm scripts to run test services in Docker for the instrumentation tests that need it.test/docker-compose.yaml. When complete this file will look quite similar to theservices: ...section of.github/workflows/unit-test.yml../test/test-services.env.npm run test-services:startstarts all the services in Docker.npm run test:with-test-servicesrunsnpm testwith the needed env vars from ./test/test-services.env.npm run test-services:stopshuts down the Docker containersinstr-redis-4andinstr-redis(e.g.npm run test:local). That was starting/stopping a redis container for each test run. That works okay for something speedy like redis, but I don't think it will work well for things like Postgres, Cassandra, etc.example run from the top-level
Start the services:
Then run the tests:
test output
Then stop the services:
open questions
NODE_OPTIONS='-r dotenv/config' DOTENV_CONFIG_PATH=./test/test-services.env .... However, I believe it is cross-platform and will ensure those envvars are set in any Node.js process. I'd be happy to take a cleaner way. We could assume bash (sorry Windows) and do something a little simpler.