Change Tests to use MMS over manually downloading & installing

[hasezoey]

Summary

This PR changes the Tests to use MMS instead of requiring a active instance and "manually" downloading in CI tests

This PR also does:

• reset color after "No shard Testing" message
• change from "mocha.opts" file to a ".mocharc.yml"
• change "common.js"(test utils) to use URL generation based on MONGOOSE_TEST_URI for both uri and uri2
• change some tests from assert.ok(number === 0) to assert.equal(number, 0) to get better error messages (expire after seconds)
• change tests to always use start.uri over alias and static coding
• update used 5.0.x version used in test.yml from 5.0.2 to 5.0.8

This PR is a DRAFT because it is still WIP, it is WIP because:

• old code is commented out instead of removed (like setting up the replica sets & "test.yml" "manual" downloading)
• tests fail because some tests expect journaling, which is not available with ephermeralForTest, and wiredTiger yet to speed up tests (current count of failing tests is 2)

re #12257 (comment)
re #12052 (somewhat)

[hasezoey]

merged latest master, updated the used mongodb version to their latest patch version (4.0.2 -> 4.0.28, 5.0.2 -> 5.0.8)

though i am still unsure what to do with the tests that require journaling

PS: the test now should also run properly in the CI

[hasezoey]

Updated to latest master and applied changes for #12356

Issues still to resolved:

• there are tests which check something with journaling, but in-memory engine is used, what should be done about those tests?

[hasezoey]

Updated to fix CI issues

Issues still to resolved:

• there are tests which check something with journaling, but in-memory engine is used, what should be done about those tests?

[vkarpov15]

Looking good so far, I like this idea 👍

[vkarpov15]

Can we set these as global variables rather than overwriting process.env? I get why setting these variables is necessary, I just really don't like overwriting process.env.

[hasezoey]

i overwrite a process.env variable, because that is the exact variable that was already used in test/common.js, and so was the easiest way of implementing it (i dont know if global carries over, at least in jest it does not)

[vkarpov15]

global does carry over in Mocha. That is one of many reasons why we advise people against using Jest: https://mongoosejs.com/docs/jest.html#globalsetup-and-globalteardown . Overwriting env var is more of an antipattern than overwriting global IMO, although I think it would be better to just export these.

[vkarpov15]

Can we do count: 1? count: 3 takes longer to start up and we don't do things that would require count: 3, like testing replica set failover.

[hasezoey]

i changed it to 3 because the previous value was 2, and mongodb recommends to not use EVEN numbers, so instead of going down to 1 i had gone up to 2, see

mongoose/test/common.js

Line 234 in 97c3bdb

count: 2,

for previous code

[hasezoey]

Update:

• merged latest master
• updated all MMS configurations to use storageEngine: 'wiredTiger'

i changed the storage-engine to be wiredTiger because this is the only engine that has journaling which is needed for some tests and it seems like mongodb is removing (or possibly renaming?) ephermerForTest in 6.1, but does not have yet any changelog entry (and it is also not mentioned in the mongodb documentation anymore from what i can tell)
tests still pass at ~40-60 seconds (normal instance + replset count 3)

[hasezoey]

actually convert it back to draft, because i am seeing some MMS startup errors, will try to fix those before un-drafting again

[hasezoey]

Updated MMS to fix the problems i had found, i think the test now failing is not related with changing to MMS

also as for performance:
locally, with now storage-engine wiredTiger and the directories used on a tmpfs, the times range from 40s to 60s
in the github CI it seems like it is not having a tmpfs, so it is slower, but not by much and the time is between 60s and 120s (mocha does not give more accurate times it seems once hitting a minute)

[vkarpov15]

I had some nits about overwriting environment variables, but I don't think it is worth blocking this PR any more. This PR is pretty great, I love reducing the amount of misc download scripts we have. Can patch that up later as time allows.