The ephemeralForTest storage engine is reaching its end of life

[LucVidal360]

Versions

• NodeJS: 16.19.0
• mongodb-memory-server-*: 8.4.2
• mongodb(the binary version): 5.0.14
• mongodb(the js package): 4.5.0
• system: Linux Kubuntu 22.04

package: mongo-memory-server-global

What is your question?

The ephemeralForTest storage engine is reaching its end of life.

Context

In the project I'm working on, each test cleans its test collections by issuing collection.deleteMany({}) on every modified collection during the test. All these deleteMany are executed in parallel.

When executing many tests, it happens that the mongod server crashes with a segfault.

After opening an issue in MongoDB's Jira, it appears the "ephemeralForTest" engine has become unstable, starting with MongoDB 5.
They even intend to suppress entirely this engine with MongoDB 7. Cf. this comment:

In 5.0, we replaced the ephemeralForTest storage engine [...] The new implementation [...] had many bugs, like the one you discovered, and in the 7.0 release, we will remove the ephemeralForTest engine entirely

Workaround

As suggested, I used the wiredTiger engine (thanks to mongo-memory-server's option instance.storageEngine) and everything works like a charm.
I made a few benchmarks, and the performances are comparable.

Question

Knowing this, I have a few interrogations/suggestions:

• should this information be documented?
• would it be possible to have a different default engine depending on the mongodb version used?

[hasezoey]

would it be possible to have a different default engine depending on the mongodb version used?

not currently

should this information be documented?

probably should be documented yes, though personally i have not run into any issues with that yet

---

i also already read that they want to remove this storage-engine (sadly). also there seems to be a separate in-memory engine for enterprise.

when the change occurs and MMS changes its default version (to that version), then the default will also be changed (because otherwise it literally wouldnt work anymore).

if it really becomes required, the default may be changed earlier (though this will only be in a major MMS version).

for now use the mentioned workaround of manually setting the storage engine

[LucVidal360]

Thank you for your answer.

The current workaround is fine by me. I mostly intended to inform you about this.

Feel free to close this issue, or change its status, whatever suits you best.

[hasezoey]

Feel free to close this issue, or change its status, whatever suits you best.

for now i will keep it open until at least a documentation note is made about that it may be unstable and that the default will be changed in the future

[github-actions]

🎉 This issue has been resolved in version 9.0.0-beta.1 🎉

The release is available on:

• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[hasezoey]

a note has been added in beta documnetation at https://nodkz.github.io/mongodb-memory-server/versions/beta/docs/api/interfaces/mongo-memory-instance-opts#helper-type-storageengine

[alexbjorlig]

Wait a moment?

Does this means that if I would like to run MongoDB memory server, with version 7, I have to use wiredTiger?

But if I understand wiredTiger correctly, this means that it will not be in-memory then?

[hasezoey]

But if I understand wiredTiger correctly, this means that it will not be in-memory then?

yes that is correct, because with 7.0.0 wiredTiger is the only engine supported (at least for the community server version)

i know that the name mongodb-memory-server might not be entirely correct anymore, but i think this package will not rebrand

[github-actions]

🎉 This issue has been resolved in version 9.0.0-beta.2 🎉

The release is available on:

• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• npm package (@beta dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[sibelius]

can we release v9 outside beta? mongo 7 is out

[hasezoey]

mms 9 i currently in beta to get some more feedback, but yes, i think it is kinda final (it works in both typegoose and mongoose, my main real-world testing use-cases)

i will try to release it this week

[sibelius]

writeTiger is slower than ephemeral and breaks some tests

[hasezoey]

🎉 This issue has been resolved in version 9.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• GitHub release

Manual message because semantic-release hit the limit

[hasezoey]

well wiredTiger is definitely slower than the in-memory engine, but it should not cause different behavior where the in-memory engine worked.

it is recommended to set the db-path to somewhere which is using tmpfs(or equivalent)

[clemp6r]

But if I understand wiredTiger correctly, this means that it will not be in-memory then?

yes that is correct, because with 7.0.0 wiredTiger is the only engine supported (at least for the community server version)

i know that the name mongodb-memory-server might not be entirely correct anymore, but i think this package will not rebrand

The project description and documentation should at least be updated because it is currently misleading.

[hasezoey]

The project description and documentation should at least be updated because it is currently misleading.

Thanks for pointing this out, completely forgot about that, for now i have updated the github description, other places (like documentation landing page) may still need to be updated.

I am not a good writer, so for now the more concise and less miss-leading text is:

Manage & spin up mongodb server binaries with zero(or slight) configuration for tests.

EDIT:
i now also updated the documentation (though only for 10.0+ and currently only available in beta)