Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8.4 #14594

Merged
merged 50 commits into from
May 17, 2024
Merged

8.4 #14594

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
cb6d61f
wip: inferRawDocType re: #13772
vkarpov15 Sep 26, 2023
092e2c5
Merge branch '8.0' into vkarpov15/gh-13772
vkarpov15 Sep 28, 2023
7f9d520
refactor(inferrawdoctype): import helpers from inferschematype
vkarpov15 Sep 28, 2023
ccfefd8
feat: `listDatabases()` function
IslandRhythms Apr 5, 2024
b15c228
Merge branch '8.4' into IslandRhythms/list-databases
vkarpov15 Apr 8, 2024
f1667e0
fix: make listDatabases() consistent with other connection methods an…
vkarpov15 Apr 8, 2024
237b517
Merge pull request #14506 from Automattic/IslandRhythms/list-databases
vkarpov15 Apr 8, 2024
4ba9f46
Merge branch 'master' into 8.4
vkarpov15 Apr 9, 2024
2d4b8c2
refactor(connection): move listDatabases implementation to node-mongo…
vkarpov15 Apr 9, 2024
d5d1ad8
Update model.js
IslandRhythms Apr 10, 2024
6605629
fix: lint
IslandRhythms Apr 10, 2024
964fa85
Merge pull request #14515 from Automattic/vkarpov15/list-databases-fix
vkarpov15 Apr 10, 2024
ec518ed
add TypeScript types, jsdoc, and options support for Model.listSearch…
vkarpov15 Apr 16, 2024
d509ae8
Merge pull request #14519 from Automattic/IslandRhythms/gh-14450
vkarpov15 Apr 22, 2024
03b7031
types(document): make document _id type default to unknown instead of…
vkarpov15 Apr 26, 2024
23869fb
fix(model): make `recompileSchema()` overwrite existing document arra…
vkarpov15 Apr 27, 2024
6ca274d
Merge branch 'master' into 8.4
vkarpov15 Apr 27, 2024
cb2e7c1
Merge pull request #14541 from Automattic/vkarpov15/gh-14520
vkarpov15 Apr 27, 2024
cecbc44
fix(error): remove model property from CastError to avoid printing al…
vkarpov15 May 5, 2024
6ca92be
types: remove model from CastError
vkarpov15 May 5, 2024
2671828
Merge branch 'master' into 8.4
vkarpov15 May 6, 2024
8eefd65
Merge pull request #14568 from Automattic/vkarpov15/gh-14529-2
vkarpov15 May 6, 2024
db29c08
Merge branch 'master' into 8.4
vkarpov15 May 7, 2024
a928cee
fix(mongoose): handle initially null driver when instantiating Mongoo…
vkarpov15 May 7, 2024
19e744a
fix(error): clean up some circular imports in errors
vkarpov15 May 7, 2024
df1c720
feat(schema): add schema-level readConcern option to apply default re…
vkarpov15 May 8, 2024
0596f55
fix: add missing file
vkarpov15 May 8, 2024
55564e1
Merge pull request #14577 from Automattic/vkarpov15/gh-12335
vkarpov15 May 9, 2024
822392e
Merge pull request #14579 from Automattic/vkarpov15/gh-14511
vkarpov15 May 9, 2024
9e75b37
feat(mongoose): export omitUndefined() helper
vkarpov15 May 9, 2024
819fde3
feat: add `transactionAsyncLocalStorage` option to opt in to automati…
vkarpov15 May 10, 2024
b9d726d
Merge branch 'master' into 8.4
vkarpov15 May 10, 2024
2df8f37
fix lint and add extra test showing session propagates down to async …
vkarpov15 May 11, 2024
2d6358b
Merge pull request #14582 from Automattic/vkarpov15/gh-14569
vkarpov15 May 11, 2024
f498cc5
fix tests
vkarpov15 May 11, 2024
eb62a76
feat: upgrade mongodb -> 6.6.1
vkarpov15 May 11, 2024
bf2f35d
Merge pull request #14584 from Automattic/vkarpov15/mongodb-6-6
vkarpov15 May 12, 2024
3c74f46
fix tests
vkarpov15 May 13, 2024
5e9977b
Update docs/transactions.md
vkarpov15 May 13, 2024
c144bf0
Merge branch 'master' into 8.4
vkarpov15 May 13, 2024
17e1ecb
Merge branch '8.4' into vkarpov15/gh-13772
vkarpov15 May 13, 2024
713c9ba
test: fix tests
vkarpov15 May 13, 2024
792bcac
adapt changes from inferSchemaType
vkarpov15 May 13, 2024
ec7bad6
Merge pull request #14583 from Automattic/vkarpov15/gh-13889
vkarpov15 May 15, 2024
5103366
Merge branch 'master' into 8.4
vkarpov15 May 15, 2024
09bdf11
Merge branch '8.4' into vkarpov15/gh-13772
vkarpov15 May 15, 2024
3660041
docs(typescript): add section on InferRawDocType to TypeScript docs
vkarpov15 May 15, 2024
f63388f
Merge pull request #13900 from Automattic/vkarpov15/gh-13772
vkarpov15 May 15, 2024
72d877d
fix: use mongodb@6.6.2
vkarpov15 May 15, 2024
26375d6
Merge branch 'master' into 8.4
vkarpov15 May 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,7 @@ Valid options:
* [methods](#methods)
* [query](#query-helpers)
* [autoSearchIndex](#autoSearchIndex)
* [readConcern](#readConcern)

<h2 id="autoIndex"><a href="#autoIndex">option: autoIndex</a></h2>

Expand Down Expand Up @@ -1473,6 +1474,24 @@ schema.searchIndex({
const Test = mongoose.model('Test', schema);
```

<h2 id="readConcern">
<a href="#readConcern">
option: readConcern
</a>
</h2>

[Read concerns](https://www.mongodb.com/docs/manual/reference/read-concern/) are similar to [`writeConcern`](#writeConcern), but for read operations like `find()` and `findOne()`.
To set a default `readConcern`, pass the `readConcern` option to the schema constructor as follows.

```javascript
const eventSchema = new mongoose.Schema(
{ name: String },
{
readConcern: { level: 'available' } // <-- set default readConcern for all queries
}
);
```

<h2 id="es6-classes"><a href="#es6-classes">With ES6 Classes</a></h2>

Schemas have a [`loadClass()` method](api/schema.html#schema_Schema-loadClass)
Expand Down
31 changes: 28 additions & 3 deletions docs/transactions.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Transactions in Mongoose

[Transactions](https://www.mongodb.com/transactions) are new in MongoDB
4.0 and Mongoose 5.2.0. Transactions let you execute multiple operations
in isolation and potentially undo all the operations if one of them fails.
[Transactions](https://www.mongodb.com/transactions) let you execute multiple operations in isolation and potentially undo all the operations if one of them fails.
This guide will get you started using transactions with Mongoose.

<h2 id="getting-started-with-transactions"><a href="#getting-started-with-transactions">Getting Started with Transactions</a></h2>
Expand Down Expand Up @@ -86,6 +84,33 @@ Below is an example of executing an aggregation within a transaction.
[require:transactions.*aggregate]
```

<h2 id="asynclocalstorage"><a href="#asynclocalstorage">Using AsyncLocalStorage</a></h2>

One major pain point with transactions in Mongoose is that you need to remember to set the `session` option on every operation.
If you don't, your operation will execute outside of the transaction.
Mongoose 8.4 is able to set the `session` operation on all operations within a `Connection.prototype.transaction()` executor function using Node's [AsyncLocalStorage API](https://nodejs.org/api/async_context.html#class-asynclocalstorage).
Set the `transactionAsyncLocalStorage` option using `mongoose.set('transactionAsyncLocalStorage', true)` to enable this feature.

```javascript
mongoose.set('transactionAsyncLocalStorage', true);

const Test = mongoose.model('Test', mongoose.Schema({ name: String }));

const doc = new Test({ name: 'test' });

// Save a new doc in a transaction that aborts
await connection.transaction(async() => {
await doc.save(); // Notice no session here
throw new Error('Oops');
}).catch(() => {});

// false, `save()` was rolled back
await Test.exists({ _id: doc._id });
```

With `transactionAsyncLocalStorage`, you no longer need to pass sessions to every operation.
Mongoose will add the session by default under the hood.

<h2 id="advanced-usage"><a href="#advanced-usage">Advanced Usage</a></h2>

Advanced users who want more fine-grained control over when they commit or abort transactions
Expand Down
27 changes: 26 additions & 1 deletion docs/typescript/schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Mongoose can automatically infer the document type from your schema definition a
We recommend relying on automatic type inference when defining schemas and models.

```typescript
import { Schema } from 'mongoose';
import { Schema, model } from 'mongoose';
// Schema
const schema = new Schema({
name: { type: String, required: true },
Expand All @@ -32,6 +32,31 @@ There are a few caveats for using automatic type inference:
2. You need to define your schema in the `new Schema()` call. Don't assign your schema definition to a temporary variable. Doing something like `const schemaDefinition = { name: String }; const schema = new Schema(schemaDefinition);` will not work.
3. Mongoose adds `createdAt` and `updatedAt` to your schema if you specify the `timestamps` option in your schema, *except* if you also specify `methods`, `virtuals`, or `statics`. There is a [known issue](https://github.com/Automattic/mongoose/issues/12807) with type inference with timestamps and methods/virtuals/statics options. If you use methods, virtuals, and statics, you're responsible for adding `createdAt` and `updatedAt` to your schema definition.

If you need to explicitly get the raw document type (the value returned from `doc.toObject()`, `await Model.findOne().lean()`, etc.) from your schema definition, you can use Mongoose's `inferRawDocType` helper as follows:

```ts
import { Schema, InferRawDocType, model } from 'mongoose';

const schemaDefinition = {
name: { type: String, required: true },
email: { type: String, required: true },
avatar: String
} as const;
const schema = new Schema(schemaDefinition);

const UserModel = model('User', schema);
const doc = new UserModel({ name: 'test', email: 'test' });

type RawUserDocument = InferRawDocType<typeof schemaDefinition>;

useRawDoc(doc.toObject());

function useRawDoc(doc: RawUserDocument) {
// ...
}

```
Comment on lines +57 to +58
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```
```

minor, remove trailing space from example


If automatic type inference doesn't work for you, you can always fall back to document interface definitions.

## Separate document interface definition
Expand Down
5 changes: 5 additions & 0 deletions lib/aggregate.js
Original file line number Diff line number Diff line change
Expand Up @@ -1022,6 +1022,11 @@ Aggregate.prototype.exec = async function exec() {
applyGlobalMaxTimeMS(this.options, model.db.options, model.base.options);
applyGlobalDiskUse(this.options, model.db.options, model.base.options);

const asyncLocalStorage = this.model()?.db?.base.transactionAsyncLocalStorage?.getStore();
if (!this.options.hasOwnProperty('session') && asyncLocalStorage?.session != null) {
this.options.session = asyncLocalStorage.session;
}

if (this.options && this.options.cursor) {
return new AggregationCursor(this);
}
Expand Down
71 changes: 46 additions & 25 deletions lib/connection.js
Original file line number Diff line number Diff line change
Expand Up @@ -398,11 +398,7 @@ Connection.prototype.createCollection = async function createCollection(collecti
throw new MongooseError('Connection.prototype.createCollection() no longer accepts a callback');
}

if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) {
await new Promise(resolve => {
this._queue.push({ fn: resolve });
});
}
await this._waitForConnect();

return this.db.createCollection(collection, options);
};
Expand Down Expand Up @@ -494,11 +490,7 @@ Connection.prototype.startSession = async function startSession(options) {
throw new MongooseError('Connection.prototype.startSession() no longer accepts a callback');
}

if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) {
await new Promise(resolve => {
this._queue.push({ fn: resolve });
});
}
await this._waitForConnect();

const session = this.client.startSession(options);
return session;
Expand Down Expand Up @@ -539,7 +531,7 @@ Connection.prototype.startSession = async function startSession(options) {
Connection.prototype.transaction = function transaction(fn, options) {
return this.startSession().then(session => {
session[sessionNewDocuments] = new Map();
return session.withTransaction(() => _wrapUserTransaction(fn, session), options).
return session.withTransaction(() => _wrapUserTransaction(fn, session, this.base), options).
then(res => {
delete session[sessionNewDocuments];
return res;
Expand All @@ -558,9 +550,16 @@ Connection.prototype.transaction = function transaction(fn, options) {
* Reset document state in between transaction retries re: gh-13698
*/

async function _wrapUserTransaction(fn, session) {
async function _wrapUserTransaction(fn, session, mongoose) {
try {
const res = await fn(session);
const res = mongoose.transactionAsyncLocalStorage == null
? await fn(session)
: await new Promise(resolve => {
mongoose.transactionAsyncLocalStorage.run(
{ session },
() => resolve(fn(session))
);
});
return res;
} catch (err) {
_resetSessionDocuments(session);
Expand Down Expand Up @@ -618,13 +617,24 @@ Connection.prototype.dropCollection = async function dropCollection(collection)
throw new MongooseError('Connection.prototype.dropCollection() no longer accepts a callback');
}

await this._waitForConnect();

return this.db.dropCollection(collection);
};

/**
* Waits for connection to be established, so the connection has a `client`
*
* @return Promise
* @api private
*/

Connection.prototype._waitForConnect = async function _waitForConnect() {
if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) {
await new Promise(resolve => {
this._queue.push({ fn: resolve });
});
}

return this.db.dropCollection(collection);
};

/**
Expand All @@ -637,16 +647,31 @@ Connection.prototype.dropCollection = async function dropCollection(collection)
*/

Connection.prototype.listCollections = async function listCollections() {
if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) {
await new Promise(resolve => {
this._queue.push({ fn: resolve });
});
}
await this._waitForConnect();

const cursor = this.db.listCollections();
return await cursor.toArray();
};

/**
* Helper for MongoDB Node driver's `listDatabases()`.
* Returns an object with a `databases` property that contains an
* array of database objects.
*
* #### Example:
* const { databases } = await mongoose.connection.listDatabases();
* databases; // [{ name: 'mongoose_test', sizeOnDisk: 0, empty: false }]
*
* @method listCollections
* @return {Promise<{ databases: Array<{ name: string }> }>}
* @api public
*/

Connection.prototype.listDatabases = async function listDatabases() {
// Implemented in `lib/drivers/node-mongodb-native/connection.js`
throw new MongooseError('listDatabases() not implemented by driver');
};

/**
* Helper for `dropDatabase()`. Deletes the given database, including all
* collections, documents, and indexes.
Expand All @@ -667,11 +692,7 @@ Connection.prototype.dropDatabase = async function dropDatabase() {
throw new MongooseError('Connection.prototype.dropDatabase() no longer accepts a callback');
}

if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) {
await new Promise(resolve => {
this._queue.push({ fn: resolve });
});
}
await this._waitForConnect();

// If `dropDatabase()` is called, this model's collection will not be
// init-ed. It is sufficiently common to call `dropDatabase()` after
Expand Down
13 changes: 13 additions & 0 deletions lib/drivers/node-mongodb-native/connection.js
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,19 @@ NativeConnection.prototype.doClose = async function doClose(force) {
return this;
};

/**
* Implementation of `listDatabases()` for MongoDB driver
*
* @return Promise
* @api public
*/

NativeConnection.prototype.listDatabases = async function listDatabases() {
await this._waitForConnect();

return await this.db.admin().listDatabases();
};

/*!
* ignore
*/
Expand Down
2 changes: 1 addition & 1 deletion lib/error/browserMissingSchema.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');


class MissingSchemaError extends MongooseError {
Expand Down
1 change: 0 additions & 1 deletion lib/error/cast.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ class CastError extends MongooseError {
* ignore
*/
setModel(model) {
this.model = model;
this.message = formatMessage(model, this.kind, this.value, this.path,
this.messageFormat, this.valueType);
}
Expand Down
2 changes: 1 addition & 1 deletion lib/error/divergentArray.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class DivergentArrayError extends MongooseError {
/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/eachAsyncMultiError.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');


/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/invalidSchemaOption.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class InvalidSchemaOptionError extends MongooseError {
/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/missingSchema.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class MissingSchemaError extends MongooseError {
/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/notFound.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* Module dependencies.
*/

const MongooseError = require('./');
const MongooseError = require('./mongooseError');
const util = require('util');

class DocumentNotFoundError extends MongooseError {
Expand Down
2 changes: 1 addition & 1 deletion lib/error/objectExpected.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');


class ObjectExpectedError extends MongooseError {
Expand Down
2 changes: 1 addition & 1 deletion lib/error/objectParameter.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class ObjectParameterError extends MongooseError {
/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/overwriteModel.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');


class OverwriteModelError extends MongooseError {
Expand Down
2 changes: 1 addition & 1 deletion lib/error/parallelSave.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* Module dependencies.
*/

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class ParallelSaveError extends MongooseError {
/**
Expand Down
2 changes: 1 addition & 1 deletion lib/error/strict.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');


class StrictModeError extends MongooseError {
Expand Down
2 changes: 1 addition & 1 deletion lib/error/strictPopulate.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

'use strict';

const MongooseError = require('./');
const MongooseError = require('./mongooseError');

class StrictPopulateError extends MongooseError {
/**
Expand Down
Loading