Merge branch 'master' into vkarpov15/gh-14114

Automattic · Dec 21, 2023 · 7aec12e · 7aec12e
2 parents 4847721 + 6d87344
commit 7aec12e
Show file tree

Hide file tree

Showing 13 changed files with 160 additions and 54 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,13 @@
+8.0.3 / 2023-12-07
+==================
+ * fix(schema): avoid creating unnecessary clone of schematype in nested array so nested document arrays use correct constructor #14128 #14101
+ * docs(connections): add example of registering connection event handlers #14150
+ * docs(populate): add example of using `refPath` and `ref` functions #14133 #13834
+ * types: handle using BigInt global class in schema definitions #14160 #14147
+ * types: make findOneAndDelete() without options return result doc, not ModifyResult #14153 #14130
+ * types(model): add no-generic override for insertMany() with options #14152 #13999
+ * types: add missing Type for applyDefaults #14159 [jaypea](https://github.com/jaypea)
+
 7.6.7 / 2023-12-06
 ==================
  * fix: avoid minimizing single nested subdocs if they are required #14151 #14058

diff --git a/docs/guides.md b/docs/guides.md
@@ -41,7 +41,7 @@ integrating Mongoose with external tools and frameworks.
 * [Transactions](transactions.html)
 * [MongoDB Driver Deprecation Warnings](deprecations.html)
 * [Testing with Jest](jest.html)
-* [SSL Connections](tutorials/ssl.html)
+* [TLS/SSL Connections](tutorials/ssl.html)
 * [MongoDB Client Side Field Level Encryption](field-level-encryption.html)
 
 ## Other Guides

diff --git a/docs/layout.pug b/docs/layout.pug
@@ -62,7 +62,7 @@ html(lang='en')
                     - if ([`${versions.versionedPath}/docs/connections`, `${versions.versionedPath}/docs/tutorials/ssl`].some(path => outputUrl.startsWith(path)))
                       ul.pure-menu-list
                         li.pure-menu-item.sub-item
-                          a.pure-menu-link(href=`${versions.versionedPath}/docs/tutorials/ssl.html`, class=outputUrl === `${versions.versionedPath}/docs/tutorials/ssl.html` ? 'selected' : '') SSL Connections
+                          a.pure-menu-link(href=`${versions.versionedPath}/docs/tutorials/ssl.html`, class=outputUrl === `${versions.versionedPath}/docs/tutorials/ssl.html` ? 'selected' : '') TLS/SSL Connections
                   li.pure-menu-item.sub-item
                     a.pure-menu-link(href=`${versions.versionedPath}/docs/models.html`, class=outputUrl === `${versions.versionedPath}/docs/models.html` ? 'selected' : '') Models
                     - if ([`${versions.versionedPath}/docs/models`, `${versions.versionedPath}/docs/change-streams`].some(path => outputUrl.startsWith(path)))

diff --git a/docs/migrating_to_8.md b/docs/migrating_to_8.md
@@ -71,6 +71,16 @@ There's a few noteable changes in MongoDB Node driver v6 that affect Mongoose:
 
 1. The `ObjectId` constructor no longer accepts strings of length 12. In Mongoose 7, `new mongoose.Types.ObjectId('12charstring')` was perfectly valid. In Mongoose 8, `new mongoose.Types.ObjectId('12charstring')` throws an error.
 
+1. Deprecated SSL options have been removed
+
+   - `sslCA` -> `tlsCAFile`
+   - `sslCRL` -> `tlsCRLFile`
+   - `sslCert` -> `tlsCertificateKeyFile`
+   - `sslKey` -> `tlsCertificateKeyFile`
+   - `sslPass` -> `tlsCertificateKeyFilePassword`
+   - `sslValidate` -> `tlsAllowInvalidCertificates`
+   - `tlsCertificateFile` -> `tlsCertificateKeyFile`
+
 <h2 id="removed-findoneandremove"><a href="#removed-findoneandremove">Removed <code>findOneAndRemove()</code></a></h2>
 
 In Mongoose 7, `findOneAndRemove()` was an alias for `findOneAndDelete()` that Mongoose supported for backwards compatibility.

diff --git a/docs/tutorials/ssl.md b/docs/tutorials/ssl.md
@@ -1,42 +1,40 @@
-# SSL Connections
+# TLS/SSL Connections
 
-Mongoose supports connecting to [MongoDB clusters that require SSL connections](https://www.mongodb.com/docs/manual/tutorial/configure-ssl/). Setting the `ssl` option to `true` in [`mongoose.connect()`](../api/mongoose.html#mongoose_Mongoose-connect) or your connection string is enough to connect to a MongoDB cluster using SSL:
+Mongoose supports connecting to [MongoDB clusters that require TLS/SSL connections](https://www.mongodb.com/docs/manual/tutorial/configure-ssl/). Setting the `tls` option to `true` in [`mongoose.connect()`](../api/mongoose.html#mongoose_Mongoose-connect) or your connection string is enough to connect to a MongoDB cluster using TLS/SSL:
 
 ```javascript
-mongoose.connect('mongodb://127.0.0.1:27017/test', { ssl: true });
+mongoose.connect('mongodb://127.0.0.1:27017/test', { tls: true });
 
 // Equivalent:
-mongoose.connect('mongodb://127.0.0.1:27017/test?ssl=true');
+mongoose.connect('mongodb://127.0.0.1:27017/test?tls=true');
 ```
 
-The `ssl` option defaults to `false` for connection strings that start with `mongodb://`. However,
-the `ssl` option defaults to `true` for connection strings that start with `mongodb+srv://`. So if you are using an srv connection string to connect to [MongoDB Atlas](https://www.mongodb.com/cloud/atlas), SSL is enabled by default.
+The `tls` option defaults to `false` for connection strings that start with `mongodb://`. However,
+the `tls` option defaults to `true` for connection strings that start with `mongodb+srv://`. So if you are using an srv connection string to connect to [MongoDB Atlas](https://www.mongodb.com/cloud/atlas), TLS/SSL is enabled by default.
 
-If you try to connect to a MongoDB cluster that requires SSL without enabling the `ssl` option, `mongoose.connect()`
-will error out with the below error:
+If you try to connect to a MongoDB cluster that requires TLS/SSL without enabling the `tls`/`ssl` option, `mongoose.connect()` will error out with the below error:
 
 ```no-highlight
 MongooseServerSelectionError: connection <monitor> to 127.0.0.1:27017 closed
     at NativeConnection.Connection.openUri (/node_modules/mongoose/lib/connection.js:800:32)
     ...
 ```
 
-## SSL Validation
+## TLS/SSL Validation
 
-By default, Mongoose validates the SSL certificate against a [certificate authority](https://en.wikipedia.org/wiki/Certificate_authority) to ensure the SSL certificate is valid. To disable this validation, set the `sslValidate` option
-to `false`.
+By default, Mongoose validates the TLS/SSL certificate against a [certificate authority](https://en.wikipedia.org/wiki/Certificate_authority) to ensure the TLS/SSL certificate is valid. To disable this validation, set the `tlsAllowInvalidCertificates` (or `tlsInsecure`) option to `true`.
 
 ```javascript
 mongoose.connect('mongodb://127.0.0.1:27017/test', {
-  ssl: true,
-  sslValidate: false
+  tls: true,
+  tlsAllowInvalidCertificates: true,
 });
 ```
 
-In most cases, you should not disable SSL validation in production. However, `sslValidate: false` is often helpful
-for debugging SSL connection issues. If you can connect to MongoDB with `sslValidate: false`, but not with
-`sslValidate: true`, then you can confirm Mongoose can connect to the server and the server is configured to use
-SSL correctly, but there's some issue with the SSL certificate.
+In most cases, you should not disable TLS/SSL validation in production. However, `tlsAllowInvalidCertificates: true` is often helpful
+for debugging SSL connection issues. If you can connect to MongoDB with `tlsAllowInvalidCertificates: true`, but not with
+`tlsAllowInvalidCertificates: false`, then you can confirm Mongoose can connect to the server and the server is configured to use
+TLS/SSL correctly, but there's some issue with the certificate.
 
 For example, a common issue is the below error message:
 
@@ -45,17 +43,14 @@ MongooseServerSelectionError: unable to verify the first certificate
 ```
 
 This error is often caused by [self-signed MongoDB certificates](https://medium.com/@rajanmaharjan/secure-your-mongodb-connections-ssl-tls-92e2addb3c89) or other situations where the certificate sent by the MongoDB
-server is not registered with an established certificate authority. The solution is to set the `sslCA` option, which essentially sets a list of allowed SSL certificates.
+server is not registered with an established certificate authority. The solution is to set the `tlsCAFile` option, which essentially sets a list of allowed SSL certificates.
 
 ```javascript
 await mongoose.connect('mongodb://127.0.0.1:27017/test', {
-  ssl: true,
-  sslValidate: true,
+  tls: true,
   // For example, see https://medium.com/@rajanmaharjan/secure-your-mongodb-connections-ssl-tls-92e2addb3c89
   // for where the `rootCA.pem` file comes from.
-  // Please note that, in Mongoose >= 5.8.3, `sslCA` needs to be
-  // the **path to** the CA file, **not** the contents of the CA file
-  sslCA: `${__dirname}/rootCA.pem`
+  tlsCAFile: `${__dirname}/rootCA.pem`,
 });
 ```
 
@@ -66,7 +61,7 @@ MongooseServerSelectionError: Hostname/IP does not match certificate's altnames:
 ```
 
 The SSL certificate's [common name](https://knowledge.digicert.com/solution/SO7239.html) **must** line up with the host name
-in your connection string. If the SSL certificate is for `hostname2.mydomain.com`, your connection string must connect to `hostname2.mydomain.com`, not any other hostname or IP address that may be equivalent to `hostname2.mydomain.com`. For replica sets, this also means that the SSL certificate's common name must line up with the [machine's `hostname`](../connections.html#replicaset-hostnames).
+in your connection string. If the SSL certificate is for `hostname2.mydomain.com`, your connection string must connect to `hostname2.mydomain.com`, not any other hostname or IP address that may be equivalent to `hostname2.mydomain.com`. For replica sets, this also means that the SSL certificate's common name must line up with the [machine's `hostname`](../connections.html#replicaset-hostnames). To disable this validation, set the `tlsAllowInvalidHostnames` option to `true`.
 
 ## X.509 Authentication
 
@@ -76,39 +71,38 @@ If you're using [X.509 authentication](https://www.mongodb.com/docs/drivers/node
 // Do this:
 const username = 'myusername';
 await mongoose.connect(`mongodb://${encodeURIComponent(username)}@127.0.0.1:27017/test`, {
-  ssl: true,
-  sslValidate: true,
-  sslCA: `${__dirname}/rootCA.pem`,
-  authMechanism: 'MONGODB-X509'
+  tls: true,
+  tlsCAFile: `${__dirname}/rootCA.pem`,
+  authMechanism: 'MONGODB-X509',
 });
 
 // Not this:
 await mongoose.connect('mongodb://127.0.0.1:27017/test', {
-  ssl: true,
-  sslValidate: true,
-  sslCA: `${__dirname}/rootCA.pem`,
+  tls: true,
+  tlsCAFile: `${__dirname}/rootCA.pem`,
   authMechanism: 'MONGODB-X509',
-  auth: { username }
+  auth: { username },
 });
 ```
 
 ## X.509 Authentication with MongoDB Atlas
 
-With MongoDB Atlas, X.509 certificates are not Root CA certificates and will not work with the `sslCA` parameter as self-signed certificates would. If the `sslCA` parameter is used an error similar to the following would be raised:
+With MongoDB Atlas, X.509 certificates are not Root CA certificates and will not work with the `tlsCAFile` parameter as self-signed certificates would. If the `tlsCAFile` parameter is used an error similar to the following would be raised:
 
 ```no-highlight
 MongoServerSelectionError: unable to get local issuer certificate
 ```
 
-To connect to a MongoDB Atlas cluster using X.509 authentication the correct option to set is `tlsCertificateKeyFile`. The connection string already specifies the `authSource` and `authMechanism`, and the DNS `TXT` record would supply the parameter and value for `sslValidate`, however they're included below as `connect()` options for completeness:
+To connect to a MongoDB Atlas cluster using X.509 authentication the correct option to set is `tlsCertificateKeyFile`. The connection string already specifies the `authSource` and `authMechanism`, however they're included below as `connect()` options for completeness:
 
 ```javascript
 const url = 'mongodb+srv://xyz.mongodb.net/test?authSource=%24external&authMechanism=MONGODB-X509';
 await mongoose.connect(url, {
-  sslValidate: true,
+  tls: true,
+  // location of a local .pem file that contains both the client's certificate and key
   tlsCertificateKeyFile: '/path/to/certificate.pem',
   authMechanism: 'MONGODB-X509',
-  authSource: '$external'
+  authSource: '$external',
 });
 ```
 

diff --git a/lib/helpers/indexes/getRelatedIndexes.js b/lib/helpers/indexes/getRelatedIndexes.js
@@ -1,5 +1,7 @@
 'use strict';
 
+const hasDollarKeys = require('../query/hasDollarKeys');
+
 function getRelatedSchemaIndexes(model, schemaIndexes) {
   return getRelatedIndexes({
     baseModelName: model.baseModelName,
@@ -46,7 +48,9 @@ function getRelatedIndexes({
 
   return indexes.filter(index => {
     const partialFilterExpression = getPartialFilterExpression(index, indexesType);
-    return !partialFilterExpression || !partialFilterExpression[discriminatorKey];
+    return !partialFilterExpression
+      || !partialFilterExpression[discriminatorKey]
+      || (hasDollarKeys(partialFilterExpression[discriminatorKey]) && !('$eq' in partialFilterExpression[discriminatorKey]));
   });
 }
 

diff --git a/lib/schema/array.js b/lib/schema/array.js
@@ -638,14 +638,14 @@ handle.$and = createLogicalQueryOperatorHandler('$and');
 handle.$nor = createLogicalQueryOperatorHandler('$nor');
 
 function createLogicalQueryOperatorHandler(op) {
-  return function logicalQueryOperatorHandler(val) {
+  return function logicalQueryOperatorHandler(val, context) {
     if (!Array.isArray(val)) {
       throw new TypeError('conditional ' + op + ' requires an array');
     }
 
     const ret = [];
     for (const obj of val) {
-      ret.push(cast(this.casterConstructor.schema, obj, null, this && this.$$context));
+      ret.push(cast(this.casterConstructor.schema ?? context.schema, obj, null, this && this.$$context));
     }
 
     return ret;

diff --git a/package.json b/package.json
@@ -1,7 +1,7 @@
 {
   "name": "mongoose",
   "description": "Mongoose MongoDB ODM",
-  "version": "8.0.2",
+  "version": "8.0.3",
   "author": "Guillermo Rauch <guillermo@learnboost.com>",
   "keywords": [
     "mongodb",

diff --git a/test/helpers/indexes.getRelatedIndexes.test.js b/test/helpers/indexes.getRelatedIndexes.test.js
@@ -92,6 +92,46 @@ describe('getRelatedIndexes', () => {
         ]
       );
     });
+    it('with base model that has discriminator, it includes discriminator indexes that only checks for existence', () => {
+      // Arrange
+      const eventSchema = new Schema(
+        { actorId: { type: Schema.Types.ObjectId } },
+        { autoIndex: false }
+      );
+      eventSchema.index({ actorId: 1 },
+        { unique: true,
+          partialFilterExpression: {
+            __t: { $exists: true }
+          }
+        });
+
+      const Event = db.model('Event', eventSchema);
+
+      const clickEventSchema = new Schema(
+        {
+          clickedAt: Date,
+          productCategory: String
+        },
+        { autoIndex: false }
+      );
+      Event.discriminator('ClickEvent', clickEventSchema);
+
+      // Act
+      const filteredSchemaIndexes = getRelatedSchemaIndexes(Event, Event.schema.indexes());
+
+      // Assert
+      assert.deepStrictEqual(
+        filteredSchemaIndexes,
+        [
+          [{ actorId: 1 },
+            { background: true,
+              unique: true,
+              partialFilterExpression: { __t: { $exists: true } }
+            }
+          ]
+        ]
+      );
+    });
     it('with discriminator model, it only gets discriminator indexes', () => {
       // Arrange
       const eventSchema = new Schema(

diff --git a/test/model.query.casting.test.js b/test/model.query.casting.test.js
@@ -791,6 +791,38 @@ describe('model query casting', function() {
     assert.ok(res);
     assert.deepStrictEqual(res.map(doc => doc.arr[1].id), ['two', 'three']);
   });
+
+  it('should not throw a cast error when dealing with an array of objects in combination with $elemMatch and nested $and', async function() {
+    const testSchema = new Schema({
+      arr: [Object]
+    });
+
+    const Test = db.model('Test', testSchema);
+    const obj1 = new Test({ arr: [{ id: 'one', name: 'sample1' }, { id: 'two' }] });
+    await obj1.save();
+
+    const obj2 = new Test({ arr: [{ id: 'two', name: 'sample1' }, { id: 'three' }] });
+    await obj2.save();
+
+    const obj3 = new Test({ arr: [{ id: 'three', name: 'sample1' }, { id: 'four' }] });
+    await obj3.save();
+    const res = await Test.find({
+      arr: {
+        $elemMatch: {
+          $and: [
+            { name: 'sample1' },
+            { $or: [
+              { id: 'one' },
+              { id: 'two' }
+            ] }
+          ]
+        }
+      }
+    }).sort({ _id: 1 });
+    assert.ok(res);
+    assert.equal(res.length, 2);
+    assert.deepStrictEqual(res.map(doc => doc.arr[1].id), ['two', 'three']);
+  });
 });
 
 function _geojsonPoint(coordinates) {

diff --git a/test/types/schema.test.ts b/test/types/schema.test.ts
@@ -1368,3 +1368,14 @@ function gh13424() {
   const doc = new TestModel({});
   expectType<Types.ObjectId | undefined>(doc.subDocArray[0]._id);
 }
+
+function gh14147() {
+  const affiliateSchema = new Schema({
+    balance: { type: BigInt, default: BigInt(0) }
+  });
+
+  const AffiliateModel = model('Affiliate', affiliateSchema);
+
+  const doc = new AffiliateModel();
+  expectType<bigint>(doc.balance);
+}
diff --git a/types/inferschematype.d.ts b/types/inferschematype.d.ts
@@ -262,15 +262,16 @@ type ResolvePathType<PathValueType, Options extends SchemaTypeOptions<PathValueT
                                     IfEquals<PathValueType, Schema.Types.Decimal128> extends true ? Types.Decimal128 :
                                       IfEquals<PathValueType, Types.Decimal128> extends true ? Types.Decimal128 :
                                         IfEquals<PathValueType, Schema.Types.BigInt> extends true ? bigint :
-                                          PathValueType extends 'bigint' | 'BigInt' | typeof Schema.Types.BigInt ? bigint :
-                                            PathValueType extends 'uuid' | 'UUID' | typeof Schema.Types.UUID ? Buffer :
-                                              IfEquals<PathValueType, Schema.Types.UUID> extends true ? Buffer :
-                                                PathValueType extends MapConstructor | 'Map' ? Map<string, ResolvePathType<Options['of']>> :
-                                                  IfEquals<PathValueType, typeof Schema.Types.Map> extends true ? Map<string, ResolvePathType<Options['of']>> :
-                                                    PathValueType extends ArrayConstructor ? any[] :
-                                                      PathValueType extends typeof Schema.Types.Mixed ? any:
-                                                        IfEquals<PathValueType, ObjectConstructor> extends true ? any:
-                                                          IfEquals<PathValueType, {}> extends true ? any:
-                                                            PathValueType extends typeof SchemaType ? PathValueType['prototype'] :
-                                                              PathValueType extends Record<string, any> ? ObtainDocumentType<PathValueType, any, { typeKey: TypeKey }> :
-                                                                unknown;
+                                          IfEquals<PathValueType, BigInt> extends true ? bigint :
+                                            PathValueType extends 'bigint' | 'BigInt' | typeof Schema.Types.BigInt | typeof BigInt ? bigint :
+                                              PathValueType extends 'uuid' | 'UUID' | typeof Schema.Types.UUID ? Buffer :
+                                                IfEquals<PathValueType, Schema.Types.UUID> extends true ? Buffer :
+                                                  PathValueType extends MapConstructor | 'Map' ? Map<string, ResolvePathType<Options['of']>> :
+                                                    IfEquals<PathValueType, typeof Schema.Types.Map> extends true ? Map<string, ResolvePathType<Options['of']>> :
+                                                      PathValueType extends ArrayConstructor ? any[] :
+                                                        PathValueType extends typeof Schema.Types.Mixed ? any:
+                                                          IfEquals<PathValueType, ObjectConstructor> extends true ? any:
+                                                            IfEquals<PathValueType, {}> extends true ? any:
+                                                              PathValueType extends typeof SchemaType ? PathValueType['prototype'] :
+                                                                PathValueType extends Record<string, any> ? ObtainDocumentType<PathValueType, any, { typeKey: TypeKey }> :
+                                                                  unknown;