Merge branch 'master' into fixSelectExclude

.github/workflows/ci.yml

@@ -62,7 +62,7 @@ jobs:
             MONGODB_STORAGE_ENGINE: wiredTiger
             NODE_VERSION: 14.16.0
           - name: Mongo 4.2, ReplicaSet, WiredTiger
-            MONGODB_VERSION: 4.2.12
+            MONGODB_VERSION: 4.2.13
             MONGODB_TOPOLOGY: replicaset
             MONGODB_STORAGE_ENGINE: wiredTiger
             NODE_VERSION: 14.16.0
@@ -72,7 +72,7 @@ jobs:
             MONGODB_STORAGE_ENGINE: wiredTiger
             NODE_VERSION: 14.16.0
           - name: Mongo 3.6, Standalone, MMAPv1
-            MONGODB_VERSION: 3.6.22
+            MONGODB_VERSION: 3.6.23
             MONGODB_TOPOLOGY: standalone
             MONGODB_STORAGE_ENGINE: mmapv1
             NODE_VERSION: 14.16.0

CHANGELOG.md

@@ -89,12 +89,13 @@ ___
 ## Unreleased (Master Branch)
 [Full Changelog](https://github.com/parse-community/parse-server/compare/4.5.0...master)
 ### Breaking Changes
+- Improved schema caching through database real-time hooks. Reduces DB queries, decreases Parse Query execution time and fixes a potential schema memory leak. If multiple Parse Server instances connect to the same DB (for example behind a load balancer), set the [Parse Server Option](https://parseplatform.org/parse-server/api/master/ParseServerOptions.html) `databaseOptions.enableSchemaHooks: true` to enable this feature and keep the schema in sync across all instances. Failing to do so will cause a schema change to not propagate to other instances and re-syncing will only happen when these instances restart. The options `enableSingleSchemaCache` and `schemaCacheTTL` have been removed. To use this feature with MongoDB, a replica set cluster with [change stream](https://docs.mongodb.com/manual/changeStreams/#availability) support is required. (Diamond Lewis, SebC) [#7214](https://github.com/parse-community/parse-server/issues/7214)
 - Added file upload restriction. File upload is now only allowed for authenticated users by default for improved security. To allow file upload also for Anonymous Users or Public, set the `fileUpload` parameter in the [Parse Server Options](https://parseplatform.org/parse-server/api/master/ParseServerOptions.html) (dblythy, Manuel Trezza) [#7071](https://github.com/parse-community/parse-server/pull/7071)
 ### Notable Changes
 - Added Parse Server Security Check to report weak security settings (Manuel Trezza, dblythy) [#7247](https://github.com/parse-community/parse-server/issues/7247)
 - EXPERIMENTAL: Added new page router with placeholder rendering and localization of custom and feature pages such as password reset and email verification (Manuel Trezza) [#6891](https://github.com/parse-community/parse-server/issues/6891)
 - EXPERIMENTAL: Added custom routes to easily customize flows for password reset, email verification or build entirely new flows (Manuel Trezza) [#7231](https://github.com/parse-community/parse-server/issues/7231)
-### Other Commits
+### Other Changes
 - Fix error when a not yet inserted job is updated (Antonio Davi Macedo Coelho de Castro) [#7196](https://github.com/parse-community/parse-server/pull/7196)
 - request.context for afterFind triggers (dblythy) [#7078](https://github.com/parse-community/parse-server/pull/7078)
 - Winston Logger interpolating stdout to console (dplewis) [#7114](https://github.com/parse-community/parse-server/pull/7114)
@@ -117,14 +118,15 @@ ___
 - Randomize test suite (Diamond Lewis) [#7265](https://github.com/parse-community/parse-server/pull/7265)
 - LDAP: Properly unbind client on group search error (Diamond Lewis) [#7265](https://github.com/parse-community/parse-server/pull/7265)
 - Improve data consistency in Push and Job Status update (Diamond Lewis) [#7267](https://github.com/parse-community/parse-server/pull/7267)
+- Excluding keys that have trailing edges.node when performing GraphQL resolver (Chris Bland) [#7273](https://github.com/parse-community/parse-server/pull/7273)
 - Fix select and excludeKey queries to properly accept JSON string arrays. Also allow nested fields in exclude. [#7242](https://github.com/parse-community/parse-server/pull/7242). Thanks to [Corey Baker](https://github.com/cbaker6).
 
 ___
 ## 4.5.0
 [Full Changelog](https://github.com/parse-community/parse-server/compare/4.4.0...4.5.0)
 ### Breaking Changes
 - FIX: Consistent casing for afterLiveQueryEvent. The afterLiveQueryEvent was introduced in 4.4.0 with inconsistent casing for the event names, which was fixed in 4.5.0. [#7023](https://github.com/parse-community/parse-server/pull/7023). Thanks to [dblythy](https://github.com/dblythy).
-### Other Commits
+### Other Changes
 - FIX: Properly handle serverURL and publicServerUrl in Batch requests. [#7049](https://github.com/parse-community/parse-server/pull/7049). Thanks to [Zach Goldberg](https://github.com/ZachGoldberg).
 - IMPROVE: Prevent invalid column names (className and length). [#7053](https://github.com/parse-community/parse-server/pull/7053). Thanks to [Diamond Lewis](https://github.com/dplewis).
 - IMPROVE: GraphQL: Remove viewer from logout mutation. [#7029](https://github.com/parse-community/parse-server/pull/7029). Thanks to [Antoine Cormouls](https://github.com/Moumouls).

README.md

@@ -122,9 +122,9 @@ Parse Server is continuously tested with the most recent releases of MongoDB to
 
 | Version     | Latest Patch Version | End-of-Life Date | Compatibility      |
 |-------------|----------------------|------------------|--------------------|
-| MongoDB 3.6 | 3.6.21               | April 2021       | ✅ Fully compatible |
+| MongoDB 3.6 | 3.6.23               | April 2021       | ✅ Fully compatible |
 | MongoDB 4.0 | 4.0.23               | January 2022     | ✅ Fully compatible |
-| MongoDB 4.2 | 4.2.12               | TBD              | ✅ Fully compatible |
+| MongoDB 4.2 | 4.2.13               | TBD              | ✅ Fully compatible |
 | MongoDB 4.4 | 4.4.4                | TBD              | ✅ Fully compatible |
 
 #### PostgreSQL

package.json

@@ -19,7 +19,7 @@
   ],
   "license": "BSD-3-Clause",
   "dependencies": {
-    "@apollographql/graphql-playground-html": "1.6.26",
+    "@apollographql/graphql-playground-html": "1.6.27",
     "@graphql-tools/links": "6.2.5",
     "@graphql-tools/stitch": "6.2.4",
     "@graphql-tools/utils": "6.2.4",
@@ -43,7 +43,7 @@
     "intersect": "1.0.1",
     "jsonwebtoken": "8.5.1",
     "jwks-rsa": "1.12.2",
-    "ldapjs": "2.2.3",
+    "ldapjs": "2.2.4",
     "lodash": "4.17.21",
     "lru-cache": "5.1.1",
     "mime": "2.5.2",
@@ -111,7 +111,7 @@
     "test:mongodb:testonly": "cross-env MONGODB_VERSION=${MONGODB_VERSION:=$npm_config_dbversion} MONGODB_TOPOLOGY=${MONGODB_TOPOLOGY:=standalone} MONGODB_STORAGE_ENGINE=${MONGODB_STORAGE_ENGINE:=wiredTiger} TESTING=1 jasmine",
     "test:mongodb": "npm run test:mongodb:runnerstart --dbversion=$npm_config_dbversion && npm run test:mongodb:testonly --dbversion=$npm_config_dbversion",
     "test:mongodb:4.0.23": "npm run test:mongodb --dbversion=4.0.23",
-    "test:mongodb:4.2.12": "npm run test:mongodb --dbversion=4.2.12",
+    "test:mongodb:4.2.13": "npm run test:mongodb --dbversion=4.2.13",
     "test:mongodb:4.4.4": "npm run test:mongodb --dbversion=4.4.4",
     "posttest:mongodb": "mongodb-runner stop",
     "pretest": "cross-env MONGODB_VERSION=${MONGODB_VERSION:=4.4.4} MONGODB_TOPOLOGY=${MONGODB_TOPOLOGY:=standalone} MONGODB_STORAGE_ENGINE=${MONGODB_STORAGE_ENGINE:=wiredTiger} mongodb-runner start",

resources/buildConfigDefinitions.js

@@ -53,6 +53,7 @@ function getENVPrefix(iface) {
     'PasswordPolicyOptions' : 'PARSE_SERVER_PASSWORD_POLICY_',
     'FileUploadOptions' : 'PARSE_SERVER_FILE_UPLOAD_',
     'SecurityOptions': 'PARSE_SERVER_SECURITY_',
+    'DatabaseOptions': 'PARSE_SERVER_DATABASE_'
   }
   if (options[iface.id.name]) {
     return options[iface.id.name]
@@ -168,7 +169,7 @@ function parseDefaultValue(elt, value, t) {
     if (type == 'NumberOrBoolean') {
       literalValue = t.numericLiteral(parsers.numberOrBoolParser('')(value));
     }
-    const literalTypes = ['Object', 'SecurityOptions', 'PagesRoute', 'IdempotencyOptions','FileUploadOptions','CustomPagesOptions', 'PagesCustomUrlsOptions', 'PagesOptions'];
+    const literalTypes = ['Object', 'SecurityOptions', 'PagesRoute', 'IdempotencyOptions','FileUploadOptions','CustomPagesOptions', 'PagesCustomUrlsOptions', 'PagesOptions', 'DatabaseOptions'];
     if (literalTypes.includes(type)) {
       const object = parsers.objectParser(value);
       const props = Object.keys(object).map((key) => {

spec/LogsRouter.spec.js

@@ -8,7 +8,9 @@ const WinstonLoggerAdapter = require('../lib/Adapters/Logger/WinstonLoggerAdapte
 
 const loggerController = new LoggerController(new WinstonLoggerAdapter());
 
-describe('LogsRouter', () => {
+describe_only(() => {
+  return process.env.PARSE_SERVER_LOG_LEVEL !== 'debug';
+})('LogsRouter', () => {
   it('can check valid master key of request', done => {
     // Make mock request
     const request = {

spec/MongoStorageAdapter.spec.js

@@ -18,6 +18,7 @@ const fakeClient = {
 describe_only_db('mongo')('MongoStorageAdapter', () => {
   beforeEach(done => {
     new MongoStorageAdapter({ uri: databaseURI }).deleteAllClasses().then(done, fail);
+    Config.get(Parse.applicationId).schemaCache.clear();
   });
 
   it('auto-escapes symbols in auth information', () => {
@@ -314,6 +315,8 @@ describe_only_db('mongo')('MongoStorageAdapter', () => {
     await user.signUp();
 
     const database = Config.get(Parse.applicationId).database;
+    await database.adapter.dropAllIndexes('_User');
+
     const preIndexPlan = await database.find(
       '_User',
       { username: 'bugs' },
@@ -546,5 +549,33 @@ describe_only_db('mongo')('MongoStorageAdapter', () => {
         });
       });
     });
+
+    describe('watch _SCHEMA', () => {
+      it('should change', async done => {
+        const adapter = new MongoStorageAdapter({
+          uri: databaseURI,
+          collectionPrefix: '',
+          mongoOptions: { enableSchemaHooks: true },
+        });
+        await reconfigureServer({ databaseAdapter: adapter });
+        expect(adapter.enableSchemaHooks).toBe(true);
+        spyOn(adapter, '_onchange');
+        const schema = {
+          fields: {
+            array: { type: 'Array' },
+            object: { type: 'Object' },
+            date: { type: 'Date' },
+          },
+        };
+
+        await adapter.createClass('Stuff', schema);
+        const myClassSchema = await adapter.getClass('Stuff');
+        expect(myClassSchema).toBeDefined();
+        setTimeout(() => {
+          expect(adapter._onchange).toHaveBeenCalled();
+          done();
+        }, 5000);
+      });
+    });
   }
 });

spec/ParseGraphQLController.spec.js

@@ -30,9 +30,7 @@ describe('ParseGraphQLController', () => {
 
   beforeEach(async () => {
     if (!parseServer) {
-      parseServer = await global.reconfigureServer({
-        schemaCacheTTL: 100,
-      });
+      parseServer = await global.reconfigureServer();
       databaseController = parseServer.config.databaseController;
       cacheController = parseServer.config.cacheController;
 

spec/ParseGraphQLSchema.spec.js

@@ -10,9 +10,7 @@ describe('ParseGraphQLSchema', () => {
   const appId = 'test';
 
   beforeEach(async () => {
-    parseServer = await global.reconfigureServer({
-      schemaCacheTTL: 100,
-    });
+    parseServer = await global.reconfigureServer();
     databaseController = parseServer.config.databaseController;
     parseGraphQLController = parseServer.config.parseGraphQLController;
     parseGraphQLSchema = new ParseGraphQLSchema({
@@ -68,7 +66,7 @@ describe('ParseGraphQLSchema', () => {
       const graphQLSubscriptions = parseGraphQLSchema.graphQLSubscriptions;
       const newClassObject = new Parse.Object('NewClass');
       await newClassObject.save();
-      await databaseController.schemaCache.clear();
+      await parseServer.config.schemaCache.clear();
       await new Promise(resolve => setTimeout(resolve, 200));
       await parseGraphQLSchema.load();
       expect(parseClasses).not.toBe(parseGraphQLSchema.parseClasses);
@@ -426,14 +424,14 @@ describe('ParseGraphQLSchema', () => {
         log: defaultLogger,
         appId,
       });
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema1 = await parseGraphQLSchema.load();
       const types1 = parseGraphQLSchema.graphQLTypes;
       const queries1 = parseGraphQLSchema.graphQLQueries;
       const mutations1 = parseGraphQLSchema.graphQLMutations;
       const user = new Parse.Object('User');
       await user.save();
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema2 = await parseGraphQLSchema.load();
       const types2 = parseGraphQLSchema.graphQLTypes;
       const queries2 = parseGraphQLSchema.graphQLQueries;
@@ -456,14 +454,14 @@ describe('ParseGraphQLSchema', () => {
       });
       const car1 = new Parse.Object('Car');
       await car1.save();
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema1 = await parseGraphQLSchema.load();
       const types1 = parseGraphQLSchema.graphQLTypes;
       const queries1 = parseGraphQLSchema.graphQLQueries;
       const mutations1 = parseGraphQLSchema.graphQLMutations;
       const car2 = new Parse.Object('car');
       await car2.save();
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema2 = await parseGraphQLSchema.load();
       const types2 = parseGraphQLSchema.graphQLTypes;
       const queries2 = parseGraphQLSchema.graphQLQueries;
@@ -486,13 +484,13 @@ describe('ParseGraphQLSchema', () => {
       });
       const car = new Parse.Object('Car');
       await car.save();
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema1 = await parseGraphQLSchema.load();
       const queries1 = parseGraphQLSchema.graphQLQueries;
       const mutations1 = parseGraphQLSchema.graphQLMutations;
       const cars = new Parse.Object('cars');
       await cars.save();
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       const schema2 = await parseGraphQLSchema.load();
       const queries2 = parseGraphQLSchema.graphQLQueries;
       const mutations2 = parseGraphQLSchema.graphQLMutations;
@@ -532,7 +530,7 @@ describe('ParseGraphQLSchema', () => {
 
       await data.save();
 
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       await parseGraphQLSchema.load();
 
       const queries1 = parseGraphQLSchema.graphQLQueries;
@@ -569,7 +567,7 @@ describe('ParseGraphQLSchema', () => {
 
       await data.save();
 
-      await parseGraphQLSchema.databaseController.schemaCache.clear();
+      await parseGraphQLSchema.schemaCache.clear();
       await parseGraphQLSchema.load();
 
       const mutations = parseGraphQLSchema.graphQLMutations;