-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: Improved IP validation performance for masterKeyIPs
, maintenanceKeyIPs
#8510
perf: Improved IP validation performance for masterKeyIPs
, maintenanceKeyIPs
#8510
Conversation
I will reformat the title to use the proper commit message syntax. |
Thanks for opening this pull request! |
I will reformat the title to use the proper commit message syntax. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## alpha #8510 +/- ##
==========================================
+ Coverage 94.31% 94.32% +0.01%
==========================================
Files 186 186
Lines 14798 14827 +29
==========================================
+ Hits 13956 13985 +29
Misses 842 842 ☔ View full report in Codecov by Sentry. |
i'll try to refactor with the native module |
Okay, i've a native implementation with a layer of optimization to prevent performance impact. I'll push it today ! |
How could we determine the perf impact of this PR? |
Compared to the current implemented version and also the the other pr @mtrezza Obviously here we have a performance improvement.
I now use it on my fork version since the current implementation has avoidable performance impact and the issue linked . I checked the code source of the current lib (ip-range), a for loop, and a parse of all provided IPS ( incoming, while list ips ) are triggered at each request. I think I'll take some time asap, to take a look about the Flame graph of a parse server request to see if we have some performance improvement available. I'll open a issue with my future investigation. |
@dblythy You have looked into this issue previously, what's your opinion? |
The approach looks similar, but with the major difference that it seems to cache every valid or invalid IP. Is the performance of |
@andreisucman The library ip-range-check that is currently used in Parse Server 6.3.0 to compare IP addresses has this limitation:
So regarding your tests the value
For example, if your request is coming from an IPv6 address other than |
Thank you @mtrezza. I didn't know that, and frankly dont quite get the significance of it. I just tried to add as many combinations as possible to effectively disable this check. But I appreciate your explanation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding 0.0.0.0/0
range to test
Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>
Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>
To allow any IPv4 and IPv6 address you would set I think the |
Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>
It seems that this PR did not handle |
LGTM @mtrezza your adjustment is right ! |
@Moumouls I'm unsure about the PR title, because in my own testing IPv6 addresses were handled correctly, see also #8421 (comment). Should we rename this to a |
honestly i don't remember exactly issue encountered, but a PR was needed, feel free to adjust the PR title if you feel it do not match. @mtrezza |
OK, so I'll change this to a
But that could be considered a "luxury feature", since IPv6 / IPv4 address cross-matching is not even supported by the underlying IP range check library that Parse Server is currently using. |
masterKeyIPs
, maintenanceKeyIPs
masterKeyIPs
, maintenanceKeyIPs
# [6.5.0-alpha.2](6.5.0-alpha.1...6.5.0-alpha.2) (2023-11-19) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba))
🎉 This change has been released in version 6.5.0-alpha.2 |
# [6.5.0](6.4.0...6.5.0) (2024-03-01) ### Bug Fixes * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Improve PostgreSQL injection detection; fixes security vulnerability [GHSA-6927-3vr9-fxf2](GHSA-6927-3vr9-fxf2) which affects Parse Server deployments using a Postgres database ([#8960](#8960)) ([a6e6549](a6e6549)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986)) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba))
🎉 This change has been released in version 6.5.0 |
# [7.0.0-beta.1](6.5.0-beta.1...7.0.0-beta.1) (2024-03-19) ### Bug Fixes * CacheAdapter does not connect when using a CacheAdapter with a JSON config ([#8633](#8633)) ([720d24e](720d24e)) * Conditional email verification not working in some cases if `verifyUserEmails`, `preventLoginWithUnverifiedEmail` set to functions ([#8838](#8838)) ([8e7a6b1](8e7a6b1)) * Deny request if master key is not set in Parse Server option `masterKeyIps` regardless of ACL and CLP ([#8957](#8957)) ([a7b5b38](a7b5b38)) * Docker image not published to Docker Hub on new release ([#8905](#8905)) ([a2ac8d1](a2ac8d1)) * Docker version releases by removing arm/v6 and arm/v7 support ([#8976](#8976)) ([1f62dd0](1f62dd0)) * GraphQL file upload fails in case of use of pointer or relation ([#8721](#8721)) ([1aba638](1aba638)) * Improve PostgreSQL injection detection; fixes security vulnerability [GHSA-6927-3vr9-fxf2](GHSA-6927-3vr9-fxf2) which affects Parse Server deployments using a Postgres database ([#8961](#8961)) ([cbefe77](cbefe77)) * Incomplete user object in `verifyEmail` function if both username and email are changed ([#8889](#8889)) ([1eb95ae](1eb95ae)) * Parse Server option `emailVerifyTokenReuseIfValid: true` generates new token on every email verification request ([#8885](#8885)) ([0023ce4](0023ce4)) * Parse Server option `fileExtensions` default value rejects file extensions that are less than 3 or more than 4 characters long ([#8699](#8699)) ([2760381](2760381)) * Server crashes on invalid Cloud Function or Cloud Job name; fixes security vulnerability [GHSA-6hh7-46r2-vf29](GHSA-6hh7-46r2-vf29) ([#9024](#9024)) ([9f6e342](9f6e342)) * Server crashes when receiving an array of `Parse.Pointer` in the request body ([#8784](#8784)) ([66e3603](66e3603)) * Username is `undefined` in email verification link on email change ([#8887](#8887)) ([e315c13](e315c13)) ### Features * Add `installationId` to arguments for `verifyUserEmails`, `preventLoginWithUnverifiedEmail` ([#8836](#8836)) ([a22dbe1](a22dbe1)) * Add `installationId`, `ip`, `resendRequest` to arguments passed to `verifyUserEmails` on verification email request ([#8873](#8873)) ([8adcbee](8adcbee)) * Add `Parse.User` as function parameter to Parse Server options `verifyUserEmails`, `preventLoginWithUnverifiedEmail` on login ([#8850](#8850)) ([972f630](972f630)) * Add password validation via POST request for user with unverified email using master key and option `ignoreEmailVerification` ([#8895](#8895)) ([633a9d2](633a9d2)) * Add support for MongoDB 7 ([#8761](#8761)) ([3de8494](3de8494)) * Add support for MongoDB query comment ([#8928](#8928)) ([2170962](2170962)) * Add support for Node 20, drop support for Node 14, 16 ([#8907](#8907)) ([ced4872](ced4872)) * Add support for Postgres 16 ([#8898](#8898)) ([99489b2](99489b2)) * Allow `Parse.Session.current` on expired session token instead of throwing error ([#8722](#8722)) ([f9dde4a](f9dde4a)) * Deprecation DEPPS5: Config option `allowClientClassCreation` defaults to `false` ([#8849](#8849)) ([29624e0](29624e0)) * Deprecation DEPPS6: Authentication adapters disabled by default ([#8858](#8858)) ([0cf58eb](0cf58eb)) * Deprecation DEPPS7: Remove deprecated Cloud Code file trigger syntax ([#8855](#8855)) ([4e6a375](4e6a375)) * Deprecation DEPPS8: Parse Server option `allowExpiredAuthDataToken` defaults to `false` ([#8860](#8860)) ([e29845f](e29845f)) * Deprecation DEPPS9: LiveQuery `fields` option is renamed to `keys` ([#8852](#8852)) ([38983e8](38983e8)) * Node process exits with error code 1 on uncaught exception to allow custom uncaught exception handling ([#8894](#8894)) ([70c280c](70c280c)) * Switch GraphQL server from Yoga v2 to Apollo v4 ([#8959](#8959)) ([105ae7c](105ae7c)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986)) * Upgrade to Parse JS SDK 5 ([#9022](#9022)) ([ad4aa83](ad4aa83)) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba)) ### BREAKING CHANGES * The Parse Server option `allowClientClassCreation` defaults to `false`. ([29624e0](29624e0)) * A request using the master key will now be rejected as unauthorized if the IP from which the request originates is not set in the Parse Server option `masterKeyIps`, even if the request does not require the master key permission, for example for a public object in a public class class. ([a7b5b38](a7b5b38)) * Node process now exits with code 1 on uncaught exceptions, enabling custom handlers that were blocked by Parse Server's default behavior of re-throwing errors. This change may lead to automatic process restarts by the environment, unlike before. ([70c280c](70c280c)) * Authentication adapters are disabled by default; to use an authentication adapter it needs to be explicitly enabled in the Parse Server authentication adapter option `auth.<provider>.enabled: true` ([0cf58eb](0cf58eb)) * Parse Server option `allowExpiredAuthDataToken` defaults to `false`; a 3rd party authentication token will be validated every time the user tries to log in and the login will fail if the token has expired; the effect of this change may differ for different authentication adapters, depending on the token lifetime and the token refresh logic of the adapter ([e29845f](e29845f)) * LiveQuery `fields` option is renamed to `keys` ([38983e8](38983e8)) * Cloud Code file trigger syntax has been aligned with object trigger syntax, for example `Parse.Cloud.beforeDeleteFile'` has been changed to `Parse.Cloud.beforeDelete(Parse.File, (request) => {})'` ([4e6a375](4e6a375)) * Removes support for Node 14 and 16 ([ced4872](ced4872)) * Removes support for Postgres 11 and 12 ([99489b2](99489b2)) * The `Parse.User` passed as argument if `verifyUserEmails` is set to a function is renamed from `user` to `object` for consistency with invocations of `verifyUserEmails` on signup or login; the user object is not a plain JavaScript object anymore but an instance of `Parse.User` ([8adcbee](8adcbee)) * `Parse.Session.current()` no longer throws an error if the session token is expired, but instead returns the session token with its expiration date to allow checking its validity ([f9dde4a](f9dde4a)) * `Parse.Query` no longer supports the BSON type `code`; although this feature was never officially documented, its removal is announced as a breaking change to protect deployments where it might be in use. ([3de8494](3de8494))
🎉 This change has been released in version 7.0.0-beta.1 |
# [7.0.0](6.4.0...7.0.0) (2024-03-19) ### Bug Fixes * CacheAdapter does not connect when using a CacheAdapter with a JSON config ([#8633](#8633)) ([720d24e](720d24e)) * Conditional email verification not working in some cases if `verifyUserEmails`, `preventLoginWithUnverifiedEmail` set to functions ([#8838](#8838)) ([8e7a6b1](8e7a6b1)) * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Deny request if master key is not set in Parse Server option `masterKeyIps` regardless of ACL and CLP ([#8957](#8957)) ([a7b5b38](a7b5b38)) * Docker image not published to Docker Hub on new release ([#8905](#8905)) ([a2ac8d1](a2ac8d1)) * Docker version releases by removing arm/v6 and arm/v7 support ([#8976](#8976)) ([1f62dd0](1f62dd0)) * GraphQL file upload fails in case of use of pointer or relation ([#8721](#8721)) ([1aba638](1aba638)) * Improve PostgreSQL injection detection; fixes security vulnerability [GHSA-6927-3vr9-fxf2](GHSA-6927-3vr9-fxf2) which affects Parse Server deployments using a Postgres database ([#8961](#8961)) ([cbefe77](cbefe77)) * Incomplete user object in `verifyEmail` function if both username and email are changed ([#8889](#8889)) ([1eb95ae](1eb95ae)) * Parse Server option `emailVerifyTokenReuseIfValid: true` generates new token on every email verification request ([#8885](#8885)) ([0023ce4](0023ce4)) * Parse Server option `fileExtensions` default value rejects file extensions that are less than 3 or more than 4 characters long ([#8699](#8699)) ([2760381](2760381)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) * Server crashes on invalid Cloud Function or Cloud Job name; fixes security vulnerability [GHSA-6hh7-46r2-vf29](GHSA-6hh7-46r2-vf29) ([#9024](#9024)) ([9f6e342](9f6e342)) * Server crashes when receiving an array of `Parse.Pointer` in the request body ([#8784](#8784)) ([66e3603](66e3603)) * Username is `undefined` in email verification link on email change ([#8887](#8887)) ([e315c13](e315c13)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add `installationId` to arguments for `verifyUserEmails`, `preventLoginWithUnverifiedEmail` ([#8836](#8836)) ([a22dbe1](a22dbe1)) * Add `installationId`, `ip`, `resendRequest` to arguments passed to `verifyUserEmails` on verification email request ([#8873](#8873)) ([8adcbee](8adcbee)) * Add `Parse.User` as function parameter to Parse Server options `verifyUserEmails`, `preventLoginWithUnverifiedEmail` on login ([#8850](#8850)) ([972f630](972f630)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Add password validation via POST request for user with unverified email using master key and option `ignoreEmailVerification` ([#8895](#8895)) ([633a9d2](633a9d2)) * Add support for MongoDB 7 ([#8761](#8761)) ([3de8494](3de8494)) * Add support for MongoDB query comment ([#8928](#8928)) ([2170962](2170962)) * Add support for Node 20, drop support for Node 14, 16 ([#8907](#8907)) ([ced4872](ced4872)) * Add support for Postgres 16 ([#8898](#8898)) ([99489b2](99489b2)) * Allow `Parse.Session.current` on expired session token instead of throwing error ([#8722](#8722)) ([f9dde4a](f9dde4a)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3)) * Deprecation DEPPS5: Config option `allowClientClassCreation` defaults to `false` ([#8849](#8849)) ([29624e0](29624e0)) * Deprecation DEPPS6: Authentication adapters disabled by default ([#8858](#8858)) ([0cf58eb](0cf58eb)) * Deprecation DEPPS7: Remove deprecated Cloud Code file trigger syntax ([#8855](#8855)) ([4e6a375](4e6a375)) * Deprecation DEPPS8: Parse Server option `allowExpiredAuthDataToken` defaults to `false` ([#8860](#8860)) ([e29845f](e29845f)) * Deprecation DEPPS9: LiveQuery `fields` option is renamed to `keys` ([#8852](#8852)) ([38983e8](38983e8)) * Node process exits with error code 1 on uncaught exception to allow custom uncaught exception handling ([#8894](#8894)) ([70c280c](70c280c)) * Switch GraphQL server from Yoga v2 to Apollo v4 ([#8959](#8959)) ([105ae7c](105ae7c)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986)) * Upgrade to Parse JS SDK 5 ([#9022](#9022)) ([ad4aa83](ad4aa83)) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba)) ### BREAKING CHANGES * The Parse Server option `allowClientClassCreation` defaults to `false`. ([29624e0](29624e0)) * A request using the master key will now be rejected as unauthorized if the IP from which the request originates is not set in the Parse Server option `masterKeyIps`, even if the request does not require the master key permission, for example for a public object in a public class class. ([a7b5b38](a7b5b38)) * Node process now exits with code 1 on uncaught exceptions, enabling custom handlers that were blocked by Parse Server's default behavior of re-throwing errors. This change may lead to automatic process restarts by the environment, unlike before. ([70c280c](70c280c)) * Authentication adapters are disabled by default; to use an authentication adapter it needs to be explicitly enabled in the Parse Server authentication adapter option `auth.<provider>.enabled: true` ([0cf58eb](0cf58eb)) * Parse Server option `allowExpiredAuthDataToken` defaults to `false`; a 3rd party authentication token will be validated every time the user tries to log in and the login will fail if the token has expired; the effect of this change may differ for different authentication adapters, depending on the token lifetime and the token refresh logic of the adapter ([e29845f](e29845f)) * LiveQuery `fields` option is renamed to `keys` ([38983e8](38983e8)) * Cloud Code file trigger syntax has been aligned with object trigger syntax, for example `Parse.Cloud.beforeDeleteFile'` has been changed to `Parse.Cloud.beforeDelete(Parse.File, (request) => {})'` ([4e6a375](4e6a375)) * Removes support for Node 14 and 16 ([ced4872](ced4872)) * Removes support for Postgres 11 and 12 ([99489b2](99489b2)) * The `Parse.User` passed as argument if `verifyUserEmails` is set to a function is renamed from `user` to `object` for consistency with invocations of `verifyUserEmails` on signup or login; the user object is not a plain JavaScript object anymore but an instance of `Parse.User` ([8adcbee](8adcbee)) * `Parse.Session.current()` no longer throws an error if the session token is expired, but instead returns the session token with its expiration date to allow checking its validity ([f9dde4a](f9dde4a)) * `Parse.Query` no longer supports the BSON type `code`; although this feature was never officially documented, its removal is announced as a breaking change to protect deployments where it might be in use. ([3de8494](3de8494))
🎉 This change has been released in version 7.0.0 |
Pull Request
Issue
Closes: #8421
Approach
Build blocklist once, add optimized validation process.
Supported options:
::
and::/0
or all ipv6 match0.0.0.0
for all ipv4Tasks