fix: added configurable connection pool support for mssql #38818
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughThe pull request introduces enhanced configurability for the MSSQL plugin's connection pool management. By integrating a Changes
Sequence DiagramsequenceDiagram
participant Executor as MssqlPluginExecutor
participant Config as ConnectionPoolConfig
participant DataSource as HikariDataSource
Executor->>Config: getMaxConnectionPoolSize()
Config-->>Executor: Mono<Integer> maxPoolSize
Executor->>DataSource: createConnectionPool(config, maxPoolSize)
Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
app/server/appsmith-plugins/mssqlPlugin/src/main/java/com/external/plugins/MssqlPlugin.java (1)
Line range hint
575-590: Add input validation for maxPoolSizeWhile the implementation is correct, consider adding validation to ensure maxPoolSize is not negative or unreasonably large.
private static HikariDataSource createConnectionPool( DatasourceConfiguration datasourceConfiguration, Integer maxPoolSize) throws AppsmithPluginException { + if (maxPoolSize != null && maxPoolSize <= 0) { + throw new AppsmithPluginException( + AppsmithPluginError.PLUGIN_DATASOURCE_ARGUMENT_ERROR, + "Maximum pool size must be greater than 0"); + } // ... rest of the method
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
app/server/appsmith-plugins/mssqlPlugin/src/main/java/com/external/plugins/MssqlPlugin.java(5 hunks)app/server/appsmith-plugins/mssqlPlugin/src/test/java/com/external/plugins/MssqlTestDBContainerManager.java(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: perform-test / client-build / client-build
- GitHub Check: perform-test / rts-build / build
- GitHub Check: server-spotless / spotless-check
- GitHub Check: server-unit-tests / server-unit-tests
🔇 Additional comments (4)
app/server/appsmith-plugins/mssqlPlugin/src/test/java/com/external/plugins/MssqlTestDBContainerManager.java (2)
23-28: LGTM - Mock implementation is clean and focusedThe mock implementation provides a clear, single-responsibility test double for connection pool configuration.
30-31: LGTM - Proper initialization with mock configThe static executor is correctly initialized with the mock connection pool configuration.
app/server/appsmith-plugins/mssqlPlugin/src/main/java/com/external/plugins/MssqlPlugin.java (2)
118-122: LGTM - Clean dependency injection patternThe connection pool configuration is properly injected through constructor, following good dependency injection practices.
366-372: LGTM - Well-structured reactive flowThe datasource creation properly handles the async nature of pool size configuration using reactive patterns.
NilanshBansal
approved these changes
Jan 23, 2025
Failed server tests
|
|
This PR has not seen activitiy for a while. It will be closed in 7 days unless further activity is detected. |
github-actions bot
pushed a commit
to Zeral-Zhang/appsmith
that referenced
this pull request
Feb 7, 2025
…g#38818) ## Description With current implementation, MSSQL is configured with HikariCP to have maximum pool size of 10. But this means that if multiple concurrent users are accessing multiple queries of the same datasource at the same time, some of the queries might time out due to max pool size of 10. We have encountered this issue in https://theappsmith.slack.com/archives/C0341RERY4R/p1736150680663059. Where 40-50 concurrent users are simultaneously making 20-30 queries to MSSQL database. This PR makes this connection pool size configurable from admin settings for MSSQL as well. Fixes #appsmithorg/appsmith-ee#5928 _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /ok-to-test tags="@tag.All" ### 🔍 Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/12925650660> > Commit: ebda936 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=12925650660&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.All` > Spec: > <hr>Thu, 23 Jan 2025 11:18:35 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Enhanced connection pool configuration for MSSQL plugin - Added support for dynamically setting maximum connection pool size - **Improvements** - Increased flexibility in database connection management - Implemented configurable connection pool settings - **Tests** - Added mock configuration for testing connection pool settings <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: “sneha122” <“sneha@appsmith.com”>
2 tasks
trishaanand
added a commit
that referenced
this pull request
Feb 11, 2025
… for Ms-sql (#39176) ## Description Production is seeing high number of pod restarts causing downtime. It seems to be correlated with user MS-SQL query triggers. On the hypothesis that its indeed the plugin executions causing this, it could be the datasource create taking up redis thread pool (possibly because of an upstream scheduling of a task on it) Sample log lines displaying the behaviour (only user email removed from the logs) : traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - boundedElastic-13: datasourceCreate() called for MSSQL plugin. Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,357] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - lettuce-epollEventLoop-10-3: Connecting to SQL Server db Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,381] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - HikariPool-3 - Starting... Feb 11 05:38:15 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:15,346] [parallel-2] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=88ee6f7444f698dc - parallel-2: Action Query3 with id 67aa94d7b95fcb7ea3d5512c execution time : 10002 ms This was possibly brought on by the following PR : #38818 As part of the above PR, scheduleOn statement moved from Mono.callable() which creates its own subscription chain to outside. This could be causing the heavy database ops moving to event loop instead of bounded elastic. Fixes #`Issue Number` _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /test sanity ### 🔍 Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/13265007600> > Commit: 3e758fb > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=13265007600&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Tue, 11 Feb 2025 14:53:05 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Refactor** - Optimized the connection creation workflow to defer resource initialization until required. This enhancement improves the scheduling and responsiveness of establishing connections, ensuring that system resources are utilized more efficiently during runtime. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
trishaanand
added a commit
that referenced
this pull request
Feb 11, 2025
… for Ms-sql (#39176) ## Description Production is seeing high number of pod restarts causing downtime. It seems to be correlated with user MS-SQL query triggers. On the hypothesis that its indeed the plugin executions causing this, it could be the datasource create taking up redis thread pool (possibly because of an upstream scheduling of a task on it) Sample log lines displaying the behaviour (only user email removed from the logs) : traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - boundedElastic-13: datasourceCreate() called for MSSQL plugin. Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,357] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - lettuce-epollEventLoop-10-3: Connecting to SQL Server db Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,381] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - HikariPool-3 - Starting... Feb 11 05:38:15 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:15,346] [parallel-2] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=88ee6f7444f698dc - parallel-2: Action Query3 with id 67aa94d7b95fcb7ea3d5512c execution time : 10002 ms This was possibly brought on by the following PR : #38818 As part of the above PR, scheduleOn statement moved from Mono.callable() which creates its own subscription chain to outside. This could be causing the heavy database ops moving to event loop instead of bounded elastic. Fixes #`Issue Number` _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /test sanity ### 🔍 Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/13265007600> > Commit: 3e758fb > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=13265007600&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Tue, 11 Feb 2025 14:53:05 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Refactor** - Optimized the connection creation workflow to defer resource initialization until required. This enhancement improves the scheduling and responsiveness of establishing connections, ensuring that system resources are utilized more efficiently during runtime. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
4 tasks
github-actions bot
pushed a commit
to Zeral-Zhang/appsmith
that referenced
this pull request
Feb 12, 2025
… for Ms-sql (appsmithorg#39176) ## Description Production is seeing high number of pod restarts causing downtime. It seems to be correlated with user MS-SQL query triggers. On the hypothesis that its indeed the plugin executions causing this, it could be the datasource create taking up redis thread pool (possibly because of an upstream scheduling of a task on it) Sample log lines displaying the behaviour (only user email removed from the logs) : traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - boundedElastic-13: datasourceCreate() called for MSSQL plugin. Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,357] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - lettuce-epollEventLoop-10-3: Connecting to SQL Server db Feb 11 05:38:05 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:05,381] [lettuce-epollEventLoop-10-3] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=00fdca579a517b66 - HikariPool-3 - Starting... Feb 11 05:38:15 appsmith-d6d86999-6djmx appsmith backend stdout | [2025-02-11 00:08:15,346] [parallel-2] requestId=dd547055-d9bc-41ed-b7b4-03b0de691984 userEmail=<> traceId=bda95acf0e182c48037214d5173d0ac9 spanId=88ee6f7444f698dc - parallel-2: Action Query3 with id 67aa94d7b95fcb7ea3d5512c execution time : 10002 ms This was possibly brought on by the following PR : appsmithorg#38818 As part of the above PR, scheduleOn statement moved from Mono.callable() which creates its own subscription chain to outside. This could be causing the heavy database ops moving to event loop instead of bounded elastic. Fixes #`Issue Number` _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /test sanity ### 🔍 Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/13265007600> > Commit: 3e758fb > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=13265007600&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Tue, 11 Feb 2025 14:53:05 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Refactor** - Optimized the connection creation workflow to defer resource initialization until required. This enhancement improves the scheduling and responsiveness of establishing connections, ensuring that system resources are utilized more efficiently during runtime. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
With current implementation, MSSQL is configured with HikariCP to have maximum pool size of 10. But this means that if multiple concurrent users are accessing multiple queries of the same datasource at the same time, some of the queries might time out due to max pool size of 10. We have encountered this issue in https://theappsmith.slack.com/archives/C0341RERY4R/p1736150680663059. Where 40-50 concurrent users are simultaneously making 20-30 queries to MSSQL database.
This PR makes this connection pool size configurable from admin settings for MSSQL as well.
Fixes #https://github.com/appsmithorg/appsmith-ee/issues/5928
or
Fixes
Issue URLWarning
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.
Automation
/ok-to-test tags="@tag.All"
🔍 Cypress test results
Tip
🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/12925650660
Commit: ebda936
Cypress dashboard.
Tags:
@tag.AllSpec:
Thu, 23 Jan 2025 11:18:35 UTC
Communication
Should the DevRel and Marketing teams inform users about this change?
Summary by CodeRabbit
New Features
Improvements
Tests