Skip to content

Commit

Permalink
fix: batched evaluate action params to a chunk size (#36482)
Browse files Browse the repository at this point in the history
## Description

In this PR, we fix the payload evaluation issue by adding a batching
mechanism.

Root cause the issue here was that using yield call in loop tends to
call stack overflow which is a known open issue in the redux-saga
library. The popular workaround to this is to add `yield delay(0` after
n number of batch.

Fixes #36383

## Automation

/test all

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11134117696>
> Commit: a3a583f
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11134117696&attempt=2"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.All`
> Spec:
> <hr>Wed, 02 Oct 2024 01:49:36 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Improvements**
- Enhanced processing efficiency by introducing a batching mechanism,
allowing smoother handling of large data sets without blocking the event
loop.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: rishabhrathod01 <rishabh.rathod@appsmith.com>
  • Loading branch information
AmanAgarwal041 and rishabhrathod01 authored Oct 9, 2024
1 parent ac91339 commit a4f9905
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion app/client/src/sagas/ActionExecution/PluginActionSaga.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import {
all,
call,
delay,
put,
select,
take,
Expand Down Expand Up @@ -440,7 +441,11 @@ function* evaluateActionParams(
const arrDatatype: Array<string> = [];

// array of objects containing blob urls that is loops and individual object is checked for resolution of blob urls.
for (const val of value) {

const BATCH_CHUNK_SIZE = 100;

for (let j = 0; j < value.length; j++) {
const val = value[j];
// TODO: Fix this the next time the file is edited
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const newVal: Record<string, any> = yield call(
Expand Down Expand Up @@ -473,6 +478,11 @@ function* evaluateActionParams(
filePickerInstrumentation["fileSizes"].push(size);
filePickerInstrumentation["fileTypes"].push(type);
}

if ((j + 1) % BATCH_CHUNK_SIZE === 0) {
// Yield control back to the event loop and empty the stack trace
yield delay(0);
}
}

//Adding array datatype along with the datatype of first element of the array
Expand Down

0 comments on commit a4f9905

Please sign in to comment.