Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: modify large queries use query more to fetch all needed records #338

Merged
merged 12 commits into from
Dec 8, 2023

Conversation

peternhale
Copy link
Collaborator

@peternhale peternhale commented Dec 1, 2023

What does this PR do?

modify large result queries to set maxFetch to the size of number of expected records

What issues does this PR fix or reference?

@W-11622957@
@W-14458776@

This fix should prevent the loss of test and coverage result records.

@W-11622957@ fix apex test result query to set use actual result size
src/i18n/i18n.ts Outdated
@@ -91,5 +91,6 @@ export const messages = {
covIdFormatErr: 'Cannot specify code coverage with a TestRunId result',
startHandshake: 'Attempting StreamingClient handshake',
finishHandshake: 'Finished StreamingClient handshake',
subscribeStarted: 'Subscribing to ApexLog events'
subscribeStarted: 'Subscribing to ApexLog events',
invalidCountQueryResult: 'Invalid query result for count query ("%s")'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a user facing error? what will the user do if one sees this error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is customer facing, but unlikely to fail. I imagine the error would become part of an issue report.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this message is not needed anymore I believe?

@@ -5,7 +5,7 @@
* For full license text, see LICENSE.txt file in the repo root or https://opensource.org/licenses/BSD-3-Clause
*/

import { ConfigAggregator, Connection } from '@salesforce/core';
import { Connection } from '@salesforce/core';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newbie question: what is the ConfigAggregator? what is it different from using it vs not using it?

'invalidCountQueryResult',
countQueries[index].substring(
0,
countQueries[index].indexOf('IN (') + 30
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is number 30 here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I brings in the 30 characters following IN ( so the user will have a bit of the set if ides being fetched. With out some sort of clipping, the message can be very large, up to ~10.5K

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this method verifyCountQueries still needed?

maxFetch:
countQueryResult[index]?.expr0 ??
config.getPropertyValue(OrgConfigProperties.ORG_MAX_QUERY_LIMIT)
maxFetch: countQueryResult[index].expr0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to add some run time logging , for e.g. found total 'x' records, etc?


return {
done: true,
totalSize: allRecords.length,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we log these details (totalSize) during the run?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diyer do you mean so they can be seen in telemetry or just in the typical local log files for the user?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log files for the user

@peternhale peternhale changed the title fix: modify large queries to control maxFetch fix: modify large queries use query more to fetch all needed records Dec 8, 2023
Copy link
Contributor

@daphne-sfdc daphne-sfdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes all look good.

All test results and code coverage results are present when running with the CLI ✅
Pulled in the dependency into the extensions and behavior remains unchanged ✅

@daphne-sfdc daphne-sfdc merged commit 77af312 into main Dec 8, 2023
11 checks passed
@daphne-sfdc daphne-sfdc deleted the phale/W-11622957-query-size branch December 8, 2023 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants