Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update nonFatalErrors in subgraphs.subgraph_deployment table #4615

Merged
merged 2 commits into from
Jun 2, 2023

Conversation

incrypto32
Copy link
Member

Solves #3350

@incrypto32 incrypto32 force-pushed the incrypto32/fix-non-fatal-errors-not-populating branch from 458997c to 3760a00 Compare May 12, 2023 14:01
@incrypto32 incrypto32 marked this pull request as ready for review May 13, 2023 13:47
@incrypto32 incrypto32 requested a review from neysofu May 16, 2023 15:01
Copy link
Collaborator

@lutter lutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be best to wait with this until I have merged #4606 since that introduces a lot of changes to how things get written.

core/src/subgraph/runner.rs Outdated Show resolved Hide resolved
@incrypto32 incrypto32 force-pushed the incrypto32/fix-non-fatal-errors-not-populating branch 3 times, most recently from aac2ad3 to 7154d87 Compare May 23, 2023 10:45
@incrypto32 incrypto32 requested a review from lutter May 23, 2023 16:03
Copy link
Collaborator

@lutter lutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few stylistic comments that would be good to address before merging, but other than that this looks good.

It would also be good to have a test that checks that the non-fatal errors on a deployment get set now when that should happen (it's enough if this is just a test within the store) so we don't accidentally regress.

One thing I am not clear on is whether fail_subgraph still needs to also store the error - it seems like that is just additional work now?

@@ -468,6 +468,7 @@ where
} = block_state;

let first_error = deterministic_errors.first().cloned();
let deterministic_errors = Arc::new(deterministic_errors);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't put this into an Arc; that way, you don't have to call clone().to_vec() below and can just pass deterministic_errors into transact_block_operations

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks for pointing it out, did so since previously non_fatal_errors were being written in the same function and deterministic_errors get moved, now we do that in transact_block_operations so its no longer required to be Arc

@@ -1118,6 +1118,16 @@ impl DeploymentStore {
)?;
}

if !batch.deterministic_errors.is_empty() && batch.is_non_fatal_errors_active {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to pull this if into the previous if statement so that you only have to say if batch.is_non_fatal_errors_active { .. }

conn: &PgConnection,
deployment_id: &DeploymentHash,
health: SubgraphHealth,
non_fatal_errors: Option<Vec<SubgraphError>>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you make this an Option<&[SubgraphError]> the caller doesn't need to clone - it's not a big deal in this case since errors happen rarely, but on hotter code paths it would make a difference

@incrypto32
Copy link
Member Author

One thing I am not clear on is whether fail_subgraph still needs to also store the error - it seems like that is just additional work now?

@lutter in this case fail_subgraph wont get called because the subgraph progresses, but i believe fail_subgraph always did extra work since all errors are being written inside transact_block_operations when insert_subgraph_errors is being called.
What fail_subgraph should ideally be doing is to just calculate the error_id (instead of inserting the error and getting the id back) and then updating this subgraph_deployment table. That is what i have done in update_non_fatal_errors.

As you said since it works and ignores the duplicate insert silenty, we should not touch fail_subgraph anyways atleast for nows?

@incrypto32 incrypto32 force-pushed the incrypto32/fix-non-fatal-errors-not-populating branch 2 times, most recently from 3892a76 to 6838968 Compare May 26, 2023 14:47
@incrypto32 incrypto32 force-pushed the incrypto32/fix-non-fatal-errors-not-populating branch from 6838968 to 92451cc Compare May 26, 2023 15:20
@incrypto32
Copy link
Member Author

@lutter i have added tests, transact some errors and then check that status to check if nonFatalErrors array has something in there.

Also squashed the commits into two.

@incrypto32 incrypto32 requested a review from lutter May 26, 2023 17:40
Copy link
Collaborator

@lutter lutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for adding the tests.

let count = || -> usize {
let store = store.subgraph_store();
let count = store.error_count(&subgraph_id).unwrap();
println!("count: {}", count);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a leftover from debugging.


let info = subgraph_store.status_for_id(deployment.id);

assert!(info.non_fatal_errors.len() == 1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's generally better to use assert_eq!(1, info.non_fatal_errors.len()) for equality comparisons since that will print both the expected and actual values when the assertion fails. It's a small quality-of-life improvement when tests fail, not that big a deal, but would be good to change before merging.

@incrypto32 incrypto32 merged commit 765e683 into master Jun 2, 2023
@incrypto32 incrypto32 self-assigned this Jul 20, 2023
@incrypto32 incrypto32 deleted the incrypto32/fix-non-fatal-errors-not-populating branch March 12, 2024 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants