Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a graphman prune command #3665

Closed
lutter opened this issue Jun 15, 2022 · 2 comments
Closed

Add a graphman prune command #3665

lutter opened this issue Jun 15, 2022 · 2 comments
Assignees

Comments

@lutter
Copy link
Collaborator

lutter commented Jun 15, 2022

The command will have the form graphman prune <subgraph> <offset> and remove all entity versions that were deleted or updated at least offset many blocks before the current subgraph head; for a pruned subgraph, the index-node API will report that block number as the earliest bock, and queries at a block height before that block will fail with an error. Pruning will not affect query results for queries at a block after the pruned block, and therefore simply limits how far back time-travel queries can reach.

Since pruning removes a huge amount of data that is usually not accessed by queries, it speeds up queries significantly.

As part of this issue, pruning will be a one-time action, i.e., it only removes history at the point in time when it is run.

@azf20
Copy link
Contributor

azf20 commented Jun 16, 2022

@lutter on the "earliest block", I wonder if that might break some application's assumptions (which might rely on that for example to identify how synced a subgraph is)

@lutter
Copy link
Collaborator Author

lutter commented Jun 16, 2022

@lutter on the "earliest block", I wonder if that might break some application's assumptions (which might rely on that for example to identify how synced a subgraph is)

The earliest block number would still be accurate, it's just that the hash for the earliest block gets filled with a dummy value. Right now, that value is always the start block for the subgraph; with pruning the block number could/would move up over time. But I have no idea if anybody is relying on the block hash for anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants