Feat graph investigate query time explosion #306
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edznux-dd
approved these changes
Dec 11, 2024
Minosity-VR
reviewed
Dec 11, 2024
Comment on lines
+182
to
+191
| // Create composite indices for the properties we want to search on | ||
| mgmt.buildIndex('byClusterAndRunIDComposite', Vertex.class).addKey(cluster).addKey(runID).buildCompositeIndex(); | ||
| mgmt.buildIndex('byClassAndRunIDComposite', Vertex.class).addKey(cls).addKey(runID).buildCompositeIndex(); | ||
| mgmt.buildIndex('byClassAndClusterComposite', Vertex.class).addKey(cls).addKey(cluster).buildCompositeIndex(); | ||
| mgmt.buildIndex('byClassAndTypeComposite', Vertex.class).addKey(cls).addKey(type).buildCompositeIndex(); | ||
| mgmt.buildIndex('byClassAndExposureComposite', Vertex.class).addKey(cls).addKey(exposure).buildCompositeIndex(); | ||
| mgmt.buildIndex('byTypeAndNameComposite', Vertex.class).addKey(type).addKey(name).buildCompositeIndex(); | ||
| mgmt.buildIndex('byImageAndRunIDComposite', Vertex.class).addKey(image).addKey(runID).buildCompositeIndex(); | ||
| mgmt.buildIndex('byAppAndRunIDComposite', Vertex.class).addKey(app).addKey(runID).buildCompositeIndex(); | ||
| mgmt.buildIndex('byNamespaceAndRunIDComposite', Vertex.class).addKey(namespace).addKey(runID).buildCompositeIndex(); |
There was a problem hiding this comment.
Nice, those will definitely be super useful.
Just by curiosity, do you know if it would have a significant impact on insert time? Or does it stay negligeable?
There was a problem hiding this comment.
For in-memory, I don't see any relevant/noticeable write performance improvements, except for the graph nuking, which is using the index now. It is more visible with concrete datastore and search index backends.
edznux-dd
reviewed
Dec 11, 2024
edznux-dd
reviewed
Dec 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Update JanusGraph to 1.1.0
Various performance-related fixes
Enable the
force-indexflag to preventg.V().count()like queries that could take down the graph by doing complete graph traversal. An error is raised when doing queries without index usage.Enforce a hard limit to
10000results returned to prevent server overload when the request is not limited.Use indices to destroy the graph on the
ingest localcommand call