Replies: 1 comment
-
Sounds like a good plan! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In #3803 we are adding 2 modes of preFetching properties:
all_properties
- retrieves all properties via a single slice query.required_properties_only
- retrieves only requested properties but via separate slice query per each property.This is a trade off a user needs to choose right now. Either using multiple slice queries (probably worse performance for cases when there are not too much properties or they are small) or use a single slice query but fetch all vertex properties which might fetch too much redundant data in some cases.
Here I explained why it's not possible to have the best from both worlds right now (i.e. why it's not possible to retrieve only requested properties via a single slice query yet).
That said, it seems we can implement a slightly smarter mode which always uses a single slice query and fetches all properties from the most left requested property to the most right requested property (sorted by key ids, of course).
For example, let say the vertex have the next properties in the row:
prop1
,prop2
,prop3
,prop4
,prop5
,prop6
,prop7
,prop8
.Let's say that the user sends the next Gremlin query:
g.V(vertexId).valueMap("prop4", "prop6", "prop2")
. Now if we try to find the most left property and the most right property from the requested list, we are getting:"prop2" - the most left key (or the key with the smallest
id
)."prop6" - the most right key (or the key with the highest
id
).In this situation we definitely know that the user isn't interested in any properties which might come before
prop2
and we definitely know that the user isn't interested in any properties which might come afterprop6
. Thus, our single slice query can be reduced to retrieve all properties betweenprop2
andprop6
. I.e., the the case withg.V(vertexId).valueMap("prop4", "prop6", "prop2")
we will retrieve propertiesprop2
,prop3
,prop4
,prop5
,prop6
, but we will not retrieve propertiesprop1
,prop7
,prop8
.Of course, in the worst case, when the user sends a request like
g.V(vertexId).valueMap("prop1", "prop8")
- we will need to retrieve ALL properties if we want to use just a single slice query.This technique won't improve worst case scenarios, but it might well improve average case scenarios.
@li-boxuan Since you had a good dive into the storage layer data-structure, I wander if you have any thoughts about this idea or weather you thought about it before.
cc @FlorianHockmann since you had related question here.
Beta Was this translation helpful? Give feedback.
All reactions