Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subset dataframe for locallayer to only used columns and util cols #551

Closed
andy-esch opened this issue Mar 1, 2019 · 3 comments · Fixed by #1186
Closed

subset dataframe for locallayer to only used columns and util cols #551

andy-esch opened this issue Mar 1, 2019 · 3 comments · Fixed by #1186
Assignees

Comments

@andy-esch
Copy link
Contributor

andy-esch commented Mar 1, 2019

Besides the_geom, the_geom_webmercator(?), cartodb_id, the set should include any involved in interactivity, and any involved in styling.
I think the only column needed is the geodataframe geometry column.

To get styling variables, something of the form could be useful:

import re

s1 = 'ramp($colname, sunset)'
s2 = 'ramp(globalQuantiles($colname, 7), sunset)'

m = re.search('\$([a-z_]+)', s1)
assert m.group(1) == 'colname'

m = re.search('\$([a-z_]+)', s2)
assert m.group(1) == 'colname'
@Jesus89
Copy link
Member

Jesus89 commented Sep 30, 2019

Could you clarify this a bit @andy-esch ?

@andy-esch
Copy link
Contributor Author

When the geodataframe is sent to the JS environment, we do gdf.to_json() to get the full geojson. What happens most of the time is that the user only needs a subset of the data: geometry, columns used in styling/widgets/popups.

What could probably be a performance gain is that if we could figure out how to select only the columns needed for sending to the JS environment:

gdf[['geometry', 'stylecol1', 'widgetcol']].to_json()

@Jesus89
Copy link
Member

Jesus89 commented Sep 30, 2019

OK. Thanks for the clarification.

This is the behavior when VL loads data from a CARTO dataset, but this can only be applied when there is a Maps API instantiation. So I like your proposal of filtering the geodataframe before passing it to the visualization. We could potentially save a lot of memory size!

I'm not sure about the implementation, but I'll definitely check this for the 1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants