Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve error handling of read_vector on geojson URL #130

Closed
soxofaan opened this issue Jun 14, 2022 · 2 comments
Closed

improve error handling of read_vector on geojson URL #130

soxofaan opened this issue Jun 14, 2022 · 2 comments
Assignees

Comments

@soxofaan
Copy link
Member

from https://discuss.eodc.eu/t/fit-class-random-forest-413-request-too-large/396:

process graph uses read_vector with a geojson URL like https://eolab.eodc.eu/hub/user-redirect/lab/tree/blabla/features.json, which forwards to a HTML login page (https://eolab.eodc.eu/hub/login?next=%2Fhub%2Fuser-redirect%2Flab%2Ftree%2Fblabla%2Ffeatures.json), which causes the json parsing to fail with

OpenEoApiError: [500] Internal: Server error: JSONDecodeError('Expecting value: line 4 column 1 (char 3)')

Confusingly, the HTML had three empty lines, causing the error to mention "line 4", which gave the impression that the URL at least started with valid JSON, so it took some time to realize that the geojson response wasn't valid JSON at all.

Some improvements to help with debugging this problem: add some logging around read_vector of what URL is loaded, what the content type is, warn if it is not json ( but still allow it: github raw urls have content type "text/plain"), improve the information of the exception (e.g. include the url)

@soxofaan
Copy link
Member Author

Another feature idea: users might use a github url like https://github.com/Open-EO/openeo-python-driver/blob/master/tests/data/geojson/FeatureCollection01.json (which does not work because there is a whole HTML page around the actual json). maybe we can easily convert that to a usable raw URL (https://raw.githubusercontent.com/Open-EO/openeo-python-driver/master/tests/data/geojson/FeatureCollection01.json in this case)

@soxofaan soxofaan self-assigned this Jun 14, 2022
soxofaan added a commit that referenced this issue Jun 14, 2022
Also require "requests>=2.28.0" to avoid excessive JSONDecodeError message size (psf/requests#6036)
@soxofaan
Copy link
Member Author

closing this for now, the current change regarding logging and error message should already be enough of an improvement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant