Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create raster tables with geography and metadata #105

Merged
merged 38 commits into from
Feb 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
1efecb5
WIP: precompute geog
jgoizueta Feb 7, 2023
c129e6b
Fix antimeridian-crossing
jgoizueta Feb 8, 2023
67fb05a
Make pseudo_planar an optional parameter
jgoizueta Feb 8, 2023
1897b39
Refactor to prepare metadata
jgoizueta Feb 8, 2023
43293c0
Create table with specific column types before uploading data
jgoizueta Feb 8, 2023
f104057
Write metadata in the raster table
jgoizueta Feb 8, 2023
f6bd0e5
Add some of the missing metadata
jgoizueta Feb 8, 2023
6d4d147
Add some of the missing metadata
jgoizueta Feb 8, 2023
f994465
Embed geojson in json properly
jgoizueta Feb 8, 2023
7b2e1d1
Fix geometry orientation
jgoizueta Feb 8, 2023
efd578b
Add comment
jgoizueta Feb 9, 2023
888e0d0
Support negatively oriented rasters
jgoizueta Feb 9, 2023
208f75a
Remove comments
jgoizueta Feb 9, 2023
0aa914c
Add areas to metadata
jgoizueta Feb 9, 2023
acfd99c
Remove comment
jgoizueta Feb 9, 2023
bfe3e0a
We don't need so many data
jgoizueta Feb 10, 2023
96bcebd
Prevent problem with ST_GEOGFROMGEOJSON
jgoizueta Feb 10, 2023
7bff268
Fix incorrect quadbin tile was being used for block area
jgoizueta Feb 10, 2023
bb1fcea
Avoid BQ API warnings and make method arguments consistent
jgoizueta Feb 10, 2023
ab8a83c
Add integration test
jgoizueta Feb 10, 2023
54b6ec5
Fix for loanding whole-earth rasters
jgoizueta Feb 10, 2023
f5c23db
Add integration test for quadbin raster
jgoizueta Feb 10, 2023
d403a9b
Update doc strings
jgoizueta Feb 13, 2023
2577b89
Use cell_area from quadbin-py
jgoizueta Feb 13, 2023
4705d2c
Fix bug
jgoizueta Feb 13, 2023
de3373f
Merge metadata when appending to a raster table
jgoizueta Feb 13, 2023
4d4acc0
change tolerance for dup points
jgoizueta Feb 14, 2023
afacaef
Leave null areas for invalid geometries
jgoizueta Feb 14, 2023
0c0da59
Fix metadata field names
jgoizueta Feb 14, 2023
9d91dd3
Remove unnecessary config
jgoizueta Feb 14, 2023
9485950
Fix bug
jgoizueta Feb 14, 2023
5b2a1ba
Reduce subdivisions with latitude scale factor
jgoizueta Feb 14, 2023
4d5b99b
Avoid unnecessary subdivisions
jgoizueta Feb 14, 2023
6026429
Use wkt for raster_boundary metadata
jgoizueta Feb 15, 2023
c83ac86
Sum block areas for quadbin
jgoizueta Feb 15, 2023
1878afd
Restric scale factor lines close to parallels
jgoizueta Feb 15, 2023
efb9cc3
Prevent spherical geometry conversion problems in BQ
jgoizueta Feb 15, 2023
3aa7db0
Fix raster boundary
jgoizueta Feb 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,14 @@ The test suite includes optional integration tests that require credentials for
BigQuery account. To run these tests, you need to set the `GOOGLE_APPLICATION_CREDENTIALS`
environment variable to the path of a JSON file containing your BigQuery credentials
(see the [GCP documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key)
for more information). After setting up your credentials, you can enable the integration
for more information).

You must also copy the `/test/.env.sample` to `/test/.env` and edit it to set a
test project and dataset in which the used credentials have permissions to create tables.

If you're working on Windows, please set manually the env variables or the .env file in your terminal.

After setting up your credentials and .env, you can enable the integration
test with the following command:

```bash
Expand Down
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ lint:
test:
$(BIN)/pytest raster_loader --cov=raster_loader --verbose

test-integration:
$(BIN)/pytest raster_loader --cov=raster_loader --verbose --runintegration

docs:
cd docs; make clean html

Expand Down
Loading