Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t.rast.univar, t.rast3d.univar: Add support for zones #2588

Merged
merged 39 commits into from
Oct 24, 2022

Conversation

ninsbl
Copy link
Member

@ninsbl ninsbl commented Sep 20, 2022

Inspired by #1474, this PR adds support for a zones map in t.rast.univar and t.rast3d.univar, covered by tests.

Maybe also semantic_labels should be added to the output (if the STRDS contains some)? What do you think. Happy to add it if you consider it useful.

Should I add a check if the zones map is of type CELL?

Of course, I don` expect you all to review, but I would be happy about some feedback, esp. with regards to the two questions above.

@ninsbl ninsbl added enhancement New feature or request temporal Related to temporal data processing Python Related code is in Python labels Sep 20, 2022
@ninsbl ninsbl added this to the 8.3.0 milestone Sep 20, 2022
@ninsbl
Copy link
Member Author

ninsbl commented Sep 20, 2022

P.S.: Like #1474, this implements only a single, static raster map for zones. That is probably the most common use case (at least it is for me). However, using another STRDS with a defined temporal relation could be interesting as well, but would be a different PR...

@veroandreo
Copy link
Contributor

Inspired by #1474, this PR adds support for a zones map in t.rast.univar and t.rast3d.univar, covered by tests.

Maybe also semantic_labels should be added to the output (if the STRDS contains some)? What do you think. Happy to add it if you consider it useful.

I think it would be useful, especially for STRDS of multiple bands. However, we faced a problem recently when teaching at FOSS4G, if you estimate NDVI with t.rast.mapcalc using a STRDS of S2 b4 and b8, the inherited semantic label is b8 (Section 10 here)... if we then ask for univar stats, this will also inherit an incorrect semantic label. In any case, it would be a nice addition.

Should I add a check if the zones map is of type CELL?

Sounds reasonable to me. Are there use cases in which zones might be float?

Time series of zones would be cool too...

@ninsbl
Copy link
Member Author

ninsbl commented Sep 25, 2022

Thanks for all the testing you did (also on addons!).
The semantic_labels and time series for zones are likely issues for different PRs.
Will address the CELL-check ASAP.

@ninsbl
Copy link
Member Author

ninsbl commented Sep 26, 2022

STRDS as input for zones could probably be implemented like https://github.com/OSGeo/grass/blob/releasebranch_8_2/temporal/t.rast.aggregate.d

However, a more general question then is if this should be implemented as a separate module (e.g. t.rast.univar.ds) or as part of t.rast.univar. Both options have pros and cons, but I am leaning towards implementing that as a new module (if there isn't an add on already)...

@ninsbl
Copy link
Member Author

ninsbl commented Sep 26, 2022

Sounds reasonable to me. Are there use cases in which zones might be float?

The point of such a check would be mostly to catch a user error, as r.univar would refuse ro work.

Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small style changes. I did not check the logic.

temporal/t.rast.univar/t.rast.univar.py Outdated Show resolved Hide resolved
temporal/t.rast3d.univar/t.rast3d.univar.py Outdated Show resolved Hide resolved
temporal/t.rast3d.univar/testsuite/test_t_rast3d_univar.py Outdated Show resolved Hide resolved
@ninsbl
Copy link
Member Author

ninsbl commented Oct 7, 2022

Thanks for the thorough review and feedback. Will address this later today.

@ninsbl
Copy link
Member Author

ninsbl commented Oct 7, 2022

Hm... Any Ctypes import seems to break the CentOS 7 build...
Is it worth keeping CentOS 7 tests in the light of that? If I understand correctly CentOS transitioned to CentOS Stream. Unfortunately, newer CentOS images, like Stream, are not available in GitHub CI out of the box...

@ninsbl ninsbl requested a review from wenzeslaus October 9, 2022 21:08
@ninsbl
Copy link
Member Author

ninsbl commented Oct 9, 2022

Tests are passing for ubuntu: https://github.com/OSGeo/grass/pull/2588/checks#step:10:216 and https://github.com/OSGeo/grass/pull/2588/checks#step:10:222
However, as mentioned CetnOS build tests fail when importing tgis: https://github.com/OSGeo/grass/actions/runs/3206937539/jobs/5241279653#step:11:9830
That worked before, so failing tests may be due to the order of when the import occurs?

@wenzeslaus
Copy link
Member

Any Ctypes import seems to break the CentOS 7 build... Is it worth keeping CentOS 7 tests in the light of that?

I would be reluctant to remove CentOS just because one module failure to build when the other modules work.

Unfortunately, newer CentOS images, like Stream, are not available in GitHub CI out of the box...

Neither is CentOS 7, we are running it using a Docker image. Are there no Docker images for Stream?

With this said, the CentOS was always on shaky ground. It is partially using conda to get acceptable dependencies, so it is a poor test of CentOS. I was hoping it will move to more pure CentOS, but that did not happen yet. However, it does catch things which are related to robustness in sub-optimal environments.

@neteler
Copy link
Member

neteler commented Oct 10, 2022

Neither is CentOS 7, we are running it using a Docker image. Are there no Docker images for Stream?

We could use AlmaLinux:
https://hub.docker.com/_/almalinux

@ninsbl
Copy link
Member Author

ninsbl commented Oct 10, 2022

I would be reluctant to remove CentOS just because one module failure to build when the other modules work.

Fair enough. And with your suggestion, tests are no longer failing. So I agree, best to keep it. Maybe improve the CentOS test in future, e.g. with the docker containers @neteler suggested...

@ninsbl
Copy link
Member Author

ninsbl commented Oct 13, 2022

OK to merge? With this in, I can take on parallelization...

Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing my concerns. The code is agreeable in general and it has tests. I did not review the functionality or logic.

If at least one of the other suggested reviewers reviews the logic or tests, that would be great. @lucadelu @neteler @NikosAlexandris @veroandreo

@ninsbl
Copy link
Member Author

ninsbl commented Oct 21, 2022

@lucadelu @neteler @NikosAlexandris @veroandreo if one of you has a chance to test, that would be very much appreciated? Otherwise, we can trust CI and merge the next days...

@lucadelu
Copy link
Contributor

@ninsbl sorry but in this period I have really no time to do this

@ninsbl
Copy link
Member Author

ninsbl commented Oct 24, 2022

@lucadelu , fully understand. Given that the module is covered by tests and results in tests are unchanged, I merge the change.

@ninsbl ninsbl merged commit 0e6fd7c into OSGeo:main Oct 24, 2022
@ninsbl ninsbl deleted the t_rast_univar_zones branch October 26, 2022 07:38
ninsbl added a commit to ninsbl/grass that referenced this pull request Feb 17, 2023
* add support for zones

* add support for zones

* add test for zones

* clean properly

* add test for zones

* add support for zones

* add credits

* add credits

* add zones in manual

* add check for zones map

* black

* use RasterRow context manager

* fix zones existence test

* try if centos fails because of context manager

* fix indent

* change import order

* zones check for t.rast3d.univar

* avoid RasterRow

* avoid RasterRow

* fix if

* fix if

* fix raster_info

* fix raster_info

* fix raster_info

* fix raster_info

* import as gs

* import as gs and zones

* string formating

* clean test

* add semantic labels

* black

* remove gscript

* name tests

* name tests

* move parser, use kwargs

* move parser, use kwargs, allow DCELL

* Import tgis after parser

* Import tgis after parser

* Fix typo
marisn pushed a commit to marisn/grass that referenced this pull request Jun 2, 2023
* add support for zones

* add support for zones

* add test for zones

* clean properly

* add test for zones

* add support for zones

* add credits

* add credits

* add zones in manual

* add check for zones map

* black

* use RasterRow context manager

* fix zones existence test

* try if centos fails because of context manager

* fix indent

* change import order

* zones check for t.rast3d.univar

* avoid RasterRow

* avoid RasterRow

* fix if

* fix if

* fix raster_info

* fix raster_info

* fix raster_info

* fix raster_info

* import as gs

* import as gs and zones

* string formating

* clean test

* add semantic labels

* black

* remove gscript

* name tests

* name tests

* move parser, use kwargs

* move parser, use kwargs, allow DCELL

* Import tgis after parser

* Import tgis after parser

* Fix typo
@wenzeslaus wenzeslaus changed the title t.rast.univar / t.rast3d.univar: Add support for zones t.rast.univar, t.rast3d.univar: Add support for zones Jun 6, 2023
neteler pushed a commit to nilason/grass that referenced this pull request Nov 7, 2023
* add support for zones

* add support for zones

* add test for zones

* clean properly

* add test for zones

* add support for zones

* add credits

* add credits

* add zones in manual

* add check for zones map

* black

* use RasterRow context manager

* fix zones existence test

* try if centos fails because of context manager

* fix indent

* change import order

* zones check for t.rast3d.univar

* avoid RasterRow

* avoid RasterRow

* fix if

* fix if

* fix raster_info

* fix raster_info

* fix raster_info

* fix raster_info

* import as gs

* import as gs and zones

* string formating

* clean test

* add semantic labels

* black

* remove gscript

* name tests

* name tests

* move parser, use kwargs

* move parser, use kwargs, allow DCELL

* Import tgis after parser

* Import tgis after parser

* Fix typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Python Related code is in Python temporal Related to temporal data processing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants