Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adaptions to run modules in gui #67

Merged
merged 18 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions .github/workflows/black.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
name: Python code style check with black

on: [push]
Expand All @@ -8,13 +9,13 @@ jobs:
runs-on: ubuntu-22.04

steps:
- uses: actions/checkout@v4
- name: Install apt dependencies
run: |
sudo apt-get update && sudo apt-get install python3 python3-pip -y
- name: Install pip dependencies
run: |
pip3 install black==23.1.0
- name: Check code style with Black
run: |
black --check --diff --line-length 79 .
- uses: actions/checkout@v4
- name: Install apt dependencies
run: |
sudo apt-get update && sudo apt-get install python3 python3-pip -y
- name: Install pip dependencies
run: |
pip3 install black==23.12.0
- name: Check code style with Black
run: |
black --check --diff --line-length 79 .
32 changes: 18 additions & 14 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
name: Python Flake8 Code Quality

on: [push]
Expand All @@ -8,17 +9,20 @@ jobs:
runs-on: ubuntu-22.04

steps:
- uses: actions/checkout@v4
- name: Install apt dependencies
run: |
sudo apt-get update && sudo apt-get install python3 python3-pip -y
- name: Install pip dependencies
run: |
pip3 install flake8 pylint
- name: Lint with Flake8
run: |
# E402: module level import not at top of file
flake8 --ignore=E265,F821,E501,W503,E203,E402 --max-line-length=80 --count --statistics --show-source .
# - name: Lint with Pylint
# run: |
# pylint --disable=R,bad-continuation,fixme,import-error --additional-builtins=_ --reports=n --score=n -j0 ${{ github.event.repository.name }}.py
- uses: actions/checkout@v4
- name: Install apt dependencies
run: |
sudo apt-get update && sudo apt-get install python3 python3-pip -y
- name: Install pip dependencies
run: |
pip3 install flake8 pylint
- name: Lint with Flake8
run: |
# E402: module level import not at top of file
flake8 --ignore=E265,F821,E501,W503,E203,E402 --max-line-length=80 \
--count --statistics --show-source .
# - name: Lint with Pylint
# run: |
# pylint --disable=R,bad-continuation,fixme,import-error \
# --additional-builtins=_ --reports=n --score=n \
# -j0 ${{ github.event.repository.name }}.py
2 changes: 1 addition & 1 deletion .github/workflows/super-linter.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
name: General linting

on:
- push
- pull_request
linakrisztian marked this conversation as resolved.
Show resolved Hide resolved

jobs:
super-linter:
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ ENV GRASS_ADDON_BASE=/usr/local/grass84

# install external dependencies
# RUN apt install musl-dev ??
RUN pip3 install py7zr tqdm requests psutil scikit-learn pyproj
RUN pip3 install py7zr tqdm requests psutil scikit-learn pyproj pandas

# install official addons
RUN grass --tmp-location EPSG:4326 --exec g.extension r.mapcalc.tiled -s
Expand Down
88 changes: 59 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,75 @@
# rvr_interface

Repo for code and script transfer between mundialis and RVR - GRASS GIS addons:

- **m.import.rvr** imports data for the processing of <b>gebaeudedetektion</b>,
<b>dachbegruenung</b> and/or <b>einzelbaumerkennung</b>.
<b>dachbegruenung</b> and/or <b>einzelbaumerkennung</b>.

- **r.import.dtm_nrw** downloads and imports the NRW digital terrain model
(DTM) 1m into the current mapset. Only the extent of the current region is
downloaded and imported with a 1m resolution.
(DTM) 1m into the current mapset. Only the extent of the current region is
downloaded and imported with a 1m resolution.

- **r.import.ndsm_nrw** calculates an nDSM by subtracting input digital terrain
model (DTM) data (defined by the <b>dtm</b> parameter) from an input DSM
indicated by the <b>dsm</b> parameter. If no DTM is defined, NRW DTM data
is automatically imported using <a href="grass-gis-addons/r.import.dtm_nrw/r.import.dtm_nrw.html">r.import.dtm_nrw</a>.
model (DTM) data (defined by the <b>dtm</b> parameter) from an input DSM
indicated by the <b>dsm</b> parameter. If no DTM is defined, NRW DTM data
is automatically imported using <a href="grass-gis-addons/r.import.dtm_nrw/r.import.dtm_nrw.html">r.import.dtm_nrw</a>.

- **r.in.pdal.worker** is a worker addon for <em>r.in.pdal</em> and is used in
<a href="grass-gis-addons/m.import.rvr/m.import.rvr.html">m.import.rvr</a>.

<a href="grass-gis-addons/m.import.rvr/m.import.rvr.html">m.import.rvr</a>.

- **m.analyse.buildings**

- **r.extract.buildings** extracts buildings as vectors and calculates height
statistics (minimum, maximum, average, standard deviation, median,
percentile) and presumable number of stories using an nDSM-raster,
NDVI-raster, and FNK-vector (Flaechennutzungskatalog).
statistics (minimum, maximum, average, standard deviation, median,
percentile) and presumable number of stories using an nDSM-raster,
NDVI-raster, and FNK-vector (Flaechennutzungskatalog).
- **r.extract.buildings.worker** is a worker module that is started by
<a href="grass-gis-addons/m.analyse.buildings/r.extract.buildings/r.extract.buildings.html">r.extract.buildings</a>.
<a href="grass-gis-addons/m.analyse.buildings/r.extract.buildings/r.extract.buildings.html">r.extract.buildings</a>.
- **v.cd.areas** calculates differences between two vector layers
(e.g. classification and reference) by making use of v.overlay with operator
"xor". Only differences with a defined minimum size are extracted.
(e.g. classification and reference) by making use of v.overlay with operator
"xor". Only differences with a defined minimum size are extracted.
- **v.cd.areas.worker** is a worker module that is started by
<a href="grass-gis-addons/m.analyse.buildings/v.cd.areas/v.cd.areas.html">v.cd.areas</a>.
<a href="grass-gis-addons/m.analyse.buildings/v.cd.areas/v.cd.areas.html">v.cd.areas</a>.
- **r.extract.greenroofs** extracts vegetated roofs from aerial photographs,
an nDSM, a building vector layer and optionally an FNK
(Flaechennutzungskatalog) and tree vector layer.
an nDSM, a building vector layer and optionally an FNK
(Flaechennutzungskatalog) and tree vector layer.
- **r.extract.greenroofs.worker** is a worker module that is started by
<a href="grass-gis-addons/m.analyse.buildings/r.extract.greenroofs/r.extract.greenroofs.html">r.extract.greenroofs</a>.

<a href="grass-gis-addons/m.analyse.buildings/r.extract.greenroofs/r.extract.greenroofs.html">r.extract.greenroofs</a>.

- **m.analyse.trees**

- **r.trees.peaks** assigns pixels to nearest peak (tree crown).
- **r.trees.mltrain** generates training data for a machine learning (ML)
model to detect trees and trains the model with these training data.
model to detect trees and trains the model with these training data.
- **r.trees.mlapply** applies the tree classification model
in parallel to the area of interest (current region).
in parallel to the area of interest (current region).
- **r.trees.mlapply.worker** applies classification model for a region
defined by a vector. This module should not be called directly, instead
it is called in parallel by <a href="grass-gis-addons/m.analyse.trees/r.trees.mlapply/r.trees.mlapply.html">r.trees.mlapply</a>.
defined by a vector. This module should not be called directly, instead
it is called in parallel by <a href="grass-gis-addons/m.analyse.trees/r.trees.mlapply/r.trees.mlapply.html">r.trees.mlapply</a>.
- **r.trees.postprocess** generates single tree delineations from tree pixels
and geomorphological peaks.
and geomorphological peaks.
- **v.trees.param** calculates various tree parameters for tree crowns given
as input vector map <b>treecrowns</b>.
as input vector map <b>treecrowns</b>.
- **v.trees.param.worker** is used within <a href="grass-gis-addons/m.analyse.trees/v.trees.param/v.trees.param.html">v.trees.param</a> to calculate various tree parameters
for tree crowns in parallel.
for tree crowns in parallel.
- **v.trees.species** classifies trees in deciduous and coniferous trees.
- **v.trees.cd** calculates the change between two given treecrown vector
maps (<b>inp_t1</b> and <b>inp_t2</b> for time t1 and t2, respectively).
maps (<b>inp_t1</b> and <b>inp_t2</b> for time t1 and t2, respectively).
- **v.trees.cd.worker** is a worker module that is started by
<a href="grass-gis-addons/m.analyse.trees/v.trees.cd/v.trees.cd.html">v.trees.cd</a>.
<a href="grass-gis-addons/m.analyse.trees/v.trees.cd/v.trees.cd.html">v.trees.cd</a>.

## Building and running a docker image

In the folder with the Dockerfile, run

```bash
docker build -t rvr_interface:latest .
```

Instead of "latest", a version number can be used. This should create a local
docker image with all needed addons and dependencies. Once the docker image
has been created locally, it can be started with e.g.
has been created locally, it can be started on Linux with e.g.

```bash
xhost local:*
docker run -it --privileged --rm --ipc host \
Expand All @@ -77,3 +80,30 @@ docker run -it --privileged --rm --ipc host \
--device="/dev/dri/card0:/dev/dri/card0" \
rvr_interface:latest bash
```

On Windows you need to do the following things before starting the docker container:
anikaweinmann marked this conversation as resolved.
Show resolved Hide resolved

1. Install Docker Desktop
2. Download and install [VcXsrv Windows X Server](https://sourceforge.net/projects/vcxsrv/)
3. Start **Xlaunch** and configure it (see [here](https://dev.to/darksmile92/run-gui-app-in-linux-docker-container-on-windows-host-4kde)):
- in the "Extra Settings" window enable "Disable access control"
- in the "Finish Configuration" window click "Save configuration" and save it e.g. on the desktop

Now you can run the docker:

```bash
# get own IP adress (take the value of IPAdress e.g. 10.211.55.10 and not 127.0.0.1)
Get-NetIPAddress
ipconfig
anikaweinmann marked this conversation as resolved.
Show resolved Hide resolved

# set DISPLAY variable (set <YOUR-IP>)
set-variable -name DISPLAY -value <YOUR-IP>:0.0

# start Docker
docker run -it --privileged --rm --ipc host \
-v C:/Users/path/to/grassdata:/grassdb \
-v C:/Users/path/to/rvr_daten:/mnt/data \
--env DISPLAY=$DISPLAY \
--device="/dev/dri/card0:/dev/dri/card0" \
rvr_interface:latest bash
```
Original file line number Diff line number Diff line change
Expand Up @@ -453,21 +453,21 @@ def main():
except Exception:
grass.fatal("m.analyse.buildings library is not installed")

ndsm = options["ndsm"]
ndvi = options["ndvi_raster"]
ndsm = options["ndsm"].split("@")[0]
ndvi = options["ndvi_raster"].split("@")[0]
min_size = options["min_size"]
max_fd = options["max_fd"]
ndvi_thresh = options["ndvi_thresh"]
memory = options["memory"]
output = options["output"]
new_mapset = options["new_mapset"]
area = options["area"]
area = options["area"].split("@")[0]

if options["fnk_vector"]:
fnk_vect = options["fnk_vector"]
fnk_vect = options["fnk_vector"].split("@")[0]
fnk_column = options["fnk_column"]
elif options["fnk_raster"]:
fnk_rast = options["fnk_raster"]
fnk_rast = options["fnk_raster"].split("@")[0]

grass.message(_(f"Applying building extraction to region {area}..."))

Expand All @@ -478,8 +478,10 @@ def main():
ndsm += f"@{old_mapset}"
ndvi += f"@{old_mapset}"
if options["fnk_vector"]:
fnk_name = fnk_vect
fnk_vect += f"@{old_mapset}"
if options["fnk_raster"]:
fnk_name = fnk_rast
fnk_rast += f"@{old_mapset}"

grass.run_command(
Expand Down Expand Up @@ -511,13 +513,13 @@ def main():

# copy FNK to temporary mapset
if options["fnk_vector"]:
fnk_vect_tmp = f'{options["fnk_vector"]}_{os.getpid()}'
fnk_vect_tmp = f"{fnk_name}_{os.getpid()}"
rm_vectors.append(fnk_vect_tmp)
grass.run_command(
"g.copy", vector=f"{fnk_vect},{fnk_vect_tmp}", quiet=True
)
elif options["fnk_raster"]:
fnk_rast_tmp = f'{options["fnk_raster"]}_{os.getpid()}'
fnk_rast_tmp = f"{fnk_name}_{os.getpid()}"
rm_rasters.append(fnk_rast_tmp)
grass.run_command(
"g.copy", raster=f"{fnk_rast},{fnk_rast_tmp}", quiet=True
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -330,10 +330,18 @@ def main():
msg = proc.outputs["stderr"].value.strip()
grass.message(_(f"\nLog of {proc.get_bash()}:"))
for msg_part in msg.split("\n"):
grass.message(_(msg_part))
if msg_part != "":
grass.message(_(msg_part))
# create mapset dict based on Log, so that only those with output are listed
if "Skipping..." not in msg:
tile_output = re.search(r"Output is:\n<(.*?)>", msg).groups()[0]
try:
# for execution in terminal
tile_output = re.search(r"Output is:\n<(.*?)>", msg).groups()[
0
]
except Exception:
# for execution in GUI
tile_output = re.search(r"Output is: <(.*?)>", msg).groups()[0]
output_list.append(tile_output)

# verify that switching back to original mapset worked
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -237,15 +237,43 @@ def main():
new_mapset = options["new_mapset"]
gisrc, newgisrc, old_mapset = switch_to_new_mapset(new_mapset)

area = f"{options['area']}@{old_mapset}"
area = (
f"{options['area']}@{old_mapset}"
if "@" not in options["area"]
else options["area"]
)
num = options["area"].rsplit("_", 1)[1]
building_outlines = f"{options['building_outlines']}@{old_mapset}"
building_outlines = (
f"{options['building_outlines']}@{old_mapset}"
if "@" not in options["building_outlines"]
else options["building_outlines"]
)
buildings = f"{options['buildings']}@{old_mapset}"
ndsm = f"{options['ndsm']}@{old_mapset}"
ndvi = f"{options['ndvi']}@{old_mapset}"
green_blue_ratio = f"{options['gb_ratio']}@{old_mapset}"
red_green_ratio = f"{options['rg_ratio']}@{old_mapset}"
brightness = f"{options['brightness']}@{old_mapset}"
ndsm = (
f"{options['ndsm']}@{old_mapset}"
if "@" not in options["ndsm"]
else options["ndsm"]
)
ndvi = (
f"{options['ndvi']}@{old_mapset}"
if "@" not in options["ndvi"]
else options["ndvi"]
)
green_blue_ratio = (
f"{options['gb_ratio']}@{old_mapset}"
if "@" not in options["gb_ratio"]
else options["gb_ratio"]
)
red_green_ratio = (
f"{options['rg_ratio']}@{old_mapset}"
if "@" not in options["rg_ratio"]
else options["rg_ratio"]
)
brightness = (
f"{options['brightness']}@{old_mapset}"
if "@" not in options["brightness"]
else options["brightness"]
)
min_veg_size = float(options["min_veg_size"])
min_veg_proportion = int(options["min_veg_proportion"])
output_vegetation = options["output_vegetation"]
Expand Down
12 changes: 10 additions & 2 deletions grass-gis-addons/m.analyse.buildings/v.cd.areas/v.cd.areas.py
Original file line number Diff line number Diff line change
Expand Up @@ -322,10 +322,18 @@ def main():
msg = proc.outputs["stderr"].value.strip()
grass.message(_(f"\nLog of {proc.get_bash()}:"))
for msg_part in msg.split("\n"):
grass.message(_(msg_part))
if msg_part != "":
grass.message(_(msg_part))
# create mapset dict based on Log, so that only those with output are listed
if "Skipping..." not in msg:
tile_output = re.search(r"Output is:\n<(.*?)>", msg).groups()[0]
try:
# for execution in terminal
tile_output = re.search(r"Output is:\n<(.*?)>", msg).groups()[
0
]
except Exception:
# for execution in GUI
tile_output = re.search(r"Output is: <(.*?)>", msg).groups()[0]
output_list.append(tile_output)
if flags["q"]:
area_identified = re.search(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ def main():
except Exception:
grass.fatal("m.analyse.trees library is not installed")

area = options["area"]
group = options["group"]
area = options["area"].split("@")[0]
group = options["group"].split("@")[0]
output = options["output"]
model = options["model"]
new_mapset = options["new_mapset"]
Expand Down
Loading