Feature Request: enabling --polygon also when feeding hgt/geotiff via command line

[keenonkites]

First of all. I'm happy that someone continues with pyhgtmap, would be a pitty.

Today a lot of countries provide better source date for elevation lines than viewfinderpanoramas or srtm type data as open data. We want to use this more precise and open data for creating better elevations.

But having to fiddle arround and put that into a so called 'cache' directory is sort of difficult and I was also not able to recreate the needed index.
If I handover all needed hgt files (or geotiff files) on the command line the elevation lines are calculated for everything handed over.
I want to be able to apply there a polygon with the argument --polygon as possible when running (after downloading) against the cache directory.

In my opinion (and especially for our need) this would be a great enhancement of the tool.

Cheers
Patrik

[agrenott]

Thanks for the feedback.
At a first glance it shouldn't be a big deal. I'll take a look when time permits.

[agrenott]

Released in v3.5.

[keenonkites]

Thanks for implementing.

But unfortunately with v3.5 not even a pyhgtmap --help works:
pyhgtmap --help
Traceback (most recent call last):
File "/home/pab/venv_pyhgtmap/bin/pyhgtmap", line 8, in <module>
sys.exit(main())
TypeError: main() missing 1 required positional argument: 'sys_args'

I can also try to run with proper arguments, same result.

[agrenott]

Sorry about that, I didn't take into account python packaging scripts expectations...
Fixed in v3.5.1

[keenonkites]

Thanks, 3.5.1 runs again also on my side... I'll try to do the functional test this evening.

[keenonkites]

No luck...
I run the command like shown below as before with the hgt files on command line and just added the --polygon option again, but it somehow triggers 'earthexplorer' stuff for downloading hgt....

pyhgtmap call:
pyhgtmap --step=20 --osm-version=0.6 --jobs=1 --line-cat=500,100 --start-node-id=750000000000 --start-way-id=750000000000 --max-nodes-per-tile=0 --pbf --polygon=./poly.work/Freizeitkarte_LUX.poly --write-timestamp --output-prefix=./pbf/ele_20_100_500/Hoehendaten_Freizeitkarte_LUX /home/xxx/sonny-europe/DTM-Luxembourg-3asec-v22-by-Sonny//N49E005.hgt /home/xxx/sonny-europe/DTM-Luxembourg-3asec-v22-by-Sonny//N49E006.hgt /home/xxx/sonny-europe/DTM-Luxembourg-3asec-v22-by-Sonny//N50E005.hgt /home/xxx/sonny-europe/DTM-Luxembourg-3asec-v22-by-Sonny//N50E006.hgt

Error I get (followed by the complete usage):
Need earthexplorer login credentials to continue. See the help for the
--earthexplorer-user and --earthexplorer-password options for details.
...

The option polyfile was before only possible if you let pyhgtmap (or old phyghtmap) manage downloads from SRTM or VIEW on it's own. And it looks as it still triggers that somehow.
But as I have different (not SRTM and not VIEW) source hgt files I don't need, better said, don't want this.

Thanks for re-checking.

Cheers

[extremecarver]

It would be easiest if before Download there is a simple check if file already exists but not in the index file. In that case just add it to the index. Then user simply puts hgt into view1, view3 or srtm1 , srtm3 folder. Right now the solution to do this is to manually put it there, make a copy of all filenames inside the folder, then remove path and .hgt and write that into the index file.
Or add a folder "prefer" and anything there is preferred and indexfile added by comparing content of folder preferred vs requested area, and if inside but not yet in the index file, add it there. Download for that folder is by user.
Command line is not a nice solution!

[agrenott]

It's a limitation I didn't spot in the option parser because my earthexplorer credentials are already stored...

It shouldn't affect the overall process though.
I'll fix it, but in the meantime can you try adding the --source=view1 option to your command line?
This should replace the default earthexplorer source, and allow you to process (with local files only which take precedence) without entering proper credentials.

[keenonkites]

Additional Info: by also handing over earthexplorer user and password I can prevent the tool from failing. Then the result looks so far ok: polygon is applied.

[keenonkites]

It would be easiest if before Download there is a simple check if file already exists but not in the index file. In that case just add it to the index. Then user simply puts hgt into view1, view3 or srtm1 , srtm3 folder. Right now the solution to do this is to manually put it there, make a copy of all filenames inside the folder, then remove path and .hgt and write that into the index file. Or add a folder "prefer" and anything there is preferred and indexfile added by comparing content of folder preferred vs requested area, and if inside but not yet in the index file, add it there. Download for that folder is by user. Command line is not a nice solution!

I agree that handing over lots of files on Command line is not the nicest...
How does this work with the 'prefer' folder ? where does this belong to ? Is there a possibility to have 'prefer3' and 'prefer1' somehow ?

[agrenott]

If you have a lot of local files, you could simply rely on shell wildcard expansion to avoid entering them manually (pyhgtmap somefolder/*.hgt) - at least until you reach max command length...

The main added value I saw when discovering phyghtmap was the automated download from known sources, much more comfortable than manual download. Do the alternate sources you use provide API which could automate the download as well?

I've not touched much the download part, which is still quite a mess, but I was thinking about refactoring/generalizing it one day. Could be the opportunity to add more sources.

[keenonkites]

I am using file globbing with * when creating the command line for pyhgtmap (which is done inside a script), what I've added above was the output from bash -x.
But it's true instead of having them on the command line for the pyhgtmap call it would be better to have them in a folder and only hand this one over to the tool.

We are making tests (www.freizeitkarte-osm.de) with the DTM provided by Sonny https://sonny.4lima.de/
These are more precise and most of the time based on open data provided by the respective countries.... actually the data itself is located on his google drive, which is probably sort of a hassle to automate.

[agrenott]

It's a limitation I didn't spot in the option parser because my earthexplorer credentials are already stored...

Fixed in v3.5.2