Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

entire state txt download for uget? #73

Open
duranduran127 opened this issue Jun 12, 2024 · 2 comments
Open

entire state txt download for uget? #73

duranduran127 opened this issue Jun 12, 2024 · 2 comments

Comments

@duranduran127
Copy link

@hobu is it possible to use the aws resources to get all of the links to download lidar for an entire state? I have tried to do this through the data download but it is limited to 5000 items per txt file.

@keythread
Copy link

@duranduran127 It is possible (though not advisable) to use our TNMAccess API to programmatically obtain links for all the available raw lidar (not Entwined) for an entire state with a call like this:

https://tnmaccess.nationalmap.gov/api/v1/products?datasets=Lidar Point Cloud (LPC)&offset=0&max=5&polyCode=22&polyType=state&

where polyCode is a state code found by calling this web service:

https://www.sciencebase.gov/catalog/extents/get?q=Louisiana&extentType="U.S. State"offset=0&fields=name,shortName,extentType,webMercatorBounds,code,abbreviation&

One would need to paginate through the records by calling with different 'offset=' values. I think max=1000 is the maximum number of records returned at a time.

The API will return the download URLs from a non-AWS web server called rockyweb. This is the API the Downloader uses to query for all the National Map product download URLs.

See: TNMAccess API

Unless you are interested in a smaller state I wouldn't recommend this approach due to the volume of files, but it is possible.

@duranduran127
Copy link
Author

@keythread Thanks for the information!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants