feat: generate initial collection-related STAC metadata TDE-1300 #1124
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amfage
changed the title
amfage
requested review from
l0b0,
schmidtnz,
MDavidson17,
paulfouquet and
tawera-manaena
l0b0
reviewed
Nov 4, 2024
paulfouquet
reviewed
Nov 4, 2024
amfage
changed the title
amfage
changed the title
paulfouquet
reviewed
Nov 5, 2024
Co-authored-by: paulfouquet <86932794+paulfouquet@users.noreply.github.com>
paulfouquet
reviewed
Nov 5, 2024
paulfouquet
approved these changes
Nov 5, 2024
l0b0
approved these changes
Nov 6, 2024
Merged
This was referenced Nov 11, 2024
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 13, 2024
🤖 I have created a release *beep* *boop* --- ## [4.8.0](v4.7.0...v4.8.0) (2024-11-12) ### Features * attempt to remove slivers by dropping small polygons TDE-1130 ([#1119](#1119)) ([aa1f931](aa1f931)) * generate initial collection-related STAC metadata TDE-1300 ([#1124](#1124)) ([e30dff9](e30dff9)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Nov 14, 2024
github-merge-queue bot
pushed a commit
to linz/imagery
that referenced
this pull request
Nov 14, 2024
### Motivation An update to `topo-imagery` ([PR #1152](linz/topo-imagery#1152)) and `argo-tasks` ([PR #1124](linz/argo-tasks#1124)) will add the the `linz:slug` attribute new `collection.json` files and expect this attribute to be present with [release 4.8.0](linz/argo-tasks#1127). This pull request is to manually (one-off) add the new `linz:slug` attribute to all existing `collection.json` files in the `elevation` and `imagery` repositories. Original motivation for this change is from BM-1076. ### Modifications Added `linz:slug` attribute to each `collection.json` file from its location in this repository. ``` #!/bin/bash for dir in ./stac/*/*; do find $dir -type f -name 'collection.json' -print -exec sed -i 's; "extent": {; "linz:slug": "'${dir#./stac/*/}'",\n "extent": {;g' {} \; done ``` ### Verification Checked for invalid `json` using `jq` ``` echo "Checking for invalid json files" shopt -s globstar for file in ./**/collection.json; do jq . "${file}" >/dev/null || echo "failed: ${file}" done echo "DONE" ``` Checked for files containing multiple `linz:slug` attributes: ``` echo "Files with number of slugs not equal to 1:" grep -R --include='collection.json' 'linz:slug' . -c --color=no | grep -v 'n:1$' echo "END OF LIST" ```
github-merge-queue bot
pushed a commit
to linz/elevation
that referenced
this pull request
Nov 14, 2024
### Motivation An update to `topo-imagery` ([PR #1152](linz/topo-imagery#1152)) and `argo-tasks` ([PR #1124](linz/argo-tasks#1124)) will add the the `linz:slug` attribute new `collection.json` files and expect this attribute to be present with [release 4.8.0](linz/argo-tasks#1127). This pull request is to manually (one-off) add the new `linz:slug` attribute to all existing `collection.json` files in the `elevation` and `imagery` repositories. Original motivation for this change is from BM-1076. ### Modifications Added `linz:slug` attribute to each `collection.json` file from its location in this repository. ``` #!/bin/bash for dir in ./stac/*/*; do find $dir -type f -name 'collection.json' -print -exec sed -i 's; "extent": {; "linz:slug": "'${dir#./stac/*/}'",\n "extent": {;g' {} \; done ``` ### Verification Checked for invalid `json` using `jq` ``` echo "Checking for invalid json files" shopt -s globstar for file in ./**/collection.json; do jq . "${file}" >/dev/null || echo "failed: ${file}" done echo "DONE" ``` Checked for files containing multiple `linz:slug` attributes: ``` echo "Files with number of slugs not equal to 1:" grep -R --include='collection.json' 'linz:slug' . -c --color=no | grep -v 'n:1$' echo "END OF LIST" ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Set up metadata that is required for the standardising workflow:
linz-slug: required for Basemaps URL creation, will also be used for generating the ODR path.collection-id: required for creating STAC metadataModification
Add a new command,
stac-setupto create these parameter files for Argo Workflows to use.The
stac-setupcommand is based on the generate-path command. Once thelinz:slugfield has been populated throughout existing collections, thegenerate-pathcommand will be updated to use the slug for ODR path generation.Checklist