Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Absolute HREFs of assets #57

Merged
merged 7 commits into from
Apr 8, 2022

Conversation

moradology
Copy link
Contributor

@moradology moradology commented Mar 8, 2022

Description

This PR adds conditional modification of Item asset hrefs so that relative paths to image assets in the DB are rendered as absolute for users. While adding this feature, it was also clear that tilejson and metadata assets could be rendered as absolute on the assumption that the only change necessary is replacement of /stac/ with /data/ - I could use a second pair of eyes on this assumption as it is possible I'm not imagining a failure condition which makes this solution unworkable.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

I've manually tested this behavior and added an item with ID al_m_3008501_ne_16_060_20191109_20200114_relative_path_test that should trigger relative->absolute path transformation. As pgstac 0.4.5 is necessary to run things locally, a couple of tests are commented out due to incompatibility with changes on this version of pgstac.

Checklist:

Please delete options that are not relevant.

  • I have performed a self-review
  • Changelog has been updated
  • Unit tests pass locally (./scripts/test)
  • Code is linted and styled (./scripts/format)

@moradology
Copy link
Contributor Author

One difficulty I ran into when developing this was that caching on get_item made it difficult to see changes/print out values at the locations I was hoping to. What's the quick fix for this and should there be some kind of cache busting/duration shortening in the dev environment?

@moradology
Copy link
Contributor Author

image

image

@mmcfarland
Copy link
Member

#59 has the fix for the broken data links, thanks for catching that. I also had friction with the caching in development while testing that fix, we don't have a known-good workaround, but I think disabling the behavior altogether by default in dev is probably a good idea.

@moradology moradology force-pushed the feature/absolute-asset-hrefs branch from 3352e3c to 023a031 Compare March 14, 2022 18:28
]
]
},
"links": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarity, a good test would be to add a new link here with a relative path to another item (collections/naip/items/{item_id}).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like my original main review comment made it into the review - sorry if this test comment was confusing. I've re-added a comment about asset vs link hrefs in this follow-up review.

Copy link
Member

@mmcfarland mmcfarland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a comment was dropped from my original review, updating here for context.

def absolute_asset_links(self, item: Item, base_url: str) -> Item:
"""Convert relative asset links to absolute links"""
# Supporting the fields extension means skipping if assets aren't here
if "assets" in item and "image" in item["assets"]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than relative asset href links, I think the use case we're trying to support is for link hrefs to be relative and replaced with the current base_url. For example, we're expecting future collections that have a derivedfrom rel type, which is a link to another item in a different collection, accessed via the STAC API. The assets in all cases should always be absolute. For links, I think it's safe to assume any relative path should be made absolute, and we shouldn't need to check for specirfic rel types

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating this with the proposed tests, but it appears to me that stac-fastapi already does what is desired?(here: https://github.com/stac-utils/stac-fastapi/blob/c948dbfcd12fff0939d0bc3ecb907878ee217416/stac_fastapi/pgstac/stac_fastapi/pgstac/models/links.py#L105)

]
]
},
"links": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like my original main review comment made it into the review - sorry if this test comment was confusing. I've re-added a comment about asset vs link hrefs in this follow-up review.

@moradology moradology force-pushed the feature/absolute-asset-hrefs branch from 541606d to ba5cf15 Compare April 7, 2022 15:33
@moradology moradology requested a review from mmcfarland April 7, 2022 15:34
Copy link
Member

@mmcfarland mmcfarland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, good catch. Good to have these tests now as we'll soon be depending on the behavior. Two comments on dropping some leftovers from your original implementation.

@@ -72,7 +72,7 @@ def inject_collection_links(self, collection: Collection) -> Collection:

return collection

def inject_item_links(self, item: Item) -> Item:
def inject_item_links(self, item: Item, base_url: str) -> Item:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can drop this base_url arg now

@@ -176,11 +176,13 @@ async def _fetch() -> ItemCollection:
# Remove context extension until we fully support it.
result.pop("context", None)

base_url = str(kwargs["request"].base_url)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, we don't need to pass in base_url to the inject method anymore

Copy link
Member

@mmcfarland mmcfarland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just clipped those out to get this in for our rrp.

@mmcfarland mmcfarland merged commit f541557 into microsoft:main Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants