Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata Viewer: API to return dict, not list #3292

Merged
merged 3 commits into from
Nov 18, 2024

Conversation

pllim
Copy link
Contributor

@pllim pllim commented Nov 14, 2024

Description

This pull request is to address the request, "File metadata extracted from the API is hard to parse. Maybe a dictionary is better?"

I avoided changing metadata itself because that would mean messing with the front-end stuff that already works. Also might be subtly confusing because the same attribute now returns something different. Maybe best to have a clean break using a different attribute name.

Change log entry

  • Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts,
    list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer
    should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

  • Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
  • Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
  • Do the proposed changes follow the STScI Style Guides?
  • Are tests added/updated as required? If so, do they follow the STScI Style Guides?
  • Are docs added/updated as required? If so, do they follow the STScI Style Guides?
  • Did the CI pass? If not, are the failures related?
  • Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone. Bugfix milestone also needs an accompanying backport label.
  • After merge, any internal documentations need updating (e.g., JIRA, Innerspace)? 🐱

@pllim pllim added this to the 4.1 milestone Nov 14, 2024
@github-actions github-actions bot added the plugin Label for plugins common to multiple configurations label Nov 14, 2024
@pllim pllim added the api API change label Nov 14, 2024
Copy link

codecov bot commented Nov 14, 2024

Codecov Report

Attention: Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Project coverage is 88.81%. Comparing base (479f68c) to head (537e4d9).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
jdaviz/core/user_api.py 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3292      +/-   ##
==========================================
- Coverage   88.81%   88.81%   -0.01%     
==========================================
  Files         125      125              
  Lines       19030    19036       +6     
==========================================
+ Hits        16901    16906       +5     
- Misses       2129     2130       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@pllim pllim marked this pull request as ready for review November 14, 2024 22:57
@camipacifici
Copy link
Contributor

Much easier to use now, thank you!
I had the same question as Kyle about meta vs metadata. I do not have strong opinions since I doubt it was widely used. I am ok with meta if metadata can be deprecated, but I let you all decide.
Is it possible to carry also the description of the keyword like it was before? Example in screenshot.
Screenshot 2024-11-15 at 10 45 17 AM

@pllim
Copy link
Contributor Author

pllim commented Nov 15, 2024

During 2024-11-15 tag-up, it was agreed to deprecate metadata as public API.

carry also the description

@camipacifici , this was also discussed at tag-up but unclear if this will be confusing to users or not. Description is only applicable when it comes from FITS header (but even then, not always there). In astropy.io.fits.Header, such things are represented by key: (value, descrip) combo but a user that is not familiar with that convention might not understand why the key gives back a tuple.

@kecnry suggested a nested dictionary. But will a nested dictionary be confusing too? Also a nested dictionary cannot be roundtripped back into a FITS Header easily.

@kecnry
Copy link
Member

kecnry commented Nov 15, 2024

nested dictionary could look something like:

{'ACT_ID': {'description': 'Activity identifier', 'value': 1},
 'ASPERNAME': {'description': 'PRD science aperture used': 'value': 'NRS_FULL_MSA'}
...
}

@rosteen
Copy link
Collaborator

rosteen commented Nov 15, 2024

Also a nested dictionary cannot be roundtripped back into a FITS Header easily.

Is there somewhere in the code that we do this in Jdaviz that would be made more complicated, or are you talking about a user who might want to put the metadata returned by this back into a Header?

@camipacifici
Copy link
Contributor

@pllim I think users are very familiar with what astropy.io.fits gives and the description (if available) is returned. I would try to stay closed to that format. The combo is fine.

@pllim
Copy link
Contributor Author

pllim commented Nov 15, 2024

are you talking about a user who might want to put the metadata returned by this back into a Header?

Yes, there is always a chance a user did a cube smooth or whatever and then wants to write their own FITS file back out and then want to stuff in header from original cube.

@pllim
Copy link
Contributor Author

pllim commented Nov 15, 2024

Okay, looks like I give fits.Header too much credit. This works if d is a simple key-value pair.

fits.Header({"a": 1})

This does not work.

fits.Header({"a": (1, "aaa")})

But if you do it keyword by keyword, this works.

hdr = fits.Header()
hdr["a"] = (1, "aaa")

So I guess simple full roundtrip is quite impossible. Just a matter of how much code you want user to write, like this.

for key, val in meta.items():
    hdr[key] = val

Or like this.

for key, val in meta.items():
    hdr[key] = (val["value"], val.get("description", ""))

Any preference?

@rosteen
Copy link
Collaborator

rosteen commented Nov 15, 2024

I'm in favor of the nested dictionary approach, I think that's the clearest thing to return. We could always provide a meta_as_header method in the API to do the fiddly bits of packing it back up in a FITS header (as a follow up, I don't think that would need to go in this PR).

@pllim pllim force-pushed the meta-as-dict branch 2 times, most recently from 811f28a to 7d56136 Compare November 15, 2024 22:35
@@ -34,6 +35,9 @@ def __getattr__(self, attr):
if attr in _internal_attrs or attr not in self._expose:
return super().__getattribute__(attr)

if attr in self._deprecated:
logging.warning("DeprecationWarning: %s is deprecated" % attr)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-11-18 114104

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about making it fancier so it can say "use meta instead" but decided against it because hopefully we won't do this often and I don't want to overcomplicate the machinery.

CHANGES.rst Outdated Show resolved Hide resolved
Co-authored-by: Kyle Conroy <kyleconroy@gmail.com>
Copy link
Collaborator

@rosteen rosteen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pllim pllim merged commit 896d4d9 into spacetelescope:main Nov 18, 2024
19 checks passed
@pllim pllim deleted the meta-as-dict branch November 18, 2024 19:03
@pllim
Copy link
Contributor Author

pllim commented Nov 18, 2024

Thanks for the reviews!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api API change plugin Label for plugins common to multiple configurations Ready for final review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants