[REVIEW]: scida: scalable analysis for scientific big data

[editorialbot]

Submitting author: @cbyrohl (Chris Byrohl)
Repository: https://github.com/cbyrohl/scida
Branch with paper.md (empty if default branch): joss
Version: v0.3.1
Editor: @mbobra
Reviewers: @EGaraldi, @kyleaoman
Archive: 10.5281/zenodo.10681463

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c7361dc4526bc2934c99b0ba4a12dc86"><img src="https://joss.theoj.org/papers/c7361dc4526bc2934c99b0ba4a12dc86/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c7361dc4526bc2934c99b0ba4a12dc86/status.svg)](https://joss.theoj.org/papers/c7361dc4526bc2934c99b0ba4a12dc86)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@EGaraldi & @kyleaoman, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @mbobra know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @kyleaoman

📝 Checklist for @EGaraldi

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.10 s (1213.7 files/s, 141251.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          68           1303           1199           6899
YAML                            20            105             87           1544
Markdown                        22            414              0           1119
SVG                              5              0              0            892
TeX                              1              9              0            138
CSS                              1             19              1             95
Jupyter Notebook                 5              0            738             93
TOML                             1              6              1             63
make                             1              5              0             16
JavaScript                       1              3              0             15
Bourne Shell                     2              1              2             13
-------------------------------------------------------------------------------
SUM:                           127           1865           2028          10887
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 867

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3847/1538-3881/aadae0 is OK
- 10.1088/0067-0049/192/1/9 is OK
- 10.5281/zenodo.7331952 is OK
- 10.1093/mnras/stad1779 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[mbobra]

@EGaraldi, @kyleaoman Thank you so much for agreeing to review. You can find the article in the comment box above ⬆️ , the software repository linked in the first comment box on this issue. To generate your checklist, use the following command:

@editorialbot generate my checklist

I think you're good to go. Again, JOSS is an open review process and we encourage communication between the reviewers, the submitting author, and the editor. And please feel free to ask me questions, I'm always around.

Can you please respond here (or give a thumbs up) so I know you're in the right place and found all the materials?

[kyleaoman]

Review checklist for @kyleaoman

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/cbyrohl/scida?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@cbyrohl) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
  Not provided.

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[EGaraldi]

Review checklist for @EGaraldi

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/cbyrohl/scida?
• License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@cbyrohl) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
  There is no directly stated guideline for contributing at this time

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[EGaraldi]

@cbyrohl I left a single comment in my checklist above about a (small) missing item to comply with JOSS requirements. I also opened a number of issues in the repository, namely:

• Filename in Tutorial->simulations
• API doc for scida.load()
• ">>>" in code snippets
• Error when loading GAIA dataset in tutorial
• 2D histogram from GAIA looks different when running the code
• Typo in FITS files documentation
• Errors in code in docs unit cookbook entry
• Missing variable names in API docs
• JOSS paper: missing comparison with tangos

Apart from these issues, I find the code very useful and potentially quite impactful.

[kyleaoman]

I've completed my review. Have opened a number of fairly minor issues:

cbyrohl/scida#118
cbyrohl/scida#117
cbyrohl/scida#116
cbyrohl/scida#115
cbyrohl/scida#112
cbyrohl/scida#111
cbyrohl/scida#110
cbyrohl/scida#109

The three things that I think should be corrected before this is accepted in JOSS are:

• Generally make sure that example input data to accompany all examples is either provided directly or signposted to (ensuring that the examples run with the example data).
• Add contribution guidelines on github, specifically including instructions to install dev dependencies and run the test suite (probably including a case where the developer has access to the machines hosting the larger input data for testing).
• Tidy up the API docs as outlined in API docs incomplete cbyrohl/scida#118.

Overall this looks like a useful package, will definitely personally be coming back to either use directly it or look for inspiration for some of my own analysis codes.

[kyleaoman]

@mbobra could you just clarify the action to take now? Is the idea to iterate with the authors on any open issues that I feel would preclude accepting the submission until I'm satisfied?

[mbobra]

@mbobra could you just clarify the action to take now? Is the idea to iterate with the authors on any open issues that I feel would preclude accepting the submission until I'm satisfied?

Yes. You don't need to take the initiative here, though -- @cbyrohl, when you've finished addressing @kyleaoman's open issues, please let us know here. Thank you both!

[mbobra]

👋 @cbyrohl Just checking in -- how is everything going? Do you need any help?

[cbyrohl]

@mbobra, should be all done now; apart from possible iteration from @EGaraldi, @kyleaoman. I addressed all raised github issues, and have closed those. Please re-open or open another issue where a follow-up is needed. In addition to the github issues, @kyleaoman raised the following three things to be addressed:

1. Availability of input data. All notebooks should run with the example SDSS/TNG50-4 simulation files. The download link to those is now available with a download link at the beginning of the respective tutorial page. This particularly includes a link to the SDSS data which was previously missing. In addition, other pages now include an info box linking to this page for the TNG50-4 snapshot and group catalog data. I was indeed surprised at the amount of examples that did not run properly with provided data with introduced changes over time. To largely reduce this happening again in the future, the markdown documentation examples are now included in the test suite (https://github.com/cbyrohl/scida/blob/main/tests/test_docs.py).
2. An elementary developer guide has been added here: https://cbyrohl.github.io/scida/developer/. With PR describe testdata used. cbyrohl/scida#136, the download paths and configuration required to run the tests is now documented in the developer guide. Larger files are currently hosted on /virgotng on MPCDF clusters. I have added the paths to these resources in describe large testdata used from /virgotng resource. cbyrohl/scida#137.
3. See answer in API docs incomplete cbyrohl/scida#118. I added docstring for most methods add docstrings cbyrohl/scida#121, I would complement the docstrings further on the go. Let me know if/what else you would like to see in the basic API functionality.

Thanks for all your help.

[mbobra]

Thank you, @cbyrohl.

@kyleaoman Can you please confirm whether this submission passed review? If so, could you please check off the two remaining items (functionality documentation and community guidelines)?

@EGaraldi Could you also confirm the same? If this paper passed your review, could you please check off the remaining item (community guidelines)?

[kyleaoman]

I've ticked off the last two items on my copy of the checklist, from my side this passes review.

[mbobra]

I've ticked off the last two items on my copy of the checklist, from my side this passes review.

Thank you for all your time and effort @kyleaoman! Your volunteer time keeps JOSS running and the JOSS team appreciates it very much ☀️

[EGaraldi]

Passed review from my side as well @mbobra

[mbobra]

Passed review from my side as well @mbobra

Thank you so much @EGaraldi! I know it is not easy to make time to volunteer -- your efforts help JOSS tremendously and we really appreciate it 🎉

[mbobra]

Congratulations, @cbyrohl -- we're almost ready to publish the paper! Could you please cut a new release of scida and deposit it in Zenodo (per these instructions) and put the link to the Zenodo deposit in this thread? Please make sure that the author list and title on the Zenodo deposit matches the author list and title on the JOSS paper. Then I can move forward with accepting the paper.

[cbyrohl]

Thanks @mbobra; find the zenodo record here: https://zenodo.org/records/10681463

[mbobra]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[mbobra]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3847/1538-3881/aadae0 is OK
- 10.1088/0067-0049/192/1/9 is OK
- 10.5281/zenodo.7331952 is OK
- 10.1093/mnras/stad1779 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[mbobra]

@editorialbot set 10.5281/zenodo.10681463 as archive

[editorialbot]

Done! archive is now 10.5281/zenodo.10681463

[mbobra]

@editorialbot set v0.3.1 as version

[editorialbot]

Done! version is now v0.3.1

[mbobra]

@editorialbot recommend-accept

[editorialbot]

Attempting dry run of processing paper acceptance...

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3847/1538-3881/aadae0 is OK
- 10.1088/0067-0049/192/1/9 is OK
- 10.5281/zenodo.7331952 is OK
- 10.1093/mnras/stad1779 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

👋 @openjournals/aass-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#5043, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

[mbobra]

Thank you for your submission and all the edits, @cbyrohl! I hope you feel it improved the code. This is a really nice paper.

[cbyrohl]

@editorialbot generate preprint

[editorialbot]

📄 Preprint file created: Find it here in the Artifacts list 📄

[dfm]

@editorialbot accept

[editorialbot]

Doing it live! Attempting automated processing of paper acceptance...

[editorialbot]

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

cff-version: "1.2.0"
authors:
- family-names: Byrohl
  given-names: Chris
  orcid: "https://orcid.org/0000-0002-0885-8090"
- family-names: Nelson
  given-names: Dylan
  orcid: "https://orcid.org/0000-0001-8421-5890"
doi: 10.5281/zenodo.10681463
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Byrohl
    given-names: Chris
    orcid: "https://orcid.org/0000-0002-0885-8090"
  - family-names: Nelson
    given-names: Dylan
    orcid: "https://orcid.org/0000-0001-8421-5890"
  date-published: 2024-02-28
  doi: 10.21105/joss.06064
  issn: 2475-9066
  issue: 94
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 6064
  title: "scida: scalable analysis for scientific big data"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.06064"
  volume: 9
title: "scida: scalable analysis for scientific big data"

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

[editorialbot]

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

[editorialbot]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.06064 joss-papers#5063
1. Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.06064
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[dfm]

Many thanks to @EGaraldi and @kyleaoman for reviewing and to @mbobra for editing! JOSS relies upon the volunteer effort of people like you and we simply wouldn't be able to do this without you!!

@cbyrohl — Your paper is now accepted and published in JOSS! ⚡🚀💥

[editorialbot]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.06064/status.svg)](https://doi.org/10.21105/joss.06064)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.06064">
  <img src="https://joss.theoj.org/papers/10.21105/joss.06064/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.06064/status.svg
   :target: https://doi.org/10.21105/joss.06064

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://reviewers.joss.theoj.org/join
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss