Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for read-only installation #303

Closed
kkappler opened this issue Oct 30, 2023 · 2 comments
Closed

Support for read-only installation #303

kkappler opened this issue Oct 30, 2023 · 2 comments

Comments

@kkappler
Copy link
Collaborator

Participants will be running their notebooks from the installed location, but will not have write access to the aurora install directory.

The Jupyter notebooks must be run in that state.

kkappler added a commit that referenced this issue Nov 3, 2023
- change the mth5_path to be part of make_mth5_from_asc instead of being defined in the StationConfig
- add target_folder argument to create_mth5_synthetic file
kkappler added a commit that referenced this issue Nov 5, 2023
- also removed some housekeeping:
- added show_processing_summary for viewing a reduced version of processing summary df
- added loguru logging to a few messages
- remove some unused imports
- fix a few typos

Issue #303, #135
@kkappler kkappler closed this as completed Nov 5, 2023
kkappler added a commit that referenced this issue Nov 6, 2023
- need pypi or condaforge to test NCI environment build, therefore
  bumping to test there if issue #303 is resolved
@kkappler kkappler reopened this Nov 9, 2023
@kkappler
Copy link
Collaborator Author

kkappler commented Nov 9, 2023

If we want to have access to the synthetic data we have to move the data within the PyPI scope. This means:

  • either adding __init__.py files inside tests/ folder or moving data to top-level data/ folder
    I created aurora/data/ and moved the data that was in tests there, and set the paths to operate on that folder.
    Note that this does not get us around the need to mkdir for test, but it should at least make the data available.
    However, for reasons I don't understand (see next bullet) the data are not available after a pip install.Existing options include
    moving the data folder down a level, or packing the synthetic data into mt_examples. I opted for the later, which meant I had to add a source_folder option in make_mth5_from_ascii.

  • Update MANIFEST.in to contain data folder. When doing this I note that tests is already in the mainfest. I do not understand why tests are in the build/ folder but not in the pip install.

What is currently breaking however is the execution of pathlib's mkdir() command when the places for the test results are being built. Since there are no tests in PyPI or conda-forge , we should be able to get around this by:

  • executing the mkdir command inside the setUpClass of the test methods, rather than pre-making the directories in test_utils (OR the folders could also be prebuilt inside tests, with a committed README so that the folders exist in github already) The solution I went with was hybrid, I import a class that can mkdirs into the tests, and mkdir only on executing the tests. That should stop the mkdir errors on gadi when importing methods.
  • Handle missing files: config/emtf_band_setup folder is empty, except for the init.py file. This seems to be be because init.py only tags .py files. The .cfg files need to be added to MANIFEST.in

kkappler added a commit that referenced this issue Nov 10, 2023
- moved synthetic ascii data from tests/synthetic/data to
  data/synthetic/ascii
- adjusted paths for new data location
- moved mkdirs exectuions into tests/synthetic
kkappler added a commit that referenced this issue Nov 10, 2023
- add source_folder option to make_mth5_from_ascii if data are not
  available in installed version
- move an import to within a calling function
- suppress assert statement that wont work if data/ folder not present
@kkappler
Copy link
Collaborator Author

This worked in the 2023 workshop in Canberra at on the NCI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant