Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt LEMI reader to handle runs split across multiple files #107

Closed
kkappler opened this issue Jul 24, 2022 · 2 comments
Closed

Adapt LEMI reader to handle runs split across multiple files #107

kkappler opened this issue Jul 24, 2022 · 2 comments

Comments

@kkappler
Copy link
Collaborator

kkappler commented Jul 24, 2022

In general, it is recommended to group the LEMI files in folders per station.

Within a station folder, there can be many files.

Every file is associated with exactly one run. However, some runs are associated with more than one file.

Therefore it is desirable to group the files according to their runs.

The basic functionality should be the ability to point at a folder and return a Data Frame that lists the files with a column for run_id. The LEMI reader also needs to be modified to accept a list of filenames (associated with a run), rather than just one .TXT file. This can be done either by

  1. Reading in each file and then providing a self._merge(other) method
  2. Replacing self.fn with self.fn_list
kkappler added a commit that referenced this issue Aug 12, 2022
Needed to update precommit stuff as well, not sure why,
may have been triggered by change to .flake8

[Issue(s): #105, #107]
@kujaku11
Copy link
Owner

Working on it.

I think we could add a function that searches for all '.TXT' files in the given directory and then in read you could specify which file numbers you want to read, default would be all.

Alternatively we could overwrite the __add__ function to just append the data dataframes.

@kujaku11
Copy link
Owner

This has been fixed in with mth5.io.lemi.LEMICollection which concatenates pandas.DataFrames of each file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants