Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for multiple recording specs per file to gaze.from_asc() #887

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

saphjra
Copy link
Collaborator

@saphjra saphjra commented Oct 25, 2024

Description

Record all the tracked eye sides in the metadata to fix the issue #875

Implemented changes

Insert a description of the changes implemented in the pull request.
I modified the code in parsing.py to record every time it can be matched in the message line of the asci files

  • Added Regex for Tracking Eye Information in the asci file
  • added a condition to match the pattern multiple times and add it to a list
  • created an additional pre_processed_metadata key : tracked_eyes, where the list is stored

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change is or requires a documentation update

How Has This Been Tested?

I ran it on the to ascii converted ch1hr007.edf of the pilot-hr-1-zh data to verify the output, but no further testing has been done so far

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

Copy link

codecov bot commented Oct 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (70b435a) to head (0267ad1).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #887   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           74        74           
  Lines         3372      3380    +8     
  Branches       594       595    +1     
=========================================
+ Hits          3372      3380    +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dkrako dkrako linked an issue Oct 25, 2024 that may be closed by this pull request
@dkrako dkrako changed the title Feature/metadata feat: add support for multiple specifiations of tracked eyes in gaze.from_asc() Oct 25, 2024
Copy link
Contributor

@dkrako dkrako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks a lot for the submission!

I would like to increase the scope of this PR to not only account for changes of tracked eyes but also changes of the overall recording configuration. They are all included in the same line and can be matched with a single regex.

Apart from this, please put tests into https://github.com/aeye-lab/pymovements/blob/feature/metadata/tests/unit/utils/parsing_test.py

You can test it two ways: either create a new test file with changing configs or just specify a string like it is done for ASC_TEXT to create a file on the fly.
(Test include only one recording config line at the moment)

src/pymovements/utils/parsing.py Outdated Show resolved Hide resolved
src/pymovements/utils/parsing.py Outdated Show resolved Hide resolved
src/pymovements/utils/parsing.py Outdated Show resolved Hide resolved
@dkrako dkrako changed the title feat: add support for multiple specifiations of tracked eyes in gaze.from_asc() feat: add support for multiple recording specs per file in gaze.from_asc() Oct 25, 2024
@dkrako dkrako changed the title feat: add support for multiple recording specs per file in gaze.from_asc() feat: add support for multiple recording specs per file to gaze.from_asc() Oct 25, 2024
@github-actions github-actions bot added the enhancement New feature or request label Nov 6, 2024
…guration are match continously instead of only once
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes
…hanges in the metadata dict, modified parsing_test.py to account for the changes, modified parsing.py to adhere to pylint
…hanges in the metadata dict, modified parsing_test.py to account for the changes, modified parsing.py to adhere to pylint
@saphjra
Copy link
Collaborator Author

saphjra commented Nov 7, 2024

I implemented your suggestions, However. a new problem arise: What value should be given  _calculate_data_loss(
    blinks=blinks,
    invalid_samples=invalid_samples,
    actual_num_samples=actual_number_of_samples,
    total_rec_duration=total_recording_duration,
    sampling_rate=sampling_rate,
) function as sampling_rate. At the moment the _check_sampling_rate() function returns the sampling_rate of the first matched recording_config, However it should probably check if the sampling rate is consistent, raise a warning if not, and probably either declines to compute the data loss, or than takes an average or something similar.

tests/unit/utils/parsing_test.py Show resolved Hide resolved
tests/unit/utils/parsing_test.py Show resolved Hide resolved
tests/unit/utils/parsing_test.py Outdated Show resolved Hide resolved
if not recording_config:
sampling_rate = None
else:
sampling_rate = float(recording_config[0]['sampling_rate'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's sufficient for now to just check for consistency and raise a warning if it's inconsistent.

We can improve on that in a follow-up. Moreover, the logic for calculating data loss will be moved away from this module into the measure module. This way users will be able to calculate these measures on any GazeDataFrame not just when parsed via from_asc().

),
pytest.param(
'** DATE: Wed Mar 8 09:25:20 2023\n'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these added date strings aren't really necessary, right?
It really doesn't matter in this case and you don't need to revert them, but usually I would advise to avoid changes to existing test logic, e.g. changing test values.

Your other changes here, like adding documentation or changing test ids, are of course the spirit that we need! 🥇

Copy link
Collaborator Author

@saphjra saphjra Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all your Feedback 😄

The thing is, without this additional line, the metadata will be empty, since the MSG line gets parsed into the recording_config and not metadata anymore. However if metadata is empty/ none the code will raise a warning and all the tests, where I added the Date line, will fail due to that.

line 367
"""
if not metadata:
raise Warning('No metadata found. Please check the file for errors.')
"""
So I figured, I add a line that should be present in any dataset, which gets parsed by the metadata.

Is there a better way, to solve this?

Copy link
Contributor

@dkrako dkrako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks a lot for your update!

Apart from the comments I left, there is the issue that #884 is now conflicting with your PR. We will probably merge #884 first, as it's close to finished.

We will then need to work out how to integrate this PR here into the new functionality from that PR.

I would probably suggest to do something similar as it's done with the screen distance, which can be statically defined in Experiment, but can also be dynamically defined as a column in GazeDataFrame.

Let's think about how to work this issue out and discuss that next week.
Until then, the work on the other comments should be straight forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Metadata records the tracked eye only once
2 participants