Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release candidate for 0.0.4 #138

Closed
wants to merge 83 commits into from
Closed

Release candidate for 0.0.4 #138

wants to merge 83 commits into from

Commits on Jul 29, 2024

  1. Added a start at a quantile implementation in the existing MR framework.

    @mmcdermott
    mmcdermott committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    f9d7a0e View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge branch 'dev' into 51_add_quantile_aggregation

    @mmcdermott
    mmcdermott committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    7685436 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Got the reducer to return reasonable numbers at least.

    @mmcdermott
    mmcdermott committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    9c3b497 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Merged.

    @mmcdermott
    mmcdermott committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    2d87cae View commit details
    Browse the repository at this point in the history

  2. Added doctests and parametrization.

    @mmcdermott
    mmcdermott committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    4b3cfec View commit details
    Browse the repository at this point in the history

  3. VERY preliminary code. Just committing so as to not lose changes. Not… 

    …hing works yet.
    @mmcdermott
    mmcdermott committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e721506 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. Merge branch 'dev' into 101_match_revise

    @mmcdermott
    mmcdermott committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    10b741a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'dev' into alt_51_raw_values

    @mmcdermott
    mmcdermott committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    76ce942 View commit details
    Browse the repository at this point in the history

  3. Merged.

    @mmcdermott
    mmcdermott committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    209abc1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'dev' into alt_51_raw_values

    @mmcdermott
    mmcdermott committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    d898cb4 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Renamed to 'code_modifiers'

    @mmcdermott
    mmcdermott committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    bb88c1c View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. Merge branch 'dev' into 101_match_revise

    @mmcdermott
    mmcdermott committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    5a39b7c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'dev' into alt_51_raw_values

    @mmcdermott
    mmcdermott committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    949867a View commit details
    Browse the repository at this point in the history

  3. Very preliminary implementation of match and revise

    @mmcdermott
    mmcdermott committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    8e1a78d View commit details
    Browse the repository at this point in the history

  4. Started adding documentation and tests; not yet complete

    @mmcdermott
    mmcdermott committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    e260088 View commit details
    Browse the repository at this point in the history

  5. Tests added for everything except bind compute function.

    @mmcdermott
    mmcdermott committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    8750b93 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Added all doctests.

    @mmcdermott
    mmcdermott committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    8e418ab View commit details
    Browse the repository at this point in the history

  2. Added an integration test via the filter_measurement transform; also … 

    …an example of loading MEDS_transforms configs as defaults dynamically.
    @mmcdermott
    mmcdermott committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    f19ecc7 View commit details
    Browse the repository at this point in the history

  3. Attempted to update test workflow, add codecov badge, and add a workf… 

    …low for pushing to pypi. The pypi push workflow requires local tagging via 'git tag 0.0.3' then 'git push origin 0.0.3'
    @mmcdermott
    mmcdermott committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    39aebbc View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. Merge pull request #123 from mmcdermott/update_CI_and_README 

    Updated workflows and README.
    @mmcdermott
    mmcdermott authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    33ebdd9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'dev' into alt_51_raw_values

    @mmcdermott
    mmcdermott committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    cff193b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'dev' into 101_match_revise

    @mmcdermott
    mmcdermott committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    3b7aae0 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Fix split_and_shard_patients when the full split definition is provided

    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    c9c581f View commit details
    Browse the repository at this point in the history

  2. External splits go after internal splits

    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    0b924b3 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'refs/heads/dev' into fix-external-split

    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    f352654 View commit details
    Browse the repository at this point in the history

  4. Added mkdocs documentation starter code.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3c69775 View commit details
    Browse the repository at this point in the history

  5. Added readthedocs file.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ebabad7 View commit details
    Browse the repository at this point in the history

  6. Added docs badge.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3840385 View commit details
    Browse the repository at this point in the history

  7. Updated tests workflow to ignore docs.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8075963 View commit details
    Browse the repository at this point in the history

  8. Updated docs to move basic docs into readthedocs.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    1dedfad View commit details
    Browse the repository at this point in the history

  9. Correct mdformat issue

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9c78213 View commit details
    Browse the repository at this point in the history

  10. linted files.

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e0d9fea View commit details
    Browse the repository at this point in the history

  11. Added revision dates and authors to docs

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    39c626e View commit details
    Browse the repository at this point in the history

  12. Merge pull request #128 from mmcdermott/77_mkdocs_readthedocs 

    Adds readthedocs via `mkdocs`. Still in progress.
    @mmcdermott
    mmcdermott authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3cc758f View commit details
    Browse the repository at this point in the history

  13. Merge branch 'dev' into 101_match_revise

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    cca82d4 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'dev' into alt_51_raw_values

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3e68573 View commit details
    Browse the repository at this point in the history

  15. Merge pull request #98 from mmcdermott/alt_51_raw_values 

    Support quantile computation over numerical values in a simple (though inefficient) manner.
    @mmcdermott
    mmcdermott authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    36a172d View commit details
    Browse the repository at this point in the history

  16. Merge branch 'dev' into 101_match_revise

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    fa1d6f9 View commit details
    Browse the repository at this point in the history

  17. Add warning when split_fracs_dict not empty but performing the split … 

    …solely based on external_splits
    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    44f18b7 View commit details
    Browse the repository at this point in the history

  18. Merge branch 'refs/heads/dev' into fix-external-split

    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3fb5e8b View commit details
    Browse the repository at this point in the history

  19. Throw ValueError when external split lengths contradict n_patient_per… 

    …_shar restriction
    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    2e1f875 View commit details
    Browse the repository at this point in the history

  20. Add .editorconfig

    @prenc
    prenc committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    2fbb293 View commit details
    Browse the repository at this point in the history

  21. Started removing unnecessary references to shards file

    @mmcdermott
    mmcdermott committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8839f67 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Removed usage of shards json file throughout, outside of the direct e… 

    …xtraction pipeline where it is guaranteed to be created.
    @mmcdermott
    mmcdermott committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ba6b5b6 View commit details
    Browse the repository at this point in the history

  2. Used a hash function instead of true randomization to order the shard… 

    …s in the iterator for test stability in github workflows
    @mmcdermott
    mmcdermott committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    d86fe1e View commit details
    Browse the repository at this point in the history

  3. Made it so that docs use submodule README files as the source documen… 

    …tation for the submodule page.
    @mmcdermott
    mmcdermott committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    86d1acd View commit details
    Browse the repository at this point in the history

  4. Merge pull request #131 from mmcdermott/add-editorconfig 

    Add .editorconfig
    @prenc
    prenc authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7d6c273 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'refs/heads/dev' into fix-external-split

    @prenc
    prenc committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f318a4a View commit details
    Browse the repository at this point in the history

  6. Allow resharding of external splits, move rng declaration out of the … 

    …if statement
    @prenc
    prenc committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    546c763 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #124 from mmcdermott/fix-external-split 

    Fix split_and_shard_patients when the full split definition is provided
    @prenc
    prenc authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    430a950 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #132 from mmcdermott/129_remove_splits_json_file_u… 

    …sage

Removed unnecessary usage of the `splits.json` file that MEDS-Extract writes during sharding.
    @mmcdermott
    mmcdermott authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    d719ab4 View commit details
    Browse the repository at this point in the history

  9. Fixed merge conflict.

    @mmcdermott
    mmcdermott committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    224a1a8 View commit details
    Browse the repository at this point in the history

  10. Fixed lint errors.

    @mmcdermott
    mmcdermott committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f2d2d0e View commit details
    Browse the repository at this point in the history

  11. Merge pull request #119 from mmcdermott/101_match_revise 

    Adds the ability to specialize existing transformations to be applied with separate configurations to different portions of the data based on configurable filters.
    @mmcdermott
    mmcdermott authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    631819f View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. Moved parser around.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    36add00 View commit details
    Browse the repository at this point in the history

  2. Updated mandatory types to be separate from mandatory MEDS columns

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    4263592 View commit details
    Browse the repository at this point in the history

  3. Fixed bug with matcher import.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    8fc4f48 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #133 from mmcdermott/update_parser_and_type_info 

    This moves the parser to a general import, and absorbs some type checking commits from another PR to clean up the PR.
    @mmcdermott
    mmcdermott authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    0fadd96 View commit details
    Browse the repository at this point in the history

  5. Fixed typo

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    dd43d5a View commit details
    Browse the repository at this point in the history

  6. Separated merge shards shard iterator out for ease of import.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    fd6e77f View commit details
    Browse the repository at this point in the history

  7. Starting to add integration test for eventual functionality; not yet … 

    …implemented fully.
    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    dd22eeb View commit details
    Browse the repository at this point in the history

  8. Improved compliance by removing creation of shards.json file and addi… 

    …ng splits parquet file.
    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    1b2d023 View commit details
    Browse the repository at this point in the history

  9. Merge pull request #137 from mmcdermott/136_testing_code_compliance 

    Improved compliance by removing creation of shards.json file and adding `patient_splits.parquet` file.
    @mmcdermott
    mmcdermott authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    fc877e3 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'dev' into 134_reshard_by_split

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    4b2dee2 View commit details
    Browse the repository at this point in the history

  11. Updated test.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    6ec7a8f View commit details
    Browse the repository at this point in the history

  12. started file; very much not ready yet.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    dd9ba08 View commit details
    Browse the repository at this point in the history

  13. Improved some documentation and error handling for splitting patients.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    4cc65b4 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'dev' into 134_reshard_by_split

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    ba9c3b4 View commit details
    Browse the repository at this point in the history

  15. Implemented preliminary version of reshard to split.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    d70128f View commit details
    Browse the repository at this point in the history

  16. fixed some more minor errors; test fails for content reasons now.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    0db3a23 View commit details
    Browse the repository at this point in the history

  17. code was actually fine; it was a test error.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    829cede View commit details
    Browse the repository at this point in the history

  18. Merge pull request #135 from mmcdermott/134_reshard_by_split 

    Adds a reshard-by-split stage as a transform.
    @mmcdermott
    mmcdermott authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    5e47f0c View commit details
    Browse the repository at this point in the history

  19. Corrected a small parallelism issue in reshard.

    @mmcdermott
    mmcdermott committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    e0d1ecb View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. Made reshard not error out if the new split is identical to the shard… 

    …s on disk.
    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    ac1587e View commit details
    Browse the repository at this point in the history

  2. Fixed formatting.

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    2d1c4cf View commit details
    Browse the repository at this point in the history

  3. Move more to locked computation.

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    7b02df8 View commit details
    Browse the repository at this point in the history

  4. changes in progress to improve robustness

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    b1378bc View commit details
    Browse the repository at this point in the history

  5. changes in progress to improve robustness

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    3a95794 View commit details
    Browse the repository at this point in the history

  6. Made file checker wait for valid parquet files

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    d9ed79a View commit details
    Browse the repository at this point in the history

  7. Removed unneeded line.

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    d1f5953 View commit details
    Browse the repository at this point in the history

  8. Made rwlock more robust and eliminated unused return mode.

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    e0c0f46 View commit details
    Browse the repository at this point in the history

  9. Made resharding use rwlock wrap

    @mmcdermott
    mmcdermott committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    3fc909e View commit details
    Browse the repository at this point in the history

  10. Merge pull request #143 from mmcdermott/139_fix_reshard_to_split_in_p… 

    …arallel_mode

Fixes brittle reduce stage checking in Reshard stage.
    @mmcdermott
    mmcdermott authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    cda4f06 View commit details
    Browse the repository at this point in the history