Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue #96: ensure the created dimension is sorted #101

Merged
merged 2 commits into from
Feb 14, 2024
Merged

Issue #96: ensure the created dimension is sorted #101

merged 2 commits into from
Feb 14, 2024

Conversation

ank1m
Copy link
Collaborator

@ank1m ank1m commented Feb 9, 2024

Github Issue: #96

Description

The order of input files was not preserved in CONCISE. The multi_core_download function was not sorting the returned list from multiprocessing.Manager. So the output order would be random.

Overview of work done

Overview of verification done

  • Built custom concise docker image with the implemented modifications and deployed it in the local harmony environment
  • Ran multiple requests and compared the order of datafiles arriving at CONCISE input with the order of datafiles in the output merged file:
Comparison before sorting implemented shows that the order is not preserved Comparison after sorting implemented shows that the order is preserved
image image
image image

Overview of integration done

See above

PR checklist:

  • Linted (manually)
  • [ N/A] Updated unit tests
  • Updated changelog
  • Integration testing

See Pull Request Review Checklist for pointers on reviewing this pull request

jamesfwood
jamesfwood approved these changes Feb 14, 2024
View reviewed changes
Copy link
Collaborator

@jamesfwood jamesfwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@jamesfwood jamesfwood merged commit 6fdeeb2 into develop Feb 14, 2024
2 checks passed
@jamesfwood jamesfwood deleted the feature/issue-96 branch February 14, 2024 18:14
@jamesfwood jamesfwood mentioned this pull request Feb 27, 2024
Closed
jamesfwood added a commit that referenced this pull request Mar 1, 2024
@jamesfwood @sliu008
@jonathansmolenski @dependabot @frankinspace @ank1m
Release/0.8.0 (#102)
c9157ec 
* Feature/update add collection test (#94)

* update add collection test to get the url for json history

* update changelog

* /version 0.7.0a24

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Feature/update add collection test (#95)

* update add collection test to get the url for json history

* update changelog

* update test to test for nan

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Bump jinja2 from 3.1.2 to 3.1.3 (#99)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Using cmr-umm-updater default branch (develop)

* use develop


* Update CONTRIBUTING.md


* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Issue #96: ensure the created dimension is sorted (#101)

* implement sorting of the output queue according to the order of the input queue to satisfy issue #96

* Update CHANGELOG.md with issue-96 fix

* release 0.8.0


---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: sliu008 <69875423+sliu008@users.noreply.github.com>
Co-authored-by: concise bot <concise@noreply.github.com>
Co-authored-by: jonathansmolenski <jonathansmolenski@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Frank Greguska <89428916+frankinspace@users.noreply.github.com>
Co-authored-by: ank1m <154441323+ank1m@users.noreply.github.com>
Co-authored-by: James Wood <James.F.Wood@jpl.nasa.gov>
jamesfwood added a commit that referenced this pull request Jun 10, 2024
@jamesfwood @sliu008
@jonathansmolenski @dependabot @frankinspace @ank1m @danielfromearth
Release 0.9.0 (#113)
7e2c251 
* Feature/update add collection test (#94)

* update add collection test to get the url for json history

* update changelog

* /version 0.7.0a24

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Feature/update add collection test (#95)

* update add collection test to get the url for json history

* update changelog

* update test to test for nan

* /version 0.7.0a25

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* /version 0.8.0a1

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Bump jinja2 from 3.1.2 to 3.1.3 (#99)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Using cmr-umm-updater default branch (develop)

* use develop

* /version 0.8.0a2

* Update CONTRIBUTING.md

* /version 0.8.0a3

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update uat_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Update ops_associations.txt with new collections

* Issue #96: ensure the created dimension is sorted (#101)

* implement sorting of the output queue according to the order of the input queue to satisfy issue #96

* Update CHANGELOG.md with issue-96 fix

* /version 0.8.0a4

* /version 0.9.0a1

* Bump idna from 3.4 to 3.7 (#108)

Bumps [idna](https://github.com/kjd/idna) from 3.4 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v3.4...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump tqdm from 4.66.1 to 4.66.3 (#109)

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.1...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* /version 0.9.0a2

* Bump jinja2 from 3.1.3 to 3.1.4 (#110)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* /version 0.9.0a3

* Feature/cleanup tests (#112)

* update dependency versions

* Removed CMR testing

* update actions versions

---------

Co-authored-by: James Wood <James.F.Wood@jpl.nasa.gov>

* /version 0.9.0a4

* relese 0.9.0

* /version 0.9.0rc1

* update actions versions

* /version 0.9.0rc2

* add type annotations and args to docstrings (#115)

* add type annotations and args to docstrings

* update CHANGELOG.md

* /version 0.9.0a5

* /version 0.9.0rc3

* update dep libs

* /version 0.9.0rc4

* update library versions

* /version 0.9.0rc5

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: sliu008 <69875423+sliu008@users.noreply.github.com>
Co-authored-by: concise bot <concise@noreply.github.com>
Co-authored-by: jonathansmolenski <jonathansmolenski@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Frank Greguska <89428916+frankinspace@users.noreply.github.com>
Co-authored-by: ank1m <154441323+ank1m@users.noreply.github.com>
Co-authored-by: James Wood <James.F.Wood@jpl.nasa.gov>
Co-authored-by: Daniel Kaufman <114174502+danielfromearth@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jamesfwood jamesfwood jamesfwood approved these changes

@sliu008 sliu008 sliu008 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ank1m @jamesfwood @sliu008