Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes bugs with erroneous messages from output logs, duplicate records in collections, and handling of high_instrument_saturation special constants #859

Merged
merged 10 commits into from
May 2, 2024

Conversation

al-niessner
Copy link
Contributor

@al-niessner al-niessner commented Mar 25, 2024

🗒️ Summary

This is an epic and had many fixes and repairs to the table system in validate.

⚙️ Test Data and/or Report

Automated unit tests below should succeed.

♻️ Related Issues

Closes #874
Closes #854
Closes #849
Closes #831

In this case the inventory file had million plus rows. The way duplicates were being computed was to iterate over all entries then iterate over all entries. Hence the loop was a million million. So, changed the code so that it iterates once. It completes the million in many milliseconds now.
@al-niessner al-niessner self-assigned this Mar 25, 2024
@al-niessner al-niessner requested a review from a team as a code owner March 25, 2024 22:16
@al-niessner al-niessner marked this pull request as draft March 25, 2024 22:16
@jordanpadams
Copy link
Member

@al-niessner are we ready for this to more out of draft?

@al-niessner
Copy link
Contributor Author

al-niessner commented Apr 1, 2024 via email

Al Niessner added 4 commits April 24, 2024 12:39
Turn off the error messages from saxon because validate generates its own messages and errors when things are too bad.
@al-niessner al-niessner marked this pull request as ready for review May 1, 2024 16:56
@al-niessner
Copy link
Contributor Author

@jordanpadams @tloubrieu-jpl @nutjob4life

Moving off this PR at @jordanpadams request. It is ready.

Copy link
Member

@nutjob4life nutjob4life left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 👍

Fix bug for duplicate filenames used in different sub-directories
@jordanpadams jordanpadams changed the title 832: table handling corrections Fixes bugs with erroneous messages from output logs, duplicate records in collections, and handling of high_instrument_saturation special constants May 2, 2024
@jordanpadams jordanpadams merged commit 7fce5bc into main May 2, 2024
3 checks passed
@jordanpadams jordanpadams deleted the issue_832 branch May 2, 2024 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment