Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SAT failing on similar nested strucutres with different order. #7491

Merged
merged 4 commits into from
Nov 1, 2021

Conversation

avida
Copy link
Contributor

@avida avida commented Oct 29, 2021

What

Resolves #7487

@avida avida changed the title Drezchykov/sat order Fix SAT failing on similar nested strucutres with different order. Oct 29, 2021
@avida avida temporarily deployed to more-secrets October 29, 2021 13:02 Inactive
@avida avida mentioned this pull request Oct 29, 2021
18 tasks
@avida
Copy link
Contributor Author

avida commented Nov 1, 2021

/publish connector=bases/source-acceptance-test

❌ bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1407963349

Copy link
Contributor

@Phlair Phlair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, like this approach. One concern on persisting hash value in object in my comment

@@ -52,13 +51,20 @@ def diff_dicts(left, right, use_markup) -> Optional[List[str]]:


@functools.total_ordering
class DictWithHash(dict):
class HashMixin:
_hash = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't be persisting the hash value in the object because this Mixin is designed to be used with mutable objects (dict and list). e.g.

a = DictWithHashMixin( {"one": 1} )
b = DictWithHashMixin( {"one": 1} )
a == b  # this will correctly be True
a["two"] = 2
a == b  # this will now INCORRECTLY be True, because we're comparing the same saved hash as before
# meaning hash(a) == hash(b) when it should be different

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was thinking on this too, but this just an utility function used by test and test data not supposed to be changed, so I choose optimization.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on current usage, yes... but ultimately this PR implements a new Dict and List that don't update their hash after being mutated. For obvious reasons that behaviour would not be assumed by a developer in the future utilising these and we could end up with problems. I think we should therefore always recompute.

I would be very surprised if performance from Python's hash function became limiting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this make sense, updated PR

@avida avida temporarily deployed to more-secrets November 1, 2021 12:57 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 12:59 Inactive
@avida avida temporarily deployed to more-secrets November 1, 2021 13:38 Inactive
Copy link
Contributor

@Phlair Phlair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks!

@avida
Copy link
Contributor Author

avida commented Nov 1, 2021

/publish connector=bases/source-acceptance-test

🕑 bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1408148351
✅ bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1408148351

@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 13:48 Inactive
@avida avida merged commit ebf35eb into master Nov 1, 2021
@avida avida deleted the drezchykov/sat-order branch November 1, 2021 13:52
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SAT not works correctly on records with nested structure and different orders
4 participants