Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(join): skip substitution of non-field references in join chains #9595

Merged
merged 4 commits into from
Jul 15, 2024

Conversation

coroa
Copy link
Contributor

@coroa coroa commented Jul 15, 2024

Description of changes

As alternative to #9594, we here avoid the same signature error raised in #9561, when a Literal or more general Value in a JoinChain fails to be included in a DerefMap.

A statement like
t3 = t1.inner_join(t2, "i").select("i", lit=1, val=2 * t1.x) creates a JoinChain looking like

JoinChain[r0]
  JoinLink[inner, r1]
    r0.i == r1.i
  values:
    i:     r0.i
    lit:   1
    val:   2 * r0.x

with one of the reference "fields" in values actually pointing to a Literal and another one to a more general Value.

Adding a subsequent JoinLink to this JoinChain like by
t5 = t3.inner_join(t4, "i") fails in prepare_predicates, since the DerefMap.from_targets(..., extra=reverse) call tries to add a substitute from the new lit-field to the Literal(1) object and from val to the Value-like expression, which both do not fit the Field type, that the subs attribute of the DerefMap are expected to have.

Here we skip those as direct substitution possibilities.

Issues closed

Copy link
Contributor

ACTION NEEDED

Ibis follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message.

Please update your PR title and description to match the specification.

@coroa coroa changed the title fix(join): Skip substitution of non-field references in join chains fix(join): skip substitution of non-field references in join chains Jul 15, 2024
@cpcloud cpcloud added bug Incorrect behavior inside of ibis internals Issues or PRs related to ibis's internal APIs ux User experience related issues labels Jul 15, 2024
@cpcloud cpcloud added this to the 9.2 milestone Jul 15, 2024
@cpcloud
Copy link
Member

cpcloud commented Jul 15, 2024

@coroa I pushed up a small test.

@coroa
Copy link
Contributor Author

coroa commented Jul 15, 2024

@coroa I pushed up a small test.

Thanks! But note that the SignatureValidationError was only triggered by adding another JoinLink.

Should i extend the test for that?

@cpcloud
Copy link
Member

cpcloud commented Jul 15, 2024

Ah, yes!

@coroa
Copy link
Contributor Author

coroa commented Jul 15, 2024

@cpcloud What do you think about that addition to the test?

@cpcloud
Copy link
Member

cpcloud commented Jul 15, 2024

LGTM, I have another case I want to add, pushing that here in a sec.

@cpcloud
Copy link
Member

cpcloud commented Jul 15, 2024

Thanks for the PR!

@coroa
Copy link
Contributor Author

coroa commented Jul 15, 2024

Doctest failure looks unrelated (table ordering instability).

Thanks for the quick turn-around, it's a delight.

@cpcloud
Copy link
Member

cpcloud commented Jul 15, 2024

Yep, failure is unrelated, and fixed in #9597.

@cpcloud cpcloud force-pushed the fix-non-field-values-in-join-chain branch from 99a7587 to 8054db1 Compare July 15, 2024 20:53
@cpcloud cpcloud enabled auto-merge (squash) July 15, 2024 20:53
@coroa
Copy link
Contributor Author

coroa commented Jul 15, 2024

FYI for debugging i found it very helpful to add some simple __repr__ methods to

class Field(Value):
    [...]
    def __repr__(self):
        return f"<Field name={self.name} rel={self.rel}>"

and

class Relation(Node, Coercible):
    [...]
    def __repr__(self):
        return f"<{self.__class__.__name__} fields={','.join(self.schema)}>"

I saw that you have some formatting facilities with pretty and so on, but having a simple Node representation would make the error messages quite a lot clearer.

@cpcloud cpcloud merged commit 61ef0ed into ibis-project:main Jul 15, 2024
81 of 82 checks passed
@coroa coroa deleted the fix-non-field-values-in-join-chain branch July 15, 2024 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis internals Issues or PRs related to ibis's internal APIs ux User experience related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: SignatureValidationError from DerefMap during repeated joins
2 participants