Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Joining on list columns is not implemented #14826

Open
shenker opened this issue Mar 3, 2024 · 2 comments
Open

Joining on list columns is not implemented #14826

shenker opened this issue Mar 3, 2024 · 2 comments
Labels
A-panic Area: code that results in panic exceptions enhancement New feature or an improvement of an existing feature

Comments

@shenker
Copy link
Contributor

shenker commented Mar 3, 2024

Description

Related issues: #10747, #12636, #4175, #2869, #13950

df1 = pl.DataFrame({"a": [[1,2,3],[1,2,3],[4,5,6]]})
df2 = pl.DataFrame({"a": [[1,2,3],[4,5,6]], "b": ["x", "y"]})
df1.join(df2, on="a")

gives
PanicException: not implemented.

Note that this error is given even if the join column is a list[cat] or list[enum] column, even though you can compute hashes of those column types and join on the hash. At the very least, polars should automatically do that for you instead of panicking. It is unexpected that group_by works on list[cat] and list[enum] columns but join doesn't.

@shenker shenker added the enhancement New feature or an improvement of an existing feature label Mar 3, 2024
@tafia
Copy link

tafia commented Mar 26, 2024

Similarly:

df1 = pl.DataFrame({"a": [1, 2], "b": [1, 2] })
df2 = df1
df2.join(df1, on=(["a"],))

Also panics (not implemented)

@coastalwhite coastalwhite added the A-panic Area: code that results in panic exceptions label Jun 18, 2024
coastalwhite added a commit to coastalwhite/polars that referenced this issue Jun 21, 2024
This PR refactors the code surrounding bit representations and properly bubbles
errors up if no bit representation is defined.

This resolves the panic in pola-rs#14826, but does not implement the wanted behavior.

I am not sure a test case is useful here as the behavior should be added in the end.
coastalwhite added a commit to coastalwhite/polars that referenced this issue Jun 21, 2024
This PR refactors the code surrounding bit representations and properly bubbles
errors up if no bit representation is defined.

This resolves the panic in pola-rs#14826, but does not implement the wanted behavior.

I am not sure a test case is useful here as the behavior should be added in the end.
@ritchie46
Copy link
Member

If we encounter list columns we should go into the row encoding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-panic Area: code that results in panic exceptions enhancement New feature or an improvement of an existing feature
Projects
None yet
Development

No branches or pull requests

4 participants