Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semi-join issue introduced in 0.15.17 #6455

Closed
2 tasks done
stinodego opened this issue Jan 26, 2023 · 3 comments · Fixed by #6459
Closed
2 tasks done

Semi-join issue introduced in 0.15.17 #6455

stinodego opened this issue Jan 26, 2023 · 3 comments · Fixed by #6459
Labels
bug Something isn't working python Related to Python Polars

Comments

@stinodego
Copy link
Member

Polars version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Issue description

This issue was introduced with 0.15.17.

Reproducible example

from datetime import datetime

import polars as pl

df = pl.DataFrame(
    {
        "id": [1, 1, 2],
        "timestamp": [datetime(2022, 12, 11), datetime(2022, 12, 12), datetime(2022, 1, 1)],
        "value": [1, 2, 4],
    }
).lazy()

latest = df.groupby("id").agg(pl.col("timestamp").max())
df = df.join(latest, on=["id", "timestamp"], how="semi")
result = df.select(["id", "value"]).collect()

print(result)
# pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: NotFound(Owned("id"))

Expected behavior

I expect the code to execute normally.

Installed versions

---Version info---
Polars: 0.15.17
Index type: UInt32
Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python: 3.11.0 (main, Nov  1 2022, 09:16:00) [GCC 11.2.0]
---Optional dependencies---
pyarrow: 10.0.1
pandas: 1.5.2
numpy: 1.24.0
fsspec: 2022.11.0
connectorx: <not installed>
xlsx2csv: 0.8.1
deltalake: <version not detected>
matplotlib: <not installed>
@stinodego stinodego added bug Something isn't working python Related to Python Polars labels Jan 26, 2023
@stinodego
Copy link
Member Author

stinodego commented Jan 26, 2023

Random semi-related question to help my Polars knowledge:

Would the code outlined in this example (groupby -> semi-join) be faster, slower, or similar in performance when compared to window -> filter? I'm guessing the window approach would be faster.

@ritchie46
Copy link
Member

So there will be a 0.15.18 🙈

@ritchie46
Copy link
Member

I think the filter as well. But it will likely depend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Related to Python Polars
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants