Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot chain column .name operations #14010

Closed
2 tasks done
thomasaarholt opened this issue Jan 26, 2024 · 4 comments
Closed
2 tasks done

Cannot chain column .name operations #14010

thomasaarholt opened this issue Jan 26, 2024 · 4 comments
Labels
bug Something isn't working documentation Improvements or additions to documentation P-low Priority: low python Related to Python Polars

Comments

@thomasaarholt
Copy link
Contributor

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

df = pl.DataFrame({"Foo bar":[1,2,3]})


(
    # This works
    df
    .select(pl.all().name.map(lambda s: s.replace(" ", "_")))
    .select(pl.all().name.to_lowercase())
    
)

(
    # This does not work
    df
    .select(
        pl.all()
        .name.map(lambda s: s.replace(" ", "_"))
        .name.to_lowercase()
    )
)

(
    # This does not work
    df
    .select(
        pl.col("Foo bar")
        .name.to_lowercase()
        .name.to_lowercase()
    )
)
thread '<unnamed>' panicked at [/Users/runner/work/polars/polars/crates/polars-plan/src/logical_plan/conversion.rs:157:37](https://file+.vscode-resource.vscode-cdn.net/Users/runner/work/polars/polars/crates/polars-plan/src/logical_plan/conversion.rs:157:37):
no `rename_alias` expected at this point
---------------------------------------------------------------------------
---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
Cell In[21], line 24
      3 (
      4     # This works
      5     df
   (...)
      8     
      9 )
     11 # (
     12 #     # This does not work
     13 #     df
   (...)
     18 #     )
     19 # )
     21 (
     22     # This does not work
     23     df
---> 24     .select(
     25         pl.all()
     26         .name.to_lowercase()
     27         .name.to_lowercase()
     28     )
     29 )
     31 # (
     32 #     # This reversed order does not work either
     33 #     df
   (...)
     42     
     43     #)#.select(pl.col("front left"))

File ~/codes/weight/.venv/lib/python3.12/site-packages/polars/dataframe/frame.py:7766, in DataFrame.select(self, *exprs, **named_exprs)
   7664 def select(
   7665     self, *exprs: IntoExpr | Iterable[IntoExpr], **named_exprs: IntoExpr
   7666 ) -> DataFrame:
   7667     """
   7668     Select columns from this DataFrame.
   7669 
   (...)
   7764 
   7765     """
-> 7766     return self.lazy().select(*exprs, **named_exprs).collect(_eager=True)

File ~/codes/weight/.venv/lib/python3.12/site-packages/polars/lazyframe/frame.py:2840, in LazyFrame.select(self, *exprs, **named_exprs)
   2835 structify = bool(int(os.environ.get("POLARS_AUTO_STRUCTIFY", 0)))
   2837 pyexprs = parse_as_list_of_expressions(
   2838     *exprs, **named_exprs, __structify=structify
   2839 )
-> 2840 return self._from_pyldf(self._ldf.select(pyexprs))

PanicException: no `rename_alias` expected at this point

Log output

No response

Issue description

I just tried to remove spaces AND set my column names to lowercase, in a dot-chained operation. Got the following panic:

Expected behavior

Chaining should work fine, not panicing

Installed versions

--------Version info---------
Polars:              0.19.12
Index type:          UInt32
Platform:            macOS-14.1.2-arm64-arm-64bit
Python:              3.12.0 (main, Jan  4 2024, 15:49:05) [Clang 17.0.6 ]

----Optional dependencies----
adbc_driver_sqlite:  0.9.0
cloudpickle:         3.0.0
connectorx:          <not installed>
deltalake:           0.15.1
fsspec:              2023.12.2
gevent:              23.9.1
matplotlib:          3.8.2
numpy:               1.26.3
openpyxl:            <not installed>
pandas:              2.2.0
pyarrow:             15.0.0
pydantic:            2.5.3
pyiceberg:           0.5.1
pyxlsb:              <not installed>
sqlalchemy:          2.0.25
xlsx2csv:            0.8.2
xlsxwriter:          3.1.9
@thomasaarholt thomasaarholt added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Jan 26, 2024
@avimallu
Copy link
Contributor

avimallu commented Jan 26, 2024

Think this is a similar to #10782?

@deanm0000
Copy link
Collaborator

I guess this is a bug for panicking but if you want to do multiple string manipulations to the name then do them all in the one name.map

.name.map(lambda s: s.replace(" ", "_").lower())

@deanm0000 deanm0000 added documentation Improvements or additions to documentation P-low Priority: low and removed needs triage Awaiting prioritization by a maintainer labels Jan 26, 2024
@deanm0000
Copy link
Collaborator

In trying to see if I could improve the handling I discovered:

pl.all().name.prefix("a").meta.tree_format() gives similar error

With backtrace on I get:

thread '<unnamed>' panicked at crates/polars-plan/src/logical_plan/conversion.rs:159:37:
no `rename_alias` expected at this point
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/panicking.rs:72:14
   2: polars_plan::logical_plan::conversion::to_aexpr
   3: polars::expr::meta::_::<impl polars::expr::PyExpr>::__pymethod_meta_tree_format__
   4: pyo3::impl_::trampoline::trampoline
   5: <unknown>
   6: PyObject_Vectorcall
   7: _PyEval_EvalFrameDefault
   8: _PyFunction_Vectorcall
   9: PyObject_Call
  10: _PyEval_EvalFrameDefault
  11: <unknown>
  12: PyEval_EvalCode
  13: <unknown>
  14: _PyEval_EvalFrameDefault
  15: <unknown>
  16: _PyEval_EvalFrameDefault
  17: <unknown>
  18: _PyEval_EvalFrameDefault
  19: <unknown>
  20: <unknown>
  21: _PyEval_EvalFrameDefault
  22: <unknown>
  23: <unknown>
  24: PyObject_Call
  25: _PyEval_EvalFrameDefault
  26: _PyFunction_Vectorcall
  27: PyObject_Call
  28: _PyEval_EvalFrameDefault
  29: <unknown>
  30: _PyEval_EvalFrameDefault
  31: <unknown>
  32: _PyEval_EvalFrameDefault
  33: <unknown>
  34: _PyEval_EvalFrameDefault
  35: <unknown>
  36: _PyEval_EvalFrameDefault
  37: <unknown>
  38: <unknown>
  39: <unknown>
  40: <unknown>
  41: <unknown>
  42: <unknown>
  43: _PyEval_EvalFrameDefault
  44: <unknown>
  45: PyEval_EvalCode
  46: <unknown>
  47: <unknown>
  48: PyObject_Vectorcall
  49: _PyEval_EvalFrameDefault
  50: _PyFunction_Vectorcall
  51: <unknown>
  52: Py_RunMain
  53: Py_BytesMain
  54: <unknown>
  55: __libc_start_main
  56: _start

@deanm0000
Copy link
Collaborator

Known limitation, documents updated reflecting said limitation. Stay tuned here for the chainable enhancement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation P-low Priority: low python Related to Python Polars
Projects
Archived in project
Development

No branches or pull requests

3 participants