scan_ndjson not handling projection pushdown correctly

[lithomas1]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
from polars.testing import assert_frame_equal
df = pl.DataFrame(
    {
        "a": [1, 2, 3, None],
        "b": ["ẅ", "x", "y", "z"],
        "c": [None, None, 4, 5],
    }
)
df.write_ndjson("file.jsonl")
q = pl.scan_ndjson(
    "file.jsonl",
    row_index_name="row-index",
    row_index_offset=0
).select(["row-index", "a"])

df1 = q.collect(projection_pushdown=True)
df2 = q.collect(projection_pushdown=False)

assert_frame_equal(df1, df2)

Log output

No response

Issue description

When scanning a ndjson file and doing a column projection afterwards with select (selecting the row-index and a column from the scanned dataframe), polars will drop the row-index.

I believe the correct solution is just to swap the row index and projection conditions here
235ebee#diff-f91032885fa26496b1aa443e731f1dbb9d346e0567d4bafc96999839209e01b1R319-R326

(When the row_index is not in the selected columns, I believe the optimizer passes it as None in the IR, so I think swapping the conditions should be correct for both row_index being selected and not being selected. I can try to submit a PR for this if the patch is correct.)

Expected behavior

The row index should be kept (like it is for CSV, and when the projection_pushdown option is False in collect.

Installed versions

--------Version info---------
Polars:               1.1.0
Index type:           UInt32
Platform:             Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python:               3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]

----Optional dependencies----
adbc_driver_manager:  1.0.0
cloudpickle:          3.0.0
connectorx:           0.3.3
deltalake:            0.18.2
fastexcel:            0.10.4
fsspec:               2024.6.1
gevent:               24.2.1
great_tables:         0.9.0
hvplot:               0.10.0
matplotlib:           3.9.0
nest_asyncio:         1.6.0
numpy:                1.26.4
openpyxl:             3.1.5
pandas:               2.2.2
pyarrow:              16.1.0
pydantic:             2.8.0
pyiceberg:            <not installed>
sqlalchemy:           2.0.31
torch:                2.3.1.post300
xlsx2csv:             0.8.2
xlsxwriter:           3.2.0

[ritchie46]

@nameexhaustion can you take a look here?