Struct is_null, is_not_null and drop_nulls inconsistent behavior

[paladin158]

Polars version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Polars.

Issue description

It seems like drop_nulls gives inconsistent behavior between python v0.17.15 and v0.18.3. And, the behavior of is_null is different from rust v0.30.0.

Reproducible example

import polars as pl

df = pl.DataFrame({"a": [1, 2, None], "b": [1, 2, 3]})

df.select(pl.struct(["a", "b"]).drop_nulls())
### results
## py v0.17.15
# {1,1}
# {2,2}

## py v0.18.3
# {1,1}
# {2,2}
# {null,3}

## rust v0.30.0
# {1,1}
# {2,2}

s = df.select(pl.struct(["a", "b"]).to_series()

s.is_null())
### results
## py v0.17.15
# false false false

## py v0.18.3
# false false false

## rust v0.30.0
# false false false

s.is_not_null())
### results
## py v0.17.15
# true true false

## py v0.18.3
# true true false

## rust v0.30.0
# true true false

s.drop_nulls())
### results
## py v0.17.15
# {1,1}
# {2,2}

## py v0.18.3
# {1,1}
# {2,2}
# {null,3}

## rust 0.30.0
# {1,1}
# {2,2}

Expected behavior

All three should have the same behavior as rust v0.30.0: drop null if at least one values on struct is null for drop_nulls.

Not sure whether is_null and is_not_null should have the opposite result.

Installed versions

Polars v0.17.15
Polars v0.18.3

[ritchie46]

There is ambiguity here. In polars we count a struct row as Null if all fields are Null.

[stinodego]

In current Polars, everything works correctly except for:

import polars as pl

df = pl.DataFrame({"a": [1, 2, None], "b": [1, 2, 3]})

s = df.select(pl.struct(["a", "b"])).to_series()

r = s.is_not_null()
print(r)

shape: (3,)
Series: 'a' [bool]
[
        true
        true
        false
]