"Cannot calculate mean, unexpected type of argument" when data has nulls

[codeheart09]

When I call .describe() on a data frame with some null values, I get this error:

return new TypeError('Cannot calculate ' + fnName + ', unexpected type of argument' + details);
           ^

TypeError: Cannot calculate mean, unexpected type of argument (type: null, value: null)

You can check the dataset here: https://raw.githubusercontent.com/ageron/handson-ml2/master/datasets/housing/housing.tgz

Here is the code I have:

// ... after downloading and extracting the file
const dataFrame = await dfd.read_csv(RAW_DATA_DIR + '/housing.csv');
dataFrame.describe().print();

And if I console.log the dataframe itself, the data seem to be imported correctly.

// ...
'$data': [
    [
      -122.23, 37.88,
      41,      880,
      129,     322,
      126,     8.3252,
      452600,  'NEAR BAY'
    ],
// ..
'$columns': [
    'longitude',
    'latitude',
    'housing_median_age',
    'total_rooms',
    'total_bedrooms',
    'population',
    'households',
    'median_income',
    'median_house_value',
    'ocean_proximity'
  ]

But it indeed has some null values in some rows.

Shouldn't the lib skip the null values? Or I'm missing something?

[risenW]

Thanks for raising this issue, and yes, we skip NaN values before computations. I’ll take a look ASAP.

[risenW]

Update:

@codeheart09 This has been fixed in the latest release.
You can install and use the latest Typescript version (v1.0.0).

If you need help migrating to the latest version, see: https://danfo.jsdata.org/examples/migrating-to-the-stable-version-of-danfo.js

[d11wtq]

I seem to be hitting this with v1.1.1. Some values are listed as null and describe throws:

TypeError: Cannot calculate mean, unexpected type of argument (type: null, value: null)

Is there a regression since v1.0.0, or do we need to manually filter out the null values?

[d11wtq]

Using nodejs v16.13.2 if that helps.

[the-code-genin]

2023 Update: I ran into the same issue TypeError: Cannot calculate mean, unexpected type of argument (type: null, value: null). I'm using the iris-classification CSV dataset.

The error occurred when the column names weren't specified in the input CSV but were specified via the readCSV config parameter i.e frameConfig.columns. The number of actual CSV records was 150, but the DataFrame was only reading 149 records, which suggests an issue with parsing.

When the column names were specified in the input CSV and readCSV was called without any config parameters, the describe() method worked as expected and 150 CSV records were read by the DataFrame.

So I'm guessing it's a bug from parsing?