Some charts allow selecting entities for which there is no data

[danielgavrilov]

Description

Some StackedArea charts show entities in select dialog that we seem to have no data for:

• Seems like only regions and the World have data, but many countries are listed: https://ourworldindata.org/grapher/vegetable-oil-production?country=~IDN
• https://ourworldindata.org/grapher/per-capita-consumption-of-cereals-by-commodity-type-daily-kilocalories?country=~AGO

Expected behaviour

Entities without (plottable) data should not be listed in select dialog for StackedArea.

[larsyencken]

Guesstimating at 1d, but it could be more complex than that.

[larsyencken]

Reviewed again with @danyx23, we still think this is important to fix

[sophiamersmann]

For reference, here is another example where this is an issue:

• https://ourworldindata.org/grapher/population-breakdown-by-highest-level-of-education-achieved-for-those-aged-15-in?country=~ABW

(The first example given above actually looks fine to me, the second one is a broken link)

[sophiamersmann]

I did an investigation into this one but can't actually fix it. I'll drop my findings here.

The gist is: The select dialog and the chart are based on two different tables – the chart is based on the transformed table, but the select dialog is based on the input table. The select dialog thus displays all entities that exist in the input table. But: In the case of stacked area charts, the transformTable function sometimes deletes all data that is available for a single entity – and no chart can be rendered. In such cases, an entity would show up in the select dialog that has in fact no plottable data.

In the examples I looked at, this function call often seemed to be the "offending" one in the transformTable function:

owid-grapher/packages/@ourworldindata/grapher/src/stackedCharts/AbstractStackedChart.tsx

Line 71 in 2881282

table = table.dropRowsWithErrorValuesForAnyColumn(this.yColumnSlugs)

It drops rows when any column is missing. If a stacked area charts displays, for example, four different variables and one happens to be missing for an entity, than all data would be discarded and no chart plotted. This is what happens here:

https://ourworldindata.org/grapher/population-breakdown-by-highest-level-of-education-achieved-for-those-aged-15-in?country=~ABW

Aruba comes with data for the variable "Population aged 0-14" only, which means that no stacked area chart can be plotted. (But Aruba shows up in the select dialog as it is does exist in the input table!)

[marcelgerber]

For future reference: There's some discussion around this in this Slack thread.

[sophiamersmann]

Note of a discussion in today's Grapher Hour (Marcel, Christian, Sophia):

• Properly fixing this would involve changes to data processing in Grapher
  • There is proposal to pull data transformation out of chart components
  • The proposal is one piece of Modular Grapher
  • It's not clear if/when we would start implementing this
• A dirty/temporary fix could involve:
  • Each chart (if necessary) filters the input table for entities that are actually available (i.e. have plottable data)
  • The table for selection could be passed to Grapher via a property on the manager
• Might be good to wait for the entity picker redesign before working on this bug