Take Pandera for a Spin

[alexrichey]

TLDR

It looks like Pandera is just a better version of the thing that I'd slapped together. It would take another few two or three hours of cleanup to swap it in completely.

Examples

MIH fails standardized values checks:

Facilities fails to have all required columns:

How it Works

In this implementation, we read in our metadata file, and translate all the columns for a given dataset into a Pandera Check, using our custom validators (e.g. for wkb types). Then we feed it a dataframe to validate. Simple!

Also, in this implementation, we read in all source data as a str type into the dataframe, then use Checks to validate when needed. Pandera Data Types seem like potentially the ideal approach for this.