Update type interface to use type hierarchy in tablecloth.api.util

[ezmiller]

Goal

In #71 , @genmeblog pointed out that there's a type hierarchy in the tablecloth.api.utils namespace. Following some discussion in #73 , this PR aims to set up a type interface where users mainly interact the types of elements in a column in terms of the "general" (rather than concrete) types. In other words, a user will want to know if an array has :integer rather than :int64 values.

Solution

I added some new functions in tablecloth.api.utils to help work with the type hierarchy tablecloth.api.utils/type-sets:

• ->general-types - Given a concrete type this function returns the set of general types, e.g.: (->general-types :int64) ;; => #{:integer :numerical}
• types - Returns the set of general types, i.e. the keys of tablecloth.api.utils/type-sets
• concrete-types - Returns the set of concrete types, i.e. the types that TMD uses.
• concrete-type? - Returns true if the given type is a concrete type.

Then I adjusted typeof and typeof? to use this hierarchy, so now:

• typeof? - Will return true if the elements of the given column match the given general type or concrete type
• typeof - Returns the set of general types that describe the concrete type of the elements of the column

Please see the tests in this PR for examples.

Open Questions

• Is there any reason to consider using Clojure's native hierarchies instead of the map for tablecloth.api.utils/type-sets?
• Is there any reason not to allow typeof? also to support concrete types?

[ezmiller]

@genmeblog just a reminder to take a look at this when you get a chance. :)

[genmeblog]

I did a review. I see mainly one issue with changing typeof contract. The other two are just optimization proposals.

[ezmiller]

@genmeblog: I changed things here so that:

• types - returns the set concrete types;
• general-types returns the set of general types;
• typeof returns the concrete type of the column;
• typeof? can validate the concrete or general type of a column; and,
• there is a general-types-lookup table.

[ezmiller]

@genmeblog if you have a moment, would you like to take another look at this?

[genmeblog]

Looks ok