feat(python, rust): timestampNtz support

[ion-elgreco]

Description

• This addresses all our timestamp inconsistencies, where we were reading Primitive:timestamp as a datatetime without UTC, and now we can properly write datetimes with no timezone as columns to Primitive::timestampNtz.
• addressing small bug where checkConstraints feature was not set in writerFeatures when you are on table writer version 7.
• bumping default protocol to 3,7
• Made the pyarrow writer and reader more flexible so we can write/read a 3,7 table as long as it has the supported features there.
• Properly parses timestamps with UTC into pyarrow timestamps with UTC
• Added configkey translation to tablefeature inside the Create Operation

Related Issue(s)

• closes IsAdjustedToUtc flag not used while while reading DeltaTable #1598
• closes Cannot create Delta Lake table from DataFusion data frame with timestamp field with timezone #1019
• closes We cannot append data to existing Delta Lake tables if the schema of data to write includes timestamp columns with timezone. #1777

[hntd187]

Also delta-scala doesn't universally cast things to UTC, it respects the isAdjustedToUTC param for timestamps, where timestampntz has this set to false, while normal timestamps are set to true

[rtyler]

@ion-elgreco why are these changes required? Are we not picking up table properties correctly right now? If so, is there a bug this is related to?

[ion-elgreco]

@rtyler I am doing that so I can go from tableConfigKeys into tableFeatures. This is what spark-delta also does if your table protocol versions are 3,7 and you do set tblproperties "delta.enableChangeDataFeed" = true, it will additionally set the feature ChangeDataFeed into the protocol commit.

Maybe there is another way of doing this but I thought it was clever to put it on the from_str :P

[roeap]

hmm, just passing by, but this seems strange. these are the values written in the feature vector list, so this seems like a strange place to do this conversion as well, isnce it describes completely differnet fields, now? one is values in the feature vecotor field, andother one is keys in the table properties, which should be handled in the table config?

[ion-elgreco]

I added the table config keys here as well for quick parsing to table features, see my comment above.

In spark when your protocol is at 3,7 it will auto convert the configs into table features so that's basically what I am doing here as well.

[guilhem-dvr]

nit: the CL addresses the TODO no?

[ion-elgreco]

Right, forgot to remove that : )

[roeap]

are we sure this is right?

[ion-elgreco]

Without this it would fail for the timestampNtz, since spark wrote it as timestamp_ntz in the log. I couldnt find his anywhere in the docs, but it only impacted timestampNtz since the rest of them are one word long