Ensure fixed casing for object names in inventory tables.

[larsgeorge-db]

Names like catalog, databases and tables etc. should be uppercased in tables.scala. This makes it easier to process them downstream in the jobs.

[nfx]

@larsgeorge-db maybe they should be lowercased?

[pohlposition]

This is 2 weeks old.

Is this something we need to do? Do we have naming conventions for things like this?

[larsgeorge-db]

I do not think we do... I am actually looking at this and was asking myself the same question. Which column should be converted to what?

col_name	data_type	casing
catalog	string	lower
database	string	lower
name	string	lower
object_type	string	upper
table_format	string	upper
location	string	leave
view_text	string	leave
upgraded_to	string	leave

Looking at what we have so far, object_type is upper already, so matching table_format to it makes sense. Given SQL objects are treated case-insensitive by DBSQL we could lower case the names, making them easier on the eyes mostly. But, this does not apply to all RDBMSs, some are case-sensitive, like Oracle, so names of federated tables are likely sensitive too. Assuming we deal with the HMS side here, even a federated table would be in the connection/location details, so the case-insensitivity still applies here.

[larsgeorge-db]

Looking at our dashboard queries, the only column we have to UPPER() is table_format. In the code we LOWER() the database and table names at a few places. Do we expect any impact on applying the above conversions directly while scanning HMS and taking stock of its data structures?

[nfx]

@HariGS-DB what's the progress here?

[HariGS-DB]

@nfx Have started working on it today. Had one question to confirm.

• The requirement is to convert all fields of all tables created by ucx to lower case? and ensure read operation dont do any upper anywhere like in dashboard queries.
• only the tables used by tables.scala (tables and table_failures)?