Add DECIMAL to VARCHAR coercion to hive

[homar]

Description

Add DECIMAL to UNBOUNDED VARCHAR coercion to hive

Non-technical explanation

This is intended to allow reading data from columns which type changed from decimal to string in hive.

Release notes

( ) This is not user-visible and no release notes are required.
( ) Release notes are required, please propose a release note for me.
() Release notes are required, with the following suggested text:

# Section
* Allows reading columns which type was changed from decimal to string in hive. ({issue}`issuenumber`)

[findepi]

nit: lowercase "UNBOUNDED" and "BOUNDED"

we sometimes writes type names in uppercase, but we don't write adjectives, that describe the these types, in uppercase.

[findepi]

lengthLimit ?

[findinpath]

I have tried your test with a different file format

        assertUpdate("CREATE TABLE " + tableName + " (col1 DECIMAL(23,3), col2 int, col3 int) with ( FORMAT = 'JSON', partitioned_by = ARRAY['col3'])");
        assertUpdate("INSERT INTO " + tableName + "(col1, col2, col3) VALUES(DECIMAL '100.13', 1, 1)", 1);
        query("ALTER TABLE " + tableName + " DROP COLUMN col1");
        query("ALTER TABLE " + tableName + " ADD COLUMN col1 VARCHAR");
        assertQuery("SELECT col1, col2, col3 FROM " + tableName, "VALUES ('100.13', 1, 1)");

and got a different result from PARQUET testing:

        String tableName = "test_long_decimal_to_varchar_coercion_" + randomTableSuffix();
        assertUpdate("CREATE TABLE " + tableName + " (col1 DECIMAL(23,3), col2 int, col3 int) with ( FORMAT = 'PARQUET', partitioned_by = ARRAY['col3'])");
        assertUpdate("INSERT INTO " + tableName + "(col1, col2, col3) VALUES(DECIMAL '100.13', 1, 1)", 1);
        query("ALTER TABLE " + tableName + " DROP COLUMN col1");
        query("ALTER TABLE " + tableName + " ADD COLUMN col1 VARCHAR");
        assertQuery("SELECT col1, col2, col3 FROM " + tableName, "VALUES ('100.130', 1, 1)");

Notice 100.13 for JSON and 100.130 for Parquet.

[findepi]

When a partition has decimal and table has varchar,
does Hive's eventual result depend on the partition's file format? 🤯

[findinpath]

I did some extra checking and realised that for JSON data, the HiveRecordCursor is employed and not the HivePageSource.

There is no coercing done as far as I see in the HiveRecordCursor.

[findepi]

Some file formats (eg CSV) are positional. Dropping column other than the last and re-adding it may have weird results (like shifting data).

Let's maybe make the interesting column the last one.

Also, do we need 3 columns, or would 2 columns be enough?
Can the columns have names more distinct than col1, col2, col3?

like dummy integer, evolved_column decimal(...) ?

[homar]

I think I will drop this class entirely, these test cases are covered by product test anyway

[homar]

and BTW it cannot be the last one because the last one is a partitioning column, and partitioning column has to be the last one. Also there need to be 3 columns at least as otherwise I won't be able to drop the first one.

[findepi]

I think I will drop this class entirely

let me know whether you want to drop it or should i continue with review of it

[homar]

After writing that comment I left it so you can review it ;)
I am trying to add other file formats here but some don't work because of the problem with dropping/adding columns

[findepi]

Since Trino doesn't have proper support for changing column type, let's run with product tests.

[homar]

I've just wanted to have a test that will check that coercion to too small varchar will fail with expected exception. If you think this is not needed I will delete it

[findepi]

i think it's fine not to have it

[findepi]

@homar please see the checks failure