Commits on Jan 10, 2021

1. [SPARK-34055][SQL] Refresh cache in ALTER TABLE .. ADD PARTITION …

   Invoke `refreshTable()` from `CatalogImpl` which refreshes the cache in v1 `ALTER TABLE .. ADD PARTITION`.
   
   This fixes the issues portrayed by the example:
   ```sql
   spark-sql> create table tbl (col int, part int) using parquet partitioned by (part);
   spark-sql> insert into tbl partition (part=0) select 0;
   spark-sql> cache table tbl;
   spark-sql> select * from tbl;
   0	0
   spark-sql> show table extended like 'tbl' partition(part=0);
   default	tbl	false	Partition Values: [part=0]
   Location: file:/Users/maximgekk/proj/add-partition-refresh-cache-2/spark-warehouse/tbl/part=0
   ...
   ```
   Create new partition by copying the existing one:
   ```
   $ cp -r /Users/maximgekk/proj/add-partition-refresh-cache-2/spark-warehouse/tbl/part=0 /Users/maximgekk/proj/add-partition-refresh-cache-2/spark-warehouse/tbl/part=1
   ```
   ```sql
   spark-sql> alter table tbl add partition (part=1) location '/Users/maximgekk/proj/add-partition-refresh-cache-2/spark-warehouse/tbl/part=1';
   spark-sql> select * from tbl;
   0	0
   ```
   
   The last query must return `0	1` since it has been added by `ALTER TABLE .. ADD PARTITION`.
   
   Yes. After the changes for the example above:
   ```sql
   ...
   spark-sql> alter table tbl add partition (part=1) location '/Users/maximgekk/proj/add-partition-refresh-cache-2/spark-warehouse/tbl/part=1';
   spark-sql> select * from tbl;
   0	0
   0	1
   ```
   
   By running the affected test suite:
   ```
   $ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *AlterTableAddPartitionSuite"
   ```
   
   Closes apache#31101 from MaxGekk/add-partition-refresh-cache-2.
   
   Lead-authored-by: Max Gekk <max.gekk@gmail.com>
   Co-authored-by: Hyukjin Kwon <gurwls223@gmail.com>
   Signed-off-by: HyukjinKwon <gurwls223@apache.org>
   (cherry picked from commit e0e06c1)
   Signed-off-by: Max Gekk <max.gekk@gmail.com>

   

   MaxGekk and HyukjinKwon committed Jan 10, 2021
   Configuration menu

   • View commit details
   • Copy full SHA for 2898875
   • Browse repository at this point

   Copy the full SHA

   2898875 View commit details

   Browse the repository at this point in the history