Code

update to new#8

Code

Merged

fengjian428 merged 52 commits intofengjian428:masterfengjian428/hudi:masterfrom apache:masterapache/hudi:masterCopy head branch name to clipboardMay 21, 2022

+6,082-2,122

Commits

Commits on May 5, 2022

• [MINOR] Optimize code logic (#5499)

  qianchutao

  authored
  d794f4f

  Copy full SHA for d794f4f

• [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor exit gracefully (#4264)

  guanziyue

  authored
  abb4893

  Copy full SHA for abb4893

Commits on May 6, 2022

• [HUDI-4042] Support truncate-partition for Spark-3.2 (#5506)

  jinxing64

  authored
  248b059

  Copy full SHA for 248b059

• [HUDI-4017] Improve spark sql coverage in CI (#5512)

  Show description for c319ee9
  xushiyan

  authored
  c319ee9

  Copy full SHA for c319ee9

• [HUDI-3675] Adding post write termination strategy to deltastreamer continuous mode (#5073)

  Show description for 52fe1c9
  nsivabalan

  authored
  52fe1c9

  Copy full SHA for 52fe1c9

Commits on May 7, 2022

• [HUDI-3849] AvroDeserializer supports AVRO_REBASE_MODE_IN_READ configuration (#5287)

  cxzl25

  authored
  9625d16

  Copy full SHA for 9625d16

• [MINOR] Fixing class not found when using flink and enable metadata table (#5527)

  BruceKellan

  authored
  80f9989

  Copy full SHA for 80f9989

• [MINOR] fixing flaky tests in deltastreamer tests (#5521)

  nsivabalan

  authored
  569a76a

  Copy full SHA for 569a76a

Commits on May 9, 2022

• [HUDI-4055]refactor ratelimiter to avoid stack overflow (#5530)

  guanziyue

  authored
  75eaa0b

  Copy full SHA for 75eaa0b

• [MINOR] Fixing close for HoodieCatalog's test (#5531)

  Show description for 4c70840
  XuQianJin-Stars

  authored
  4c70840

  Copy full SHA for 4c70840

• [HUDI-4053] Flaky ITTestHoodieDataSource.testStreamWriteBatchReadOpti… (#5526)

  Show description for 6b47ef6
  

  xicm

  and

  xicm

  authored
  6b47ef6

  Copy full SHA for 6b47ef6

• [HUDI-3995] Making perf optimizations for bulk insert row writer path (#5462)

  Show description for 6285a23
  nsivabalan

  authored
  6285a23

  Copy full SHA for 6285a23

Commits on May 10, 2022

• [HUDI-4044] When reading data from flink-hudi to external storage, the … (#5516)

  Show description for 6fd21d0
  

  aliceyyan

  and

  aliceyyan

  authored
  6fd21d0

  Copy full SHA for 6fd21d0

• [HUDI-4003] Try to read all the log file to parse schema (#5473)

  lanyuanxiaoyao

  authored
  4258a71

  Copy full SHA for 4258a71

Commits on May 11, 2022

• [HUDI-4038] Avoid calling getDataSize after every record written (#5497)

  Show description for 4a8589f
  

  Alexey Kudinkin

  and

  nsivabalan

  authored
  4a8589f

  Copy full SHA for 4a8589f

• [HUDI-4079] Supports showing table comment for hudi with spark3 (#5546)

  jinxing64

  authored
  7f0c1f3

  Copy full SHA for 7f0c1f3

• [HUDI-4085] Fixing flakiness with parquet empty batch tests in TestHoodieDeltaStreamer (#5559)

  nsivabalan

  authored
  b10ca7e

  Copy full SHA for b10ca7e

Commits on May 12, 2022

• [HUDI-3963][Claim RFC number 53] Use Lock-Free Message Queue Improving Hoodie Writing Efficiency. (#5562)

  Show description for ecd47e7
  

  zhangyue19921010

  and

  yuezhang

  authored
  ecd47e7

  Copy full SHA for ecd47e7

Commits on May 13, 2022

• [HUDI-4018][HUDI-4027] Adding integ test yamls for immutable use-cases. Added delete partition support to integ tests (#5501)

  Show description for 0cec955
  nsivabalan

  authored
  0cec955

  Copy full SHA for 0cec955

• [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink (#5528)

  Show description for 701f8c0
  cuibo01

  authored
  701f8c0

  Copy full SHA for 701f8c0

• [MINOR] Fix a NPE for Option (#5461)

  xccui

  authored
  8ad0bb9

  Copy full SHA for 8ad0bb9

• [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact… (#5545)

  Show description for 7fb436d
  cuibo01

  authored
  7fb436d

  Copy full SHA for 7fb436d

• [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink (#5574)

  Show description for a704e37
  cuibo01

  authored
  a704e37

  Copy full SHA for a704e37

• [HUDI-4072] Fix NULL schema for empty batches in deltastreamer (#5543)

  nsivabalan

  authored
  5c4813f

  Copy full SHA for 5c4813f

Commits on May 14, 2022

• [HUDI-4097] add table info to jobStatus (#5529)

  Show description for 52e63b3
  

  wqwl611

  and

  wqwl611

  authored
  52e63b3

  Copy full SHA for 52e63b3

• [HUDI-3980] Suport kerberos hbase index (#5464)

  Show description for 6e16e71
  

  xicm

  and

  xicm

  authored
  6e16e71

  Copy full SHA for 6e16e71

Commits on May 16, 2022

• [HUDI-4001] Filter the properties should not be used when create table for Spark SQL (#5495)

  dongkelun

  authored
  75f8476

  Copy full SHA for 75f8476

• fix hive sync no partition table error (#5585)

  bettermouse

  authored
  1fded18

  Copy full SHA for 1fded18

• [HUDI-3123] consistent hashing index: basic write path (upsert/insert) (#4480)

  Show description for 61030d8
  YuweiXiao

  authored
  61030d8

  Copy full SHA for 61030d8

• [HUDI-4098] Metadata table heartbeat for instant has expired, last heartbeat 0 (#5583)

  danny0405

  authored
  43e0819

  Copy full SHA for 43e0819

• [HUDI-4103] [HUDI-4001] Filter the properties should not be used when create table for Spark SQL

  dongkelun

  authored
  a7a42e4

  Copy full SHA for a7a42e4

Commits on May 17, 2022

• [HUDI-3654] Preparations for hudi metastore. (#5572)

  Show description for ad773b3
  

  minihippo

  and

  gengxiaoyu

  authored
  ad773b3

  Copy full SHA for ad773b3

• [HUDI-4104] DeltaWriteProfile includes the pending compaction file slice when deciding small buckets (#5594)

  danny0405

  authored
  fdd96cc

  Copy full SHA for fdd96cc

• [HUDI-4101] BucketIndexPartitioner should take partition path for better dispersion (#5590)

  danny0405

  authored
  d52d133

  Copy full SHA for d52d133

• [HUDI-4087] Support dropping RO and RT table in DropHoodieTableCommand (#5564)

  Show description for d422f69
  jinxing64

  authored
  d422f69

  Copy full SHA for d422f69

• [HUDI-4110] Clean the marker files for flink compaction (#5604)

  BruceKellan

  authored
  99555c8

  Copy full SHA for 99555c8

• [MINOR] Fixing spark long running yaml for non-partitioned (#5607)

  nsivabalan

  authored
  f8b9399

  Copy full SHA for f8b9399

Commits on May 18, 2022

• [minor] Some code refactoring for LogFileComparator and Instant instantiation (#5600)

  danny0405

  authored
  ebbe56e

  Copy full SHA for ebbe56e

• [HUDI-4109] Copy the old record directly when it is chosen for merging (#5603)

  danny0405

  authored
  f1f8a1a

  Copy full SHA for f1f8a1a

• Clean the marker files for flink compaction (#5611)

  Show description for a1017c6
  loukey-lj

  authored
  a1017c6

  Copy full SHA for a1017c6

• [HUDI-3942] [RFC-50] Improve Timeline Server (#5392)

  yuzhaojing

  authored
  008616c

  Copy full SHA for 008616c

• [HUDI-4111] Bump ANTLR runtime version in Spark 3.x (#5606)

  cxzl25

  authored
  199f642

  Copy full SHA for 199f642

• Revert "[HUDI-3870] Add timeout rollback for flink online compaction (#5314)" (#5622)

  Show description for 551aa95
  danny0405

  authored
  551aa95

  Copy full SHA for 551aa95

Commits on May 19, 2022

• [HUDI-4116] Unify clustering/compaction related procedures' output type (#5620)

  Show description for 6573469
  huberylee

  authored
  6573469

  Copy full SHA for 6573469

• [HUDI-4114] Remove the unnecessary fs view sync for BaseWriteClient#initTable (#5617)

  Show description for 6f37863
  danny0405

  authored
  6f37863

  Copy full SHA for 6f37863

Commits on May 20, 2022

• [HUDI-4119] the first read result is incorrect when Flink upsert- Kafka connector is used in HUDi (#5626)

  Show description for 1da0b21
  

  aliceyyan

  and

  aliceyyan

  authored
  1da0b21

  Copy full SHA for 1da0b21

• [HUDI-4130] Remove the upgrade/downgrade for flink #initTable (#5642)

  danny0405

  authored
  c7576f7

  Copy full SHA for c7576f7

• [HUDI-3985] Refactor DLASyncTool to support read hoodie table as spark datasource table (#5532)

  huberylee

  authored
  85b146d

  Copy full SHA for 85b146d

• [MINOR] Minor fixes to exception log and removing unwanted metrics flush in integ test (#5646)

  nsivabalan

  authored
  7d02b1f

  Copy full SHA for 7d02b1f

Commits on May 21, 2022

• [HUDI-4122] Fix NPE caused by adding kafka nodes (#5632)

  wangxianghu

  authored
  2af9830

  Copy full SHA for 2af9830

• [MINOR] remove unused gson test dependency (#5652)

  xushiyan

  authored
  b5adba3

  Copy full SHA for b5adba3

• [HUDI-3858] Shade javax.servlet for Spark bundle jar (#5295)

  Show description for 8ec625d
  

  zhangyue19921010

  and

  yuezhang

  authored
  8ec625d

  Copy full SHA for 8ec625d