You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jan 5, 2024. It is now read-only.
I'm trying to use Hudi sink but the write to Hudi-client is not working. Has anyone seen this issue before:
2019-10-08 20:29:46 INFO HoodieTableConfig:69 - Loading dataset properties from /tmp/hoodie/payment_reports_18/.hoodie/hoodie.properties
2019-10-08 20:29:46 INFO HoodieTableMetaClient:94 - Finished Loading Table of type COPY_ON_WRITE from /tmp/hoodie/payment_reports_18
2019-10-08 20:29:46 INFO HoodieTableMetaClient:96 - Loading Active commit timeline for /tmp/hoodie/payment_reports_18
2019-10-08 20:29:46 INFO HoodieActiveTimeline:77 - Loaded instants java.util.stream.ReferencePipeline$Head@1a850eab
2019-10-08 20:29:46 INFO HoodieActiveTimeline:212 - Marking instant complete [==>20191008202942__commit__INFLIGHT]
2019-10-08 20:29:46 INFO HoodieActiveTimeline:377 - Created a new file in meta path: /tmp/hoodie/payment_reports_18/.hoodie/20191008202942.inflight
2019-10-08 20:29:46 ERROR JobDag:188 - Failed in JobDag
com.uber.hoodie.exception.HoodieIOException: Could not rename /tmp/hoodie/payment_reports_18/.hoodie/20191008202942.inflight to /tmp/hoodie/payment_reports_18/.hoodie/20191008202942.commit
at com.uber.hoodie.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:330)
at com.uber.hoodie.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:215)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:518)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:491)
at com.uber.marmaray.common.sinks.hoodie.HoodieSink$HoodieWriteClientWrapper.commit(HoodieSink.java:489)
at com.uber.marmaray.common.sinks.hoodie.HoodieSink.commit(HoodieSink.java:290)
at com.uber.marmaray.common.sinks.hoodie.HoodieSink.commit(HoodieSink.java:267)
at com.uber.marmaray.common.sinks.hoodie.HoodieSink.write(HoodieSink.java:184)
at com.uber.marmaray.common.sinks.hoodie.HoodieSink.write(HoodieSink.java:161)
at com.uber.marmaray.common.job.SingleSinkSubDag.executeNode(SingleSinkSubDag.java:51)
at com.uber.marmaray.common.job.JobSubDag.execute(JobSubDag.java:149)
at com.uber.marmaray.common.job.JobDag.execute(JobDag.java:171)
at com.uber.marmaray.common.job.JobManager.lambda$null$0(JobManager.java:207)
at com.uber.marmaray.common.job.ThreadPoolService$ThreadPoolServiceCallable.call(ThreadPoolService.java:415)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The text was updated successfully, but these errors were encountered:
2020-02-10 18:04:24,707 [main] INFO com.uber.hoodie.common.table.timeline.HoodieActiveTimeline - Marking instant complete [==>20200210180348__commit__INFLIGHT]
Exception in thread "main" com.uber.hoodie.exception.HoodieIOException: Could not rename s3a://x/.hoodie/20200210180348.inflight to s3a://x/.hoodie/20200210180348.commit
at com.uber.hoodie.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:328)
at com.uber.hoodie.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:213)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:529)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:489)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:480)
I'm trying to use Hudi sink but the write to Hudi-client is not working. Has anyone seen this issue before:
The text was updated successfully, but these errors were encountered: