Skip to content
This repository has been archived by the owner on Sep 26, 2023. It is now read-only.

Remove s3committer in Transformer #158

Closed
chuwy opened this issue Nov 17, 2021 · 0 comments
Closed

Remove s3committer in Transformer #158

chuwy opened this issue Nov 17, 2021 · 0 comments
Assignees
@chuwy
Milestone
0.9.0

Comments

@chuwy
Copy link
Contributor

chuwy commented Nov 17, 2021

Ref #67

For a long time we were using custom Netflix committer in Transformer codebase to avoid a bottleneck in committing results due v1 protocol renaming. Now EMR provides multiple options out of the box:

In this commit I'm planning to remove the s3committer from working tree and allow users to switch between s3 and s3a
@chuwy chuwy added this to the 0.8.3 milestone Nov 17, 2021
@chuwy chuwy self-assigned this Nov 17, 2021
chuwy added a commit that referenced this issue Nov 17, 2021
@chuwy
Remove s3committer in Transformer (close #158)
1507ae4
@chuwy chuwy closed this as completed in 0aef734 Nov 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@chuwy chuwy

Labels
None yet
Projects
None yet
Milestone
0.9.0
Development

No branches or pull requests
1 participant
@chuwy