Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-7600][kinesis] shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception #4657

Closed
wants to merge 3 commits into from
Closed

[FLINK-7600][kinesis] shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception #4657

wants to merge 3 commits into from

Conversation

bowenli86
Copy link
Member

What is the purpose of the change

we saw the following warning in Flink log:

2017-08-11 02:33:24,473 WARN  org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon  - Exception during updateCredentials
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$5.run(Daemon.java:316)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

According to discussion in awslabs/amazon-kinesis-producer#10, setting the delay to 100 will fix this issue

Brief change log

  • shorten aws credentials refresh delay to 100 millisec

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

I've been running the fixed flink-connector-kinesis for a few days on AWS, and that warning log never happens again

Does this pull request potentially affect one of the following parts:

Documentation

  • Does this pull request introduce a new feature? (no)

@bowenli86
Copy link
Member Author

@tzulitai Hi Gordon, can you please take a look at this PR?

@bowenli86 bowenli86 mentioned this pull request Sep 14, 2017
Closed
@tzulitai
Copy link
Contributor

LGTM, thanks for the investigation.

I would add a comment about the specific change though, otherwise it would be meaningless for other readers. Will add that when merging.

@bowenli86
Copy link
Member Author

@tzulitai Thank you, Gordon!

tzulitai pushed a commit to tzulitai/flink that referenced this pull request Sep 22, 2017
@bowenli86 @tzulitai
[FLINK-7600] [kinesis] Shorten credential update delay to avoid updat…
a552583 
…eCredentials Exception

The updateCredentials delay is an ignorable warning that occurs due to
the fact that the default credential update delay is longer than the
await termination timeout when shutting down KPL.

See awslabs/amazon-kinesis-producer#10 for
details.

This closes apache#4657.
@asfgit asfgit closed this in 6c1a946 Sep 22, 2017
tzulitai pushed a commit to tzulitai/flink that referenced this pull request Sep 22, 2017
@bowenli86 @tzulitai
[FLINK-7600] [kinesis] Shorten credential update delay to avoid updat…
cc0de84 
…eCredentials Exception

The updateCredentials delay is an ignorable warning that occurs due to
the fact that the default credential update delay is longer than the
await termination timeout when shutting down KPL.

See awslabs/amazon-kinesis-producer#10 for
details.

This closes apache#4657.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
component=Connectors/Kinesis
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bowenli86 @tzulitai @rmetzger