Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix-9352][datasource-api]data source client uses a cache with an expiration time #9353

Merged
merged 11 commits into from
Apr 27, 2022

Conversation

zhuxt2015
Copy link
Contributor

@zhuxt2015 zhuxt2015 commented Apr 5, 2022

Purpose of the pull request

This pull request add a cache with an expiration time for all data source client

close #9352

Brief change log

use Guava Cache for data source client instead of ConcurrentHashMap

Verify this pull request

This pull request is already covered by existing tests, DatasourceUtilTest.testCheckConnection

@codecov-commenter
Copy link

codecov-commenter commented Apr 5, 2022

Codecov Report

Merging #9353 (7ab75c2) into dev (88d2803) will increase coverage by 0.04%.
The diff coverage is 31.25%.

@@             Coverage Diff              @@
##                dev    #9353      +/-   ##
============================================
+ Coverage     40.01%   40.06%   +0.04%     
- Complexity     4475     4482       +7     
============================================
  Files           835      835              
  Lines         33550    33599      +49     
  Branches       3710     3713       +3     
============================================
+ Hits          13426    13462      +36     
- Misses        18876    18887      +11     
- Partials       1248     1250       +2     
Impacted Files Coverage Δ
...r/plugin/datasource/hive/HiveDataSourceClient.java 1.35% <0.00%> (-0.17%) ⬇️
.../dolphinscheduler/plugin/task/datax/DataxTask.java 0.00% <0.00%> (ø)
...atasource/api/plugin/DataSourceClientProvider.java 25.00% <83.33%> (+15.00%) ⬆️
...dolphinscheduler/plugin/alert/http/HttpSender.java 53.96% <0.00%> (-4.97%) ⬇️
...org/apache/dolphinscheduler/remote/utils/Host.java 37.77% <0.00%> (-2.23%) ⬇️
...e/dolphinscheduler/remote/NettyRemotingClient.java 52.11% <0.00%> (-1.41%) ⬇️
...duler/api/service/impl/EnvironmentServiceImpl.java 82.32% <0.00%> (-0.39%) ⬇️
...r/api/service/impl/ProcessInstanceServiceImpl.java 60.10% <0.00%> (-0.12%) ⬇️
...e/dolphinscheduler/server/master/MasterServer.java 0.00% <0.00%> (ø)
...e/dolphinscheduler/server/worker/WorkerServer.java 0.00% <0.00%> (ø)
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88d2803...7ab75c2. Read the comment docs.

@lenboo
Copy link
Contributor

lenboo commented Apr 6, 2022

close #9352

@davidzollo
Copy link
Contributor

LGTM

@zhuxt2015 zhuxt2015 changed the base branch from dev to 2.0.6-prepare April 23, 2022 02:45
@zhuxt2015 zhuxt2015 changed the base branch from 2.0.6-prepare to dev April 23, 2022 02:47
zhongjiajie
zhongjiajie previously approved these changes Apr 23, 2022
Copy link
Member

@zhongjiajie zhongjiajie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @caishunfeng do you have time to take a look at this PR?

@sonarcloud
Copy link

sonarcloud bot commented Apr 26, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

25.0% 25.0% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@caishunfeng caishunfeng merged commit 206b7c1 into apache:dev Apr 27, 2022
Tianqi-Dotes pushed a commit to Tianqi-Dotes/dolphinscheduler that referenced this pull request Jun 16, 2022
…iration time (apache#9353)

* datasource client cache with expiration time

* remove unused import

* fixed unit test

* fix

* fix

* fix

* fix

* fix

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] [datasource-api] Hive datasource connection failed when kerberos renew ticket lifetime expire
6 participants