optimize engine cache checker #539
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| }() | ||
|
|
||
| keysToCheck.Range(func(key, _ interface{}) bool { | ||
| pool.Go(ctx, func() { |
There was a problem hiding this comment.
sync.Map. Range 里用 goroutine 好像会导致并发不安全, 本来有互斥锁, 但是 goroutine 逃逸了这个锁.
There was a problem hiding this comment.
改了，不知道为啥这里没标outdated
DuodenumL
force-pushed
the
feature/engine-cache
branch
from
aa94e3e to
7e17f11
Compare
CMGS
reviewed
Jan 19, 2022
| } | ||
|
|
||
| // ImageRemoteDigest . | ||
| func (f *Engine) ImageRemoteDigest(ctx context.Context, image string) (string, error) { | ||
| return "", types.ErrNilEngine | ||
| return "", f.DefaultErr | ||
| } | ||
|
|
||
| // ImageBuildFromExist . | ||
| func (f *Engine) ImageBuildFromExist(ctx context.Context, ID string, refs []string, user string) (string, error) { |
There was a problem hiding this comment.
这里是为了让error进入缓存，比如创建Engine的时候错误是connection refused，那么就让这个fake engine每次调用都返回connection refused
engine/factory/factory.go
Outdated
| } | ||
| if err := validateEngine(ctx, client, config.ConnectionTimeout); err != nil { | ||
| log.Errorf(ctx, "[EngineCacheChecker] engine %v is unavailable, will be replaced with a fake engine, err: %v", cacheKey, err) | ||
| engineCache.Set(cacheKey, &fake.Engine{DefaultErr: err}) |
There was a problem hiding this comment.
这里更新 cache 用了两步操作, 先 delete 再 set, 可以合并成一步吗, 合并成一步是不是就并发安全了
engine/factory/factory.go
Outdated
| // EngineCacheChecker checks if the engine in cache is available | ||
| func EngineCacheChecker(ctx context.Context, timeout time.Duration) { | ||
| func EngineCacheChecker(ctx context.Context, config types.Config) { |
There was a problem hiding this comment.
这里已经应该建模了吧, 不要再操作一个全局变量了, 创建一个结构体, 包含 cache, 然后这个函数作为其中一个方法.
DuodenumL
force-pushed
the
feature/engine-cache
branch
3 times, most recently
from
44dd656 to
b60db40
Compare
engine/factory/factory.go
Outdated
| }() | ||
|
|
||
| // getEngine get engine | ||
| func getEngine(ctx context.Context, config types.Config, nodename, endpoint, ca, cert, key string) (client engine.API, err error) { |
DuodenumL
force-pushed
the
feature/engine-cache
branch
from
b60db40 to
472f41c
Compare
jschwinger233
approved these changes
Jan 19, 2022
跑了一下，截至目前没发现问题 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
优化了一下Engine Cache Check的逻辑，即使GetEngine报错了也要记录到缓存里，否则在endpoint三层不通的时候，每次ping都要花很长时间，导致一些api例如GetWorkload耗时巨大。