Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize history calculation to reuse previously computed results #115

Merged
merged 6 commits into from
Mar 29, 2022

Conversation

hannesg
Copy link
Member

@hannesg hannesg commented Mar 18, 2022

The history algorithm currently recalculates the whole history for the repository on every run. However, most of the things in a repo do not change on each commit. I introduced a cache that memorizes the last ten calculation results so that we can reuse them later. This bring the history calculation in most cases down from O(N) to O(1) for N = number of commits in repo.

Benchmark:

/build/services/cd-service/pkg/history $ go test -benchmem -bench=.
goos: linux
goarch: amd64
pkg: github.com/freiheit-com/kuberpult/services/cd-service/pkg/history
cpu: Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
BenchmarkHistoryNoCache-12             4         289058522 ns/op        36413094 B/op    2330716 allocs/op
BenchmarkHistoryCache-12             465           2560525 ns/op          596726 B/op      23879 allocs/op
PASS
ok      github.com/freiheit-com/kuberpult/services/cd-service/pkg/history       45.740s

Those who do not --learn-from-- cache history are doomed to repeat it.

Lenin

services/cd-service/pkg/history/history.go Outdated Show resolved Hide resolved
services/cd-service/pkg/history/history.go Outdated Show resolved Hide resolved
Copy link
Contributor

@medo-freiheit medo-freiheit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea and the new LRU part looks great, but I got confused with history structure, so maybe @sven-urbanski-freiheit-com could review this

services/cd-service/pkg/history/history.go Outdated Show resolved Hide resolved
services/cd-service/pkg/history/history.go Outdated Show resolved Hide resolved
@hannesg hannesg merged commit cdbcec8 into main Mar 29, 2022
@hannesg hannesg deleted the lenin branch March 29, 2022 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants