Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Add full program benchmark for kustomize build #5425

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Commits on Apr 16, 2024

  1. perf: Add full program benchmark for kustomize build

    This change introduces a benchmarking test that constructs a
    complete kustomization tree using various features of Kustomize.
    
    This update aims to address several objectives:
    * Demonstrating current performance challenges in Kustomize in a reproducible manner.
    * Evaluating the effects of performance enhancements.
    * Guarding against potential performance setbacks and inadvertent quadratic behavior in the future.
    * Considering the possibility of incorporating profile-guided optimization (PGO) in future iterations.
    
    Usage:
    
        $ make run-benchmarks
        go test -run=x -bench=BenchmarkBuild ./kustomize/commands/build
    
        pkg: sigs.k8s.io/kustomize/kustomize/v5/commands/build
        BenchmarkBuild-8               1        48035946042 ns/op
        PASS
        ok      sigs.k8s.io/kustomize/kustomize/v5/commands/build       48.357s
    
    *Currently*, this benchmark requires 48 seconds to run on my machine.
    
    Updates kubernetes-sigs#5084
    
    Notes on PGO:
    
    Real-life profiles would be better, but creating one based on a benchmark should not hurt:
    
    https://go.dev/doc/pgo#collecting-profiles
    
    > Will PGO with an unrepresentative profile make my program slower than no PGO?
    > It should not. While a profile that is not representative of production behavior will result in optimizations in cold parts of the application, it should not make hot parts of the application slower. If you encounter a program where PGO results in worse performance than disabling PGO, please file an issue at https://go.dev/issue/new.
    
    Collecting a profile:
    
        go test -cpuprofile cpu1.pprof -run=^$ -bench ^BenchmarkBuild$ sigs.k8s.io/kustomize/kustomize/v5/commands/build
    
        go build -pgo=./cpu1.pprof  -o kust-pgo ./kustomize
        go build -o kust-nopgo ./kustomize
    
    Compare PGO and non-PGO-builds:
    
        ./kust-pgo build -o /dev/null testdata/  21.88s user 2.00s system 176% cpu 13.505 total
        ./kust-nopgo build -o /dev/null testdata/  22.76s user 1.98s system 174% cpu 14.170 total
    chlunde committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3fe3dfd View commit details
    Browse the repository at this point in the history