Commits on Jul 31, 2024

1. [PROF-10128] Fix profiling benchmarking configuration still having au… …

   …to_instrument overhead
   
   **What does this PR do?**
   
   This PR tweaks the `ADD_TO_GEMFILE` environment variable for
   benchmarking configurations where only the profiler should be active to
   not include `require: 'datadog/auto_instrument'`.
   
   It turns out this change was added in #3614 when we fixed the benchmarks
   not running. Loading the instrumentation code and then disabling tracing
   has a measurable performance overhead vs not loading it at all (20%
   regression of median throughput in high load gitlab tests).
   
   **Motivation:**
   
   I expect that with this change, the `only-profiling` configuration will
   reflect historical data we had before the change in #3614.
   
   This is relevant because we use this data as basis for comparison with
   other profiling features (e.g. allocation) and thus it's important to
   have a correct baseline.
   
   **Additional Notes:**
   
   I noticed a similar-ish configuration issue in our reliability
   environment. I'll open a separate PR for that one as well.
   
   **How to test the change?**
   
   I've tested a similar configuration using `bp-runner` locally and
   confirmed that overhead is reduced. Once we merge this in, I'll validate
   that the nightly runs show performance is back to where it was on the
   18th of April.

   

   ivoanjo committed Jul 31, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for a795558
   • Browse repository at this point

   Copy the full SHA

   a795558 View commit details

   Browse the repository at this point in the history