Add profiling to API server

[sayan-biswas]

Changes

This feature add golang's native profiling server to expose debug profiles.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you review them:

• Has Docs included if any changes are user facing
• Has Tests included if any functionality added or changed
• Tested your changes locally (if this is a code change)
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user-facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
• Release notes contain the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-results-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/api/main.go	11.9%	11.4%	-0.5
pkg/api/server/config/config.go	Do not exist	0.0%

[sayan-biswas]

@gabemontero Here's the PR for enabling basic pprof.

[khrm]

/approve

[khrm]

We also need to put this Port in the service, isn't it? Or we can just port-forward.

[sayan-biswas]

Although we can port-forward from pod also, I think having it in the service is the right way.
I'll make the changes, also fix the lint issue.

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-results-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/api/main.go	11.9%	11.4%	-0.5
pkg/api/server/config/config.go	Do not exist	0.0%
pkg/api/server/config/config.go	Do not exist	0.0%
pkg/api/server/config/config.go	Do not exist	0.0%
pkg/api/server/config/config.go	Do not exist	0.0%

[khrm]

/approve
This looks good to me.
@avinal @gabemontero

[gabemontero]

/lgtm

yeah per the wg call let's start with this vs. the knative approach I tried, and we can iterate on this over time

next time I get back to api perf testing I hope to try this out, either in conjunction, or as an alternative to, my temporary thread dumps in known problematic calls like ListRecords or UpdateLogs

[tekton-robot]

@gabemontero: changing LGTM is restricted to collaborators

In response to this:

/lgtm

yeah per the wg call let's start with this vs. the knative approach I tried, and we can iterate on this over time

next time I get back to api perf testing I hope to try this out, either in conjunction, or as an alternative to, my temporary thread dumps in known problematic calls like ListRecords or UpdateLogs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[khrm]

@enarha Let's merge this.

[avinal]

/lgtm

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: avinal, gabemontero, khrm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [avinal,khrm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gabemontero]

fyi @sayan-biswas I just tried this and it looks like the PROFILING_PORT defaulting to 6060 does not take effect

{"level":"info","ts":1715275165.595052,"caller":"api/main.go:237","msg":"Profiling server listening on: "}
{"level":"info","ts":1715275165.5953324,"caller":"api/main.go:259","msg":"Prometheus server listening on: 9090"}

I tried 6060, 8008, and few others anyway, no luck. Unfortunately, you can't run command like netstat in the container.

I'll try explicitly setting it to 6060 and see what happens

[gabemontero]

it works OK when I explicitly set PROFILING_PORT @sayan-biswas