operator/certrotation: introduce an optional lock to CABundleConfigMap and RotatedSigningCASecret

[p0lyn0mial]

To preserve the current behaviour and make the controllers thread safe
I've decided to introduce an optional mutex which can be used when the instance
is shared across multiple controllers.

This approach requires the fewest changes and resolves the race condition.

Perhaps the best approach would be to create separate controllers for CABundleConfigMap and RotatedSigningCASecret.

proof: https://gist.github.com/p0lyn0mial/176bece543907dbd9ac6b5306a170eda

a simpler version of #1752

Alternative for #1722

[p0lyn0mial]

/assign @vrutkovs @tkashem

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: p0lyn0mial
Once this PR has been reviewed and has the lgtm label, please assign soltysh for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/operator/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@p0lyn0mial: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[p0lyn0mial]

/hold

before merging we should run the test that fails due to the race condition, see: #1722 (comment)

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[p0lyn0mial]

/close

now, once we know where the issue is we can apply a different fix #1763