Add native support/handling for Finalizers

[mjnagel]

Is your feature request related to a problem? Please describe.

When building operators with Pepr and other tools it is often useful to add a finalizer to custom resources to handle cleanup when a resource is deleted.

To manage finalizers an end user has to:

• Add a finalizer to new CRs that are created (could be done via mutation or a patch)
• Watch for resource updates that contain the deletionTimestamp metadata field (when a resource with a finalizer is deleted it appears as an update, rather than a deletion)
• Based on ^ filter updates to a resource to either perform some cleanup OR process "reconciliation"
• Remove the finalizer to finish the resource deletion

Describe the solution you'd like

It would be great if Pepr natively supported the concept of finalizers and handled the operations around them for me. What this might look like in my code (referencing uds-core):

When(UDSPackage)
  .IsCreatedOrUpdated()
  // Advanced CR validation
  .Validate(validator)
  // Enqueue the package for processing
  .Reconcile(packageReconciler)
  // Run a `cleanupPackage` function for the finalizer
  .Finalizer(cleanupPackage);

In effect this would mean that Pepr would need to manage:

• Adding the finalizer to the resource
• Triggering the function I specify at the proper time (when a deletionTimestamp is added)
• Removing the finalizer after the function completes

Describe alternatives you've considered

This could be left to individual module developers to handle on their own, which is a reasonable path all things considered (I did a rough draft of this here). However this does seem like it may be a common need as people develop operators with Pepr.

Additional context

One thing to note - it is common to patch status on a resource when deletion kicks off, while finalizers are still running. Typically you'd see this reflected as a Terminating or Deleting phase on the resource. This could be beneficial for Pepr to also include in the finalization lifecycle, but I'm unsure if that would be possible since the CRD may have a restrictive/custom type for their status field. The other option would be for the module creator to have it as part of their "cleanup" function, and either Pepr or the cleanup function would need to properly filter out the status update event so that finalization only happens once.

References for other "operator builders" and how they handle this:

• Operator SDK: Would be nearly a 1:1 with the proposal in this issue, operator author adds a finalizer to their watch and the SDK manages the lifecycle of that
• KubeBuilder: Provides some helper functionality to add/remove finalizers but requires operator author to handle the lifecycle otherwise
• KUDO: Appears to handle something like finalizers, but not as sure if this is 1:1/uses finalizers - KUDO is no longer an active project so might not be a good reference