Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve race condition between CARM ConfigMap and reconciler for annotated namespaces #138

Merged
merged 1 commit into from
Feb 22, 2024

Commits on Feb 19, 2024

  1. Resolve race condition between CARM ConfigMap and reconciler for anno…

    …tated namespaces
    
    In certain scenarios, where a user deploys a resource to a namespace
    annotated with a specific ownner accountID, a race condition was
    identified between the reconciler and the CARM (Cross Account Resource
    Management) `ConfigMap`. This race condition resulted in the controller
    setting an empty roleARN, preventing the aws-sdk-go client from pivoting
    (calling `STS::AssumeRole`) and managing resourecs in the correct
    account. Instead, resources were inadvertently managed in the default
    account instead of the namespace assigned account.
    
    This issue stemmed from the initial implementation of the CARM feature,
    where the method responsible for retrieving the accountID from the
    cache, didn't not properly verify the existance and content of the CARM
    configMap and instead returned an empty stringy when these conditions
    were not satisfied. This led to selection of the default account (empty
    `RoleARN` for resource management.
    
    Although these scenarios are rare, they can occur in clusters with a
    significantly high number of namespaces, causing a delay between
    naemsapce/configmap events and the informer's event handlers.
    
    This patch addresses the race issue by implementing two main things:
    - Proper error propagation: an error is no propagated when a `ConfigMap`
      is missing or when an accountID entry is missing in the `ConfigMap`.
      This helps the reconciler make the right decision on how to handle
      these cases.
    - Improved error handling: The reconciler now carefully handles these
      errors and requeues whenever a user has issued an
      owneraccountid-annotated namespace but the Configmap is not create or
      properly propagated.
    
    Signed-off-by: Amine Hilaly <hilalyamine@gmail.com>
    a-hilaly committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    91ccfed View commit details
    Browse the repository at this point in the history