Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reserve overhead when validating that a Filesystem has enough space #1319

Merged
merged 8 commits into from
Oct 1, 2020

Commits on Sep 24, 2020

  1. When validating disk space, reserve space for filesystem overhead

    The amount of available space in a filesystem is not exactly
    the advertise amount. Things like indirect blocks or metadata
    may use up some of this space. Reserving it to avoid reaching
    full capacity by default.
    
    This value is configurable from the CDIConfig object spec,
    both globally and per-storageclass.
    
    The default value is 0.055, or "5.5% of the space is
    reserved". This value was chosen because some filesystems
    reserve 5% of the space as overhead for the root user and
    this space doubles as reservation for the worst case
    behaviour for unclear space usage. I've chosen a value
    that is slightly higher.
    
    This validation is only necessary because we use sparse
    images instead of fallocated ones, which was done to have
    reasonable alerts regarding space usage from various
    storage providers.
    
    ---
    
    Update CDIConfig filesystemOverhead status, validate, and
    pass the final value to importer/upload pods.
    
    Only the status values controlled by the config controller
    are used, and it's filled out for all available storage
    classes in the cluster.
    
    Use this value in Validate calls to ensure that some of the
    space is reserved for the filesystem overhead to guard from
    accidents.
    
    Caveats:
    
    Doesn't use Default: to define the default of 0.055, instead
    it is hard-coded in reconcile. It seems like we can't use a
    default value.
    
    Validates the per-storageClass values in reconcile, and
    doesn't reject bad values.
    
    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    41a371b View commit details
    Browse the repository at this point in the history
  2. Use util GetStorageClassByName

    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    e644de8 View commit details
    Browse the repository at this point in the history
  3. Test filesystem overhead validation against async upload endpoint

    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    eca28b1 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2020

  1. wait for NFS PVs to be deleted before continuing

    Intended to help with flakes, but didn't make a difference.
    Probably still worth doing.
    
    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    ac7efc3 View commit details
    Browse the repository at this point in the history
  2. Avoid using the uncached client unnecessarily

    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    838a5d3 View commit details
    Browse the repository at this point in the history
  3. Add error handling for the case where even a default SC is not found

    Note that this change isn't expected to make a difference, as we
    check if the targetStorageClass is nil later on and have the same
    behaviour, but this is probably more correct API usage.
    
    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    208231a View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2020

  1. Add testing for the validation of filesystem overhead values

    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Oct 1, 2020
    Configuration menu
    Copy the full SHA
    cec1791 View commit details
    Browse the repository at this point in the history
  2. Fix logical error in waiting for NFS PVs.

    Wait for all of them, not just the last one.
    
    Signed-off-by: Maya Rashish <mrashish@redhat.com>
    maya-r committed Oct 1, 2020
    Configuration menu
    Copy the full SHA
    89a137d View commit details
    Browse the repository at this point in the history