Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce wait time between resizes #274

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

namm2
Copy link

@namm2 namm2 commented Jul 16, 2024

I'd like to add support for pvc-autoresizer controller to wait x hours between resizes to comply with cloud providers like AWS.

Signed-off-by: Nam Hai Nguyen <namhai.nguyen-ext@commercetools.com>
@namm2 namm2 requested a review from a team as a code owner July 16, 2024 07:57
@llamerada-jp
Copy link
Contributor

Could you please tell me more about the AWS storage behavior, limitations, etc. that would explain why this feature is needed? For example, are there any side effects, such as charges for performing resizing at high frequencies?

Signed-off-by: Nam Hai Nguyen <namhai.nguyen-ext@commercetools.com>
@namm2
Copy link
Author

namm2 commented Jul 17, 2024

Could you please tell me more about the AWS storage behavior, limitations, etc. that would explain why this feature is needed? For example, are there any side effects, such as charges for performing resizing at high frequencies?

This is the limitation of AWS EBS volumes:

After modifying a volume, you must wait at least six hours and ensure that the volume is in the in-use or available state before you can modify the same volume.

I'm not aware of any additional charges for high API requests, but having this time check would reduce the useless API calls up to 6 hours .

https://docs.aws.amazon.com/ebs/latest/userguide/requesting-ebs-volume-modifications.html#modify-ebs-volume

@llamerada-jp
Copy link
Contributor

I have read the document you gave me and it reads that the 6h is a guideline and nothing is guaranteed for the time. It is also unclear what actually happens if this is not adhered to. What are the real issues that have occurred at your site?

@namm2
Copy link
Author

namm2 commented Jul 17, 2024

Yes, 6h is just the guideline and I could set to any thing >= 6h to "comply" with AWS. This is not a big issue but optimization for the known limitations, we (ops) know that don't bother to increase the EBS volume size again if it's just resized in < 6h, then the controller should behave the same.

@llamerada-jp
Copy link
Contributor

Upon investigation, I found that modifying the pvc-autoresizer does not satisfy the guidelines. First, the guidelines say to ensure if it is in use or available, but it is not possible to ensure these states by the pvc-autoresizer. Also, it cannot handle cases where the PVC has been edited by another controller or manually.
I believe that the EBS CSI driver can provide functionality to satisfy this guideline. The EBS CSI driver can check the status. Also, it can handle modifications by anyone. It can probably handle modifications from outside of K8s. As far as I checked the EBS CSI driver code, the above features have not been implemented yet, so I recommend you discuss them with them first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

None yet

3 participants