Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CheckOnHostCommand: add missing timeout setting #9677

Open
wants to merge 1 commit into
base: 4.19
Choose a base branch
from

Conversation

rp-
Copy link
Contributor

@rp- rp- commented Sep 13, 2024

Description

The new CheckOnHostCommand constructor was missing a reasonable timeout value, which meant it would fallback to the wait (1800s) timeout. On a Linstor cluster this resulted in over 15 minutes wait time until a host was recognized as down.
With timeout of 20s (as the other constructor) it takes 4-5 mins for a host to become recognized as down.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Failover tests (force shutdown of a host) in a Linstor cluster.

How did you try to break this feature and the system with this change?

Copy link

codecov bot commented Sep 13, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 15.11%. Comparing base (a0932b0) to head (eca66f8).
Report is 13 commits behind head on 4.19.

Files with missing lines Patch % Lines
...n/java/com/cloud/agent/api/CheckOnHostCommand.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.19    #9677      +/-   ##
============================================
+ Coverage     15.08%   15.11%   +0.02%     
+ Complexity    11192    11190       -2     
============================================
  Files          5406     5406              
  Lines        473215   473214       -1     
  Branches      61680    58585    -3095     
============================================
+ Hits          71386    71521     +135     
- Misses       393880   393883       +3     
+ Partials       7949     7810     -139     
Flag Coverage Δ
uitests 4.76% <ø> (+0.46%) ⬆️
unittests 15.80% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

The new CheckOnHostCommand constructor was missing a reasonable timeout
value, which meant it would fallback to the wait (1800s) timeout.
On a Linstor cluster this resulted in over 15 minutes wait time until
a host was recognized as down.
With timeout of 20s (as the other constructor) it takes 4-5 mins for a host
to become recognized as down.
@rp- rp- force-pushed the checkonhostcommand-missing-timeout branch from 5ce9077 to eca66f8 Compare September 16, 2024 07:58
@weizhouapache
Copy link
Member

@blueorangutan package

Copy link
Member

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@sureshanaparti
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11163

Copy link
Contributor

@slavkap slavkap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code LGTM but I haven't tested it

@rohityadavcloud rohityadavcloud added this to the 4.19.2.0 milestone Oct 17, 2024
@rohityadavcloud
Copy link
Member

@blueorangutan package

@blueorangutan
Copy link

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11374

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-11709)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 43298 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9677-t11709-kvm-ol8.zip
Smoke tests completed. 133 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants