-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable leader election in ws-manager-mk2 (v3) #18539
Conversation
/gh run recreate-vm=true Comment triggered a workflow runStarted workflow run: 5892091350
|
The manifest for the CRD has not been updated. Run |
This actually used a build from main ( |
2575ccb
to
b44fb71
Compare
/gh run recreate-vm=true Comment triggered a workflow runStarted workflow run: 5895723228
|
bd7f036
to
7f968fa
Compare
done |
/gh run recreate-vm=true Comment triggered a workflow runStarted workflow run: 5902316472
|
36c751a
to
c73dc1d
Compare
/gh run recreate-vm=true Comment triggered a workflow runStarted workflow run: 5912178632
|
db24b0f
to
cd14a5b
Compare
Change lgtm with some minor comments. |
wsMetrics *controllerMetrics | ||
ctx context.Context | ||
cancel context.CancelFunc | ||
wsMetrics *controllerMetrics |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do the ws-manager metrics behave when two replicas are enabled? Do both report metrics, or only one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Metrics will be scrapped from all available replicas
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will break some of the dashboards then (and maybe alerts too). For instance we sum all workspaces by cluster, this will then double the amount of reported workspaces
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please check this gist (integration tests are running there)
Only the leader will have metrics related to the controller. If the current leader is not elected anymore, the pod is restarted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for the gists. Both replicas report workspace metrics, e.g.:
gitpod_ws_manager_mk2_workspace_phase_total{class="g1-standard",phase="Running",type="Regular"} 1
is in both. Summing them as done in dashboards/alerts would give 2 replicas, while there's only 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gist updated after running the integration test
I'd also like to see some benchmarks, as both a new replica is added watching for events, and the large increase in CR activity. I don't think though that |
bba6c41
to
d390827
Compare
Description
Reverts #18537
Change:
Here we remove the activity and switch the state to the workspace status.
Summary generated by Copilot
🤖 Generated by Copilot at 27cc1a0
Refactor and improve the
ws-manager-mk2
component, update its dependencies and tests, and add a new fieldLastActivity
to theWorkspaceStatus
struct.Related Issue(s)
Fixes #
How to test
Documentation
Preview status
Gitpod was successfully deployed to your preview environment.
Build Options
Build
Run the build with werft instead of GHA
Run Leeway with
--dont-test
Publish
Installer
Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
If enabled this will build
install/preview
If enabled this will create the environment on GCE infra
Valid options are
all
,workspace
,webapp
,ide
,jetbrains
,vscode
,ssh
. If enabled,with-preview
andwith-large-vm
will be enabled./hold