Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add truncation to status reporting #386

Open
kdorosh opened this issue Oct 4, 2022 · 1 comment
Open

Add truncation to status reporting #386

kdorosh opened this issue Oct 4, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@kdorosh
Copy link
Contributor

kdorosh commented Oct 4, 2022

similar to solo-io/solo-kit#523

we should protect etcd from gloo mesh as a potentially buggy or erroneous client. if resources get too large, we can take down their k8s cluster itself and put GM into a state where it cannot start.

the proposed solution:

  • try to truncate status before writing
  • if that fails and still too large, don't write the status

cc @Sodman @ilackarms @EItanya (I'm told this may not be an issue for GM, but not sure why we are so confident)

@nrjpoddar as part of stability effort

@kdorosh kdorosh added the bug Something isn't working label Oct 4, 2022
@Sodman
Copy link
Member

Sodman commented Oct 4, 2022

The design for Gloo Mesh statuses was that the entire status would be written to and held in redis, but the status written back to the actual CRs in k8s would be truncated/trimmed to a reasonable size, eg max 10 entities. For large statuses, users can get more information via the UI, and we also plan to add tooling to meshctl to help with this.

Looking at the code however, I'm not sure this has been implemented yet, at least I'm not seeing the truncation logic where I'd expect it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants