Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: incorrect pod matcher for compactor in mixin when using ssd mode #12846

Merged

Conversation

QuentinBisson
Copy link
Contributor

What this PR does / why we need it:

This PR fixes the compactor pod matchers in ssd mode. I was sure I covered them all but I apparently missed one again @cstyan :)

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@QuentinBisson QuentinBisson requested a review from a team as a code owner May 1, 2024 14:59
@QuentinBisson
Copy link
Contributor Author

@cstyan could you please take a look at this one ?

Copy link
Contributor

@cstyan cstyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was confused for a minute how this could have made it past the linting check, but at first glance I missed that there was another change to the precompiled jsonnet dashboard

Comment on lines 7 to 8
else if $._config.ssd.enabled then 'container="loki", pod=~"%s-backend.*"' % $._config.ssd.pod_prefix_matcher
else 'container="compactor"',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could have been okay without this change? and just doing the container="compactor" -> compactor_matcher change below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh definitely, I just thought it was easier to read. If you prefer I will change this tomorrow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think then 'pod=~"(compactor|%s-backend.*|loki-single-binary)"' % $._config.ssd.pod_prefix_matcher should cover the case where we were previously used container="compactor" right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should yes but I'm not familiar with loki in distributed mode.

Then we could change

local compactor_matcher = if $._config.meta_monitoring.enabled
  then 'pod=~"(compactor|%s-backend.*|loki-single-binary)"' % $._config.ssd.pod_prefix_matcher
  else if $._config.ssd.enabled then 'container="loki", pod=~"%s-backend.*"' % $._config.ssd.pod_prefix_matcher
  else 'container="compactor"'

to just:

local compactor_matcher = 'pod=~"(compactor|%s-backend.*|loki-single-binary)"' % $._config.ssd.pod_prefix_matcher

right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's mostly right, but it looks like it wouldn't work for %s-backend % $._config.ssd.pod_prefix_matcher because AFAICT the default value for that variable is (loki.*|enterprise-logs)

@MichelHollands can you confirm whether we really need separate selectors for when meta monitoring is enabled?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what would you like me to do here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichelHollands can you confirm what the matcher should be when meta montioring is enabled? I still think @QuentinBisson suggestion of local compactor_matcher = 'pod=~"(compactor|%s-backend.*|loki-single-binary)"' % $._config.ssd.pod_prefix_matcher covers all cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @MichelHollands would you have time to check this one? :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@QuentinBisson lets move forward with the simplified matcher you've suggested and open follow ups with fixes if anyone reports any

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pushed :)

@cstyan cstyan changed the title fix: incorrect pod matcher for compactor in mixin fix: incorrect pod matcher for compactor in mixin when using ssd mode Jul 17, 2024
@cstyan cstyan merged commit 515e13c into grafana:main Jul 17, 2024
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants