Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Ingester RF-1 #13365

Merged
merged 9 commits into from
Jul 3, 2024
Merged

feat: Ingester RF-1 #13365

merged 9 commits into from
Jul 3, 2024

Conversation

grobinson-grafana
Copy link
Contributor

@grobinson-grafana grobinson-grafana commented Jul 1, 2024

What this PR does / why we need it:

This pull request starts a work-in-progress of Ingester RF-1 (replication factor = 1). It is disabled by default.

Co-authored by @benclive and @grobinson-grafana.

Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@grobinson-grafana grobinson-grafana self-assigned this Jul 1, 2024
@grobinson-grafana grobinson-grafana requested a review from a team as a code owner July 1, 2024 14:08
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Jul 1, 2024
This commit is a work-in-progress of Ingester RF-1 (replication
factor = 1). It is disabled by default.
@grobinson-grafana grobinson-grafana force-pushed the grobinson/ingester-rf1 branch from 2afad46 to 64306a0 Compare July 1, 2024 14:19
@grobinson-grafana grobinson-grafana changed the title Ingester RF-1 feat: Ingester RF-1 Jul 1, 2024
@@ -159,7 +163,9 @@ func (c *Config) RegisterFlags(f *flag.FlagSet) {
c.CompactorHTTPClient.RegisterFlags(f)
c.CompactorGRPCClient.RegisterFlags(f)
c.IngesterClient.RegisterFlags(f)
//c.IngesterRF1Client.RegisterFlags(f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//c.IngesterRF1Client.RegisterFlags(f)
//c.IngesterRF1Client.RegisterFlags(f)

Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

A lot will change but let's unlock so we can play with this.

Copy link
Contributor

@benclive benclive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - A couple of small things to tidy up but not worth blocking for since we'll be revisiting this soon

cmd/loki/loki-local-config.yaml Outdated Show resolved Hide resolved
pkg/loki/loki.go Outdated Show resolved Hide resolved
pkg/storage/chunk/client/gcp/bigtable_object_client.go Outdated Show resolved Hide resolved
case <-i.loopQuit:
return
}
}
}

func (i *Ingester) doFlushTick() {
i.flushCtx.lock.Lock()
defer i.flushCtx.lock.Unlock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably doesn't hurt but we actually don't want to unlock this - the context is only locked once when its no longer accepting writes.
I wonder if its possible that this goroutine can gets pre-empted and another tick can fire before we swap in a new context? In that case we'd block the timer thread completely waiting on the lock... Best to leave the unlock here for now in that case!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right!

@grobinson-grafana grobinson-grafana merged commit 7f35179 into main Jul 3, 2024
61 checks passed
@grobinson-grafana grobinson-grafana deleted the grobinson/ingester-rf1 branch July 3, 2024 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/XXL type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants