Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default values for two GRPC setting we have to set so the queriers can connect to a frontend or scheduler #4435

Merged
merged 2 commits into from
Oct 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## Main
* [4400](https://github.com/grafana/loki/pull/4400) **trevorwhitney**: Config: automatically apply memberlist config too all rings when provided
* [4435](https://github.com/grafana/loki/pull/4435) **trevorwhitney**: Change default values for two GRPC settings so querier can connect to frontend/scheduler
* [4443](https://github.com/grafana/loki/pull/4443) **DylanGuedes**: Loki: Change how push API checks for contentType

# 2.3.0 (2021/08/06)
Expand Down
21 changes: 21 additions & 0 deletions docs/sources/upgrading/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,27 @@ ruler:
host: consul.namespace.svc.cluster.local:8500
```

#### Changed defaults for some GRPC server settings
* [4435](https://github.com/grafana/loki/pull/4435) **trevorwhitney**: Change default values for two GRPC settings so querier can connect to frontend/scheduler

This changes two default values, `grpc_server_min_time_between_pings` and `grpc_server_ping_without_stream_allowed` used by the GRPC server.

*Previous Values*:
```
server:
grpc_server_min_time_between_pings: '5m'
grpc_server_ping_without_stream_allowed: false
```

*New Values*:
```
server:
grpc_server_min_time_between_pings: '10s'
grpc_server_ping_without_stream_allowed: true
```

Please manually provide the values of `5m` and `true` (respectively) in your config if you rely on those values.

-_add changes here which are unreleased_


Expand Down
23 changes: 22 additions & 1 deletion pkg/loki/loki.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func (c *Config) RegisterFlags(f *flag.FlagSet) {
"The alias 'all' can be used in the list to load a number of core modules and will enable single-binary mode. ")
f.BoolVar(&c.AuthEnabled, "auth.enabled", true, "Set to false to disable auth.")

c.Server.RegisterFlags(f)
c.registerServerFlagsWithChangedDefaultValues(f)
c.Distributor.RegisterFlags(f)
c.Querier.RegisterFlags(f)
c.IngesterClient.RegisterFlags(f)
Expand All @@ -108,6 +108,27 @@ func (c *Config) RegisterFlags(f *flag.FlagSet) {
c.QueryScheduler.RegisterFlags(f)
}

func (c *Config) registerServerFlagsWithChangedDefaultValues(fs *flag.FlagSet) {
throwaway := flag.NewFlagSet("throwaway", flag.PanicOnError)

// Register to throwaway flags first. Default values are remembered during registration and cannot be changed,
// but we can take values from throwaway flag set and reregister into supplied flags with new default values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: how is it different than just changing default values of those two configs directly in c.Server.RegisterFlags?

It's confusing because, all our default values are in c.Server.RegisterFlags() currently. And IMHO, its bit odd to move only those two into separate function with throwaway flagset :)

I see you tried explaining it in the comment. But I think I failed to understand what that exactly means and why it matters here :( Shouldn't this be treated as normal default values?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kavirajk c.Server.RegisterFlags() is defined in the weaveworks/common library. We could make a PR into that if we think this is a better default for all cases, but I'm not sure if that's the case. I think we need to change this because of something to do specifically with memberlist. They're currently using the grpc defaults which they might prefer to keep. Either way, I still think this makes sense in the meantime to override them here until such a PR is made and merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wish there were a better way.

c.Server.RegisterFlags(throwaway)

throwaway.VisitAll(func(f *flag.Flag) {
// Ignore errors when setting new values. We have a test to verify that it works.
switch f.Name {
case "server.grpc.keepalive.min-time-between-pings":
_ = f.Value.Set("10s")

case "server.grpc.keepalive.ping-without-stream-allowed":
_ = f.Value.Set("true")
}

fs.Var(f.Value, f.Name, f.Usage)
})
}

// Clone takes advantage of pass-by-value semantics to return a distinct *Config.
// This is primarily used to parse a different flag set without mutating the original *Config.
func (c *Config) Clone() flagext.Registerer {
Expand Down
58 changes: 58 additions & 0 deletions pkg/loki/loki_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package loki

import (
"bytes"
"flag"
"io"
"strings"
"testing"
"time"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestFlagDefaults(t *testing.T) {
c := Config{}

f := flag.NewFlagSet("test", flag.PanicOnError)
c.RegisterFlags(f)

buf := bytes.Buffer{}

f.SetOutput(&buf)
f.PrintDefaults()

const delim = '\n'

minTimeChecked := false
pingWithoutStreamChecked := false
for {
line, err := buf.ReadString(delim)
if err == io.EOF {
break
}

require.NoError(t, err)

if strings.Contains(line, "-server.grpc.keepalive.min-time-between-pings") {
nextLine, err := buf.ReadString(delim)
require.NoError(t, err)
assert.Contains(t, nextLine, "(default 10s)")
minTimeChecked = true
}

if strings.Contains(line, "-server.grpc.keepalive.ping-without-stream-allowed") {
nextLine, err := buf.ReadString(delim)
require.NoError(t, err)
assert.Contains(t, nextLine, "(default true)")
pingWithoutStreamChecked = true
}
}

require.True(t, minTimeChecked)
require.True(t, pingWithoutStreamChecked)

require.Equal(t, true, c.Server.GRPCServerPingWithoutStreamAllowed)
require.Equal(t, 10*time.Second, c.Server.GRPCServerMinTimeBetweenPings)
}