Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ruler/loki rule validator #2589

Merged
merged 4 commits into from
Sep 1, 2020
Merged

Ruler/loki rule validator #2589

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7e972f9
revendors cortex
owen-d Sep 1, 2020
ef80737
loki multitenantmanager
owen-d Sep 1, 2020
7f4b719
vendoring compat
owen-d Sep 1, 2020
e8953e1
go mod
owen-d Sep 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ require (
github.com/cespare/xxhash/v2 v2.1.1
github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448 // indirect
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
github.com/cortexproject/cortex v1.3.1-0.20200824064113-84c06a83898e
github.com/cortexproject/cortex v1.3.1-0.20200901164804-97d13c1ef16f
github.com/davecgh/go-spew v1.1.1
github.com/docker/docker v17.12.0-ce-rc1.0.20200706150819-a40b877fbb9e+incompatible
github.com/docker/go-metrics v0.0.0-20181218153428-b84716841b82 // indirect
Expand Down Expand Up @@ -80,3 +80,7 @@ replace github.com/gocql/gocql => github.com/grafana/gocql v0.0.0-20200605141915

// Same as Cortex, we can't upgrade to grpc 1.30.0 until go.etcd.io/etcd will support it.
replace google.golang.org/grpc => google.golang.org/grpc v1.29.1

// Same as Cortex
// Using a 3rd-party branch for custom dialer - see https://github.com/bradfitz/gomemcache/pull/86
replace github.com/bradfitz/gomemcache => github.com/themihai/gomemcache v0.0.0-20180902122335-24332e2d58ab
9 changes: 5 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,6 @@ github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 h1:DDGfHa7BWjL4Yn
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
github.com/bmizerany/pat v0.0.0-20170815010413-6226ea591a40/go.mod h1:8rLXio+WjiTceGBHIoTvn60HIbs7Hm7bcHjyrSqYB9c=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b h1:L/QXpzIa3pOvUGt1D1lA5KjYhPBAN/3iWdP7xeFS9F0=
github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA=
github.com/c-bata/go-prompt v0.2.2/go.mod h1:VzqtzE2ksDBcdln8G7mk2RX9QyGjH+OVqOCSiVIqS34=
github.com/c2h5oh/datasize v0.0.0-20200112174442-28bbd4740fee h1:BnPxIde0gjtTnc9Er7cxvBk8DHLWhEux0SxayC8dP6I=
github.com/c2h5oh/datasize v0.0.0-20200112174442-28bbd4740fee/go.mod h1:S/7n9copUssQ56c7aAgHqftWO4LTf4xY6CGWt8Bc+3M=
Expand Down Expand Up @@ -237,8 +235,8 @@ github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f h1:lBNOc5arjvs8E5mO2tbp
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cortexproject/cortex v0.6.1-0.20200228110116-92ab6cbe0995/go.mod h1:3Xa3DjJxtpXqxcMGdk850lcIRb81M0fyY1MQ6udY134=
github.com/cortexproject/cortex v1.2.1-0.20200805064754-d8edc95e2c91/go.mod h1:PVPxNLrxKH+yc8asaJOxuz7TiRmMizFfnSMOnRzM6oM=
github.com/cortexproject/cortex v1.3.1-0.20200824064113-84c06a83898e h1:f9Si1KNKw1ulAUnb++STgNSoRca+KABemwPtIq2HcF4=
github.com/cortexproject/cortex v1.3.1-0.20200824064113-84c06a83898e/go.mod h1:w0J/iHsF4Hh2KxmmpuYIBo/I9t8dxKztThqr+683p7M=
github.com/cortexproject/cortex v1.3.1-0.20200901164804-97d13c1ef16f h1:g+MHBeXc4V6JRVzAYzvZUJFfxsCXHmuRpz5M10mvl30=
github.com/cortexproject/cortex v1.3.1-0.20200901164804-97d13c1ef16f/go.mod h1:ub8BpRZrRa02BOM8NJTnI2YklxW/mGhEkJDrhsDfcfg=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/cznic/b v0.0.0-20180115125044-35e9bbe41f07/go.mod h1:URriBxXwVq5ijiJ12C7iIZqlA69nTlI+LgI6/pwftG8=
Expand Down Expand Up @@ -1073,6 +1071,7 @@ github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIK
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/soheilhy/cmux v0.1.4 h1:0HKaf1o97UwFjHH9o5XsHUOF+tqmdA7KEzXLpiyaw0E=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/sony/gobreaker v0.4.1 h1:oMnRNZXX5j85zso6xCPRNPtmAycat+WcoKbklScLDgQ=
github.com/sony/gobreaker v0.4.1/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY=
github.com/soundcloud/go-runit v0.0.0-20150630195641-06ad41a06c4a/go.mod h1:LeFCbQYJ3KJlPs/FvPz2dy1tkpxyeNESVyCNNzRXFR0=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72 h1:qLC7fQah7D6K1B0ujays3HV9gkFtllcxhzImRR7ArPQ=
Expand Down Expand Up @@ -1107,6 +1106,8 @@ github.com/thanos-io/thanos v0.13.1-0.20200731083140-69b87607decf h1:yq9nWz5Iv6e
github.com/thanos-io/thanos v0.13.1-0.20200731083140-69b87607decf/go.mod h1:G8caR6G7pSDreRDvFm9wFuyjEBztmr8Ag3kBYpa/fEc=
github.com/thanos-io/thanos v0.13.1-0.20200807203500-9b578afb4763 h1:c84P3YUu8bxLWE2csCSK4XJNi5FxcC+HL4WDNDEbTwA=
github.com/thanos-io/thanos v0.13.1-0.20200807203500-9b578afb4763/go.mod h1:KyW0a93tsh7v4hXAwo2CVAIRYuZT1Kkf4e04gisQjAg=
github.com/themihai/gomemcache v0.0.0-20180902122335-24332e2d58ab h1:7ZR3hmisBWw77ZpO1/o86g+JV3VKlk3d48jopJxzTjU=
github.com/themihai/gomemcache v0.0.0-20180902122335-24332e2d58ab/go.mod h1:eheTFp954zcWZXCU8d0AT76ftsQOTo4DTqkN/h3k1MY=
github.com/tidwall/pretty v0.0.0-20180105212114-65a9db5fad51/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/tidwall/pretty v1.0.0 h1:HsD+QiTn7sK6flMKIvNmpqz1qrpP3Ps6jOKIKMooyg4=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
Expand Down
9 changes: 9 additions & 0 deletions pkg/distributor/distributor_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,3 +307,12 @@ func (r mockRing) IngesterCount() int {
func (r mockRing) Subring(key uint32, n int) ring.ReadRing {
return r
}

func (r mockRing) HasInstance(instanceID string) bool {
for _, ing := range r.ingesters {
if ing.Addr != instanceID {
return true
}
}
return false
}
9 changes: 9 additions & 0 deletions pkg/querier/querier_mock_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,15 @@ func (r *readRingMock) Subring(key uint32, n int) ring.ReadRing {
return r
}

func (r *readRingMock) HasInstance(instanceID string) bool {
for _, ing := range r.replicationSet.Ingesters {
if ing.Addr != instanceID {
return true
}
}
return false
}

func mockReadRingWithOneActiveIngester() *readRingMock {
return newReadRingMock([]ring.IngesterDesc{
{Addr: "test", Timestamp: time.Now().UnixNano(), State: ring.ACTIVE, Tokens: []uint32{1, 2, 3}},
Expand Down
6 changes: 6 additions & 0 deletions pkg/querier/queryrange/codec.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ func (r *LokiRequest) LogToSpan(sp opentracing.Span) {
)
}

func (*LokiRequest) GetCachingOptions() (res queryrange.CachingOptions) { return }

func (r *LokiSeriesRequest) GetEnd() int64 {
return r.EndTs.UnixNano() / (int64(time.Millisecond) / int64(time.Nanosecond))
}
Expand Down Expand Up @@ -108,6 +110,8 @@ func (r *LokiSeriesRequest) LogToSpan(sp opentracing.Span) {
)
}

func (*LokiSeriesRequest) GetCachingOptions() (res queryrange.CachingOptions) { return }

func (r *LokiLabelNamesRequest) GetEnd() int64 {
return r.EndTs.UnixNano() / (int64(time.Millisecond) / int64(time.Nanosecond))
}
Expand Down Expand Up @@ -143,6 +147,8 @@ func (r *LokiLabelNamesRequest) LogToSpan(sp opentracing.Span) {
)
}

func (*LokiLabelNamesRequest) GetCachingOptions() (res queryrange.CachingOptions) { return }

func (codec) DecodeRequest(_ context.Context, r *http.Request) (queryrange.Request, error) {
if err := r.ParseForm(); err != nil {
return nil, httpgrpc.Errorf(http.StatusBadRequest, err.Error())
Expand Down
3 changes: 3 additions & 0 deletions pkg/querier/queryrange/roundtrip.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,9 @@ func NewMetricTripperware(
codec,
extractor,
nil,
func(r queryrange.Request) bool {
return !r.GetCachingOptions().Disabled
},
registerer,
)
if err != nil {
Expand Down
21 changes: 18 additions & 3 deletions pkg/ruler/manager/compat.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,21 @@ func engineQueryFunc(engine *logql.Engine, delay time.Duration) rules.QueryFunc

}

// MultiTenantManagerAdapter will wrap a MultiTenantManager which validates loki rules
func MultiTenantManagerAdapter(mgr ruler.MultiTenantManager) *MultiTenantManager {
return &MultiTenantManager{mgr}
}

// MultiTenantManager wraps a cortex MultiTenantManager but validates loki rules
type MultiTenantManager struct {
ruler.MultiTenantManager
}

// ValidateRuleGroup validates a rulegroup
func (m *MultiTenantManager) ValidateRuleGroup(grp rulefmt.RuleGroup) []error {
return validateGroups(grp)
}

func MemstoreTenantManager(
cfg ruler.Config,
engine *logql.Engine,
Expand Down Expand Up @@ -148,13 +163,13 @@ func (GroupLoader) parseRules(content []byte) (*rulefmt.RuleGroups, []error) {
return nil, errs
}

return &groups, validateGroup(&groups)
return &groups, validateGroups(groups.Groups...)
}

func validateGroup(grps *rulefmt.RuleGroups) (errs []error) {
func validateGroups(grps ...rulefmt.RuleGroup) (errs []error) {
set := map[string]struct{}{}

for i, g := range grps.Groups {
for i, g := range grps {
if g.Name == "" {
errs = append(errs, errors.Errorf("group %d: Groupname must not be empty", i))
}
Expand Down
4 changes: 2 additions & 2 deletions pkg/ruler/ruler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ type Config struct {

func NewRuler(cfg Config, engine *logql.Engine, reg prometheus.Registerer, logger log.Logger, ruleStore cRules.RuleStore) (*ruler.Ruler, error) {

manager, err := ruler.NewDefaultMultiTenantManager(
mgr, err := ruler.NewDefaultMultiTenantManager(
cfg.Config,
manager.MemstoreTenantManager(
cfg.Config,
Expand All @@ -30,7 +30,7 @@ func NewRuler(cfg Config, engine *logql.Engine, reg prometheus.Registerer, logge
}
return ruler.NewRuler(
cfg.Config,
manager,
manager.MultiTenantManagerAdapter(mgr),
reg,
logger,
ruleStore,
Expand Down
39 changes: 6 additions & 33 deletions vendor/github.com/bradfitz/gomemcache/memcache/memcache.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 7 additions & 9 deletions vendor/github.com/cortexproject/cortex/pkg/alertmanager/alertmanager.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions vendor/github.com/cortexproject/cortex/pkg/alertmanager/alertmanager_metrics.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 44 additions & 4 deletions vendor/github.com/cortexproject/cortex/pkg/alertmanager/multitenant.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading