Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn frontend Tripperware into a Middleware. #10688

Merged
merged 106 commits into from
Oct 23, 2023

Conversation

jeschkies
Copy link
Contributor

@jeschkies jeschkies commented Sep 22, 2023

What this PR does / why we need it:
Currently, a request to Loki's frontend API goes through these conversions:

        http.Request 
            ↓ limitedRoundTripper
        queryrangebase.Request
            ↓ queryrangebase.Middlware
        … 
            ↓
        queryrangebase.Request
            ↓ limitedRoundTripper  
        http.Request
            ↓ grpcRoundTripperAdapter
        httpgrpc
            ↓ grpcRoundTripperAdapter
        http.Response
            ↓ limitedRoundTripper
        queryrangebase.Response
            ↓ limitedRoundtripper
        http.Response

Since httgrpc and queryrangebase.Request are Protobufs there's no good reason to encode and decode them to HTTP responses/requests. Furthermore, the encoding to HTTP makes it harder for us to encode query plans.

Thus the conversions are changed to the following:

        http.Request 
            ↓
        queryrangebase.Request
            ↓ queryrangebase.Middlware
        … 
            ↓
        queryrangebase.Request
            ↓
        httpgrpc
            ↓
        queryrangebase.Response
            ↓
        http.Response

In order to achieve this the http.RoundTripper is pushed to the outside. Only the serialization layer from http.Request to queryrangebase.Request and http.Response to queryrangebase.Response will be an http.RoundTripper. Everything else is either a queryrangebase.Handler or queryrangebase.Middleware.

This change has been split up

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • CHANGELOG.md updated
    • If the change is worth mentioning in the release notes, add add-to-release-notes label
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR

@jeschkies jeschkies changed the title Trun frontend Tripperware into a Middleware. Turn frontend Tripperware into a Middleware. Sep 22, 2023
slim-bean added a commit that referenced this pull request Nov 7, 2023
**What this PR does / why we need it**:

Fallout of a recent change to reduce complexity in our
tripperware/middleware in #10688 lead to non-retryable errors being
retried by the retry middleware in the frontend.

This PR is a solution to that problem, I'm not sure it's the best
solution, but it's a solution....

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Signed-off-by: Edward Welch <edward.welch@grafana.com>
grafanabot pushed a commit that referenced this pull request Nov 7, 2023
**What this PR does / why we need it**:

Fallout of a recent change to reduce complexity in our
tripperware/middleware in #10688 lead to non-retryable errors being
retried by the retry middleware in the frontend.

This PR is a solution to that problem, I'm not sure it's the best
solution, but it's a solution....

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Signed-off-by: Edward Welch <edward.welch@grafana.com>
(cherry picked from commit 9418fb7)
slim-bean added a commit that referenced this pull request Nov 7, 2023
Backport 9418fb7 from #11164

---

**What this PR does / why we need it**:

Fallout of a recent change to reduce complexity in our
tripperware/middleware in #10688 lead to non-retryable errors being
retried by the retry middleware in the frontend.

This PR is a solution to that problem, I'm not sure it's the best
solution, but it's a solution....

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Co-authored-by: Ed Welch <edward.welch@grafana.com>
jeschkies added a commit that referenced this pull request Nov 10, 2023
**What this PR does / why we need it**:
This will be a follow up to #10688
finally switching over to protos. Hang on tight!

Everything is behind the feature flag `-frontend.encoding=protobuf` which will disable the transcoding to HTTP and HTTPgRPC.


**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
trevorwhitney pushed a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
trevorwhitney pushed a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.

(cherry picked from commit 8d34f85)
trevorwhitney pushed a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.

(cherry picked from commit 8d34f85)
grafanabot pushed a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.

(cherry picked from commit 8d34f85)
trevorwhitney added a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.

(cherry picked from commit 8d34f85)

---------

Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>
trevorwhitney pushed a commit that referenced this pull request Nov 20, 2023
Backport 8d34f85 from #11251

---

The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
grafanabot pushed a commit that referenced this pull request Nov 20, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.

(cherry picked from commit 8d34f85)
trevorwhitney pushed a commit that referenced this pull request Nov 20, 2023
Backport 8d34f85 from #11251

---

**What this PR does / why we need it**:
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
socheatsok78 pushed a commit to socheatsok78/loki that referenced this pull request Nov 21, 2023
The changes in grafana#10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
jeschkies added a commit that referenced this pull request Nov 21, 2023
The changes in #10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
…a#10801)

**What this PR does / why we need it**:
This is an attempt to unify our API models and use the definitions in
`logproto` which is helpful for grafana#10688.

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
**What this PR does / why we need it**:
This is one step to support pure protobuf encoding without `httpgrpc`.
It is only on the scheduler and is fully backwards compatible.

**Which issue(s) this PR fixes**:
This is a sub change of grafana#10688

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)

---------

Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>
Co-authored-by: Kaviraj <kavirajkanagaraj@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
**What this PR does / why we need it**:
Currently, a request to Loki's frontend API goes through these
conversions:
```
        http.Request 
            ↓ limitedRoundTripper
        queryrangebase.Request
            ↓ queryrangebase.Middlware
        … 
            ↓
        queryrangebase.Request
            ↓ limitedRoundTripper  
        http.Request
            ↓ grpcRoundTripperAdapter
        httpgrpc
            ↓ grpcRoundTripperAdapter
        http.Response
            ↓ limitedRoundTripper
        queryrangebase.Response
            ↓ limitedRoundtripper
        http.Response
```

Since `httgrpc` and `queryrangebase.Request` are Protobufs there's no
good reason to encode and decode them to HTTP responses/requests.
Furthermore, the encoding to HTTP makes it harder for us to encode query
plans.

Thus the conversions are changed to the following:
```
        http.Request 
            ↓
        queryrangebase.Request
            ↓ queryrangebase.Middlware
        … 
            ↓
        queryrangebase.Request
            ↓
        httpgrpc
            ↓
        queryrangebase.Response
            ↓
        http.Response
```

In order to achieve this the `http.RoundTripper` is pushed to the
outside. Only the serialization layer from `http.Request` to
`queryrangebase.Request` and `http.Response` to
`queryrangebase.Response` will be an `http.RoundTripper`. Everything
else is either a `queryrangebase.Handler` or
`queryrangebase.Middleware`.

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
**What this PR does / why we need it**:
The `TestMetricsTripperware` became flaky after grafana#10688. The race
condition in the splitter has been there before but the change exposed
it. When an error occurs, e.g. when query would be too large, the
splitter would return. However, there was a small time window when the
loop could still be fed a new request. grafana#10688 just amplified this effect
because it removed the serialization of requests and responses.

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory. <!--
TODO(salvacorts): Add example PR -->

---------

Co-authored-by: Danny Kopping <dannykopping@gmail.com>
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
**What this PR does / why we need it**:

Fallout of a recent change to reduce complexity in our
tripperware/middleware in grafana#10688 lead to non-retryable errors being
retried by the retry middleware in the frontend.

This PR is a solution to that problem, I'm not sure it's the best
solution, but it's a solution....

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

Signed-off-by: Edward Welch <edward.welch@grafana.com>
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
**What this PR does / why we need it**:
This will be a follow up to grafana#10688
finally switching over to protos. Hang on tight!

Everything is behind the feature flag `-frontend.encoding=protobuf` which will disable the transcoding to HTTP and HTTPgRPC.


**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
The changes in grafana#10688 did not
propage the trace ID from the context. `Frontend.RoundTripGRPC` would
inject the trace ID into the request. That's not done in `Frontend.Do`.
This changes extends the `codec.EncodeRequest` to inject the trace ID
there. This is more inline with other metadata.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants