Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loki: Remove alpha feature toggle lokiDataframeApi #65918

Merged
merged 1 commit into from
Apr 13, 2023

Conversation

gabor
Copy link
Contributor

@gabor gabor commented Apr 4, 2023

(fixes #65783)

after some investigation it seems the only useful way to use the lokiLive is to use it together with lokiDataframeApi,
so we are removing lokiDataframeApi, and assume it is always true.

this is all behind the lokiLive feature-flag, so will not affect normal grafana usage.

@gabor gabor requested a review from svennergr April 4, 2023 12:47
@gabor gabor requested review from a team and chri2547 as code owners April 4, 2023 12:47
@gabor gabor requested review from tskarhed, ashharrison90, andresmgot, papagian, zserge and mildwonkey and removed request for a team April 4, 2023 12:47
@gabor gabor added this to the 10.0.0 milestone Apr 4, 2023
@gabor gabor added no-backport Skip backport of PR no-changelog Skip including change in changelog/release notes and removed type/docs labels Apr 4, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Apr 4, 2023

Backend code coverage report for PR #65918

Plugin Main PR Difference
loki 58.4% 57.6% -.8%

@github-actions
Copy link
Contributor

github-actions bot commented Apr 4, 2023

Frontend code coverage report for PR #65918
No changes

@github-advanced-security
Copy link

You have successfully added a new CodeQL configuration .github/workflows/pr-codeql-analysis-javascript.yml:analyze. As part of the setup process, we have scanned this repository and found no existing alerts. In the future, you will see all code scanning alerts on the repository Security tab.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 4, 2023

⚠️   Possible breaking changes

(Open the links below in a new tab to go to the correct steps)

grafana-data has possible breaking changes (more info)

Console output
Read our guideline

@grafanabot grafanabot added the levitate breaking change A label indicating a breaking change and assigned by Levitate. label Apr 4, 2023
Copy link
Collaborator

@chri2547 chri2547 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs lgtm! Thank you for the contribution!

Copy link
Contributor

@svennergr svennergr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
❤️

@ryantxu
Copy link
Member

ryantxu commented Apr 4, 2023

@slim-bean are you still using this to debug our clusters? If yes, it seems worthwhile to keep it around -- and also still feels like a good direction to explore: have loki produce the format we use in the UI rather than a complex process in between.

@gabor gabor changed the title loki: remove experimental feature-flag Loki: remove alpha feature toggle lokiDataframeApi Apr 4, 2023
@gabor gabor added add to changelog and removed no-changelog Skip including change in changelog/release notes labels Apr 4, 2023
@gabor gabor changed the title Loki: remove alpha feature toggle lokiDataframeApi Loki: Remove alpha feature toggle lokiDataframeApi Apr 4, 2023
@@ -13,8 +13,6 @@ import (
"github.com/gorilla/websocket"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana-plugin-sdk-go/data"

"github.com/grafana/grafana/pkg/services/featuremgmt"
)

func (s *Service) SubscribeStream(_ context.Context, req *backend.SubscribeStreamRequest) (*backend.SubscribeStreamResponse, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this ONLY called when lokiLive feature toggle is enabled? If so this is fine, but I thought we were trying to do both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @ryantxu ... i'll try to summarize the situation here to be sure there are no misunderstandings:
(note: this functionality has nothing to do with the "tail" functionality in explore-mode)

  1. the lokiLive feature flag adds the mode=streaming option into the loki query editor. when that mode is chosen, the request comes here, to loki/streaming.go
  2. in loki/streaming.go there are two ways to communicate with Loki: the "stable" /loki/api/v1/tail endpoint, and the experimental /loki/api/v2alpha/tail endpoint. the system chooses between them based on the lokiDataframeApi feature-flag.

i talked with @slim-bean about how to move forward with these experimental feature-flags, and got the info that the combination of lokiLive=true & lokiDataframeApi=false combination does not make sense (meaning, we stream, but go to the old endpoint). based on this info, i removed the feature-flag lokiDataframeApi, and everywhere i assumed it is set to true. this is the PR we are looking at.

is this acceptable?

(in general, we can always add code back, i'm just trying to move somehow forward with these experimental feature-flags. if you have a different idea, i'm fine with discussing it 👍 )

@@ -84,15 +82,9 @@ func (s *Service) RunStream(ctx context.Context, req *backend.RunStreamRequest,
params := url.Values{}
params.Add("query", query.Expr)

lokiDataframeApi := s.features.IsEnabled(featuremgmt.FlagLokiDataframeApi)
Copy link
Member

@ryantxu ryantxu Apr 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this just check the lokiLive flag instead? so the stream endpoint still works?

IIRC, this got split into two so we could explore using the direct format in other places as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @ryantxu , this PR is basically:
"replace lokiDataFrameApi feature-flag with true, and remove the feature-flag".

currently it's not possible to use the direct format (i assume you mean the loki-emits-a-dataframe mode) anywhere except the mode=streaming option, and this PR is not regressing on that aspect... but maybe i misunderstood something?

@gabor gabor requested a review from ryantxu April 5, 2023 07:12
Copy link
Contributor

@slim-bean slim-bean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With my concern about the normal Tail operation addressed, this change looks good to me 👍

@gabor gabor merged commit b928fce into main Apr 13, 2023
32 of 33 checks passed
@gabor gabor deleted the gabor/loki-live-simplify branch April 13, 2023 13:22
alexmobo pushed a commit that referenced this pull request Apr 14, 2023
loki: remove experimental feature-flag
@zerok zerok modified the milestones: 10.0.0, 10.0.0-preview May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
add to changelog area/backend area/frontend datasource/Loki levitate breaking change A label indicating a breaking change and assigned by Levitate. no-backport Skip backport of PR type/docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature toggle removal: lokiDataframeApi
7 participants