alerting and error handling improvements

OP_CHANGES.md

@@ -1,63 +1,6 @@
 # OP Changes
 
-## July 2023
-
-### <a name="july2023Builds"></a> Builds
-
-1. initial repository was cloned with currently running version commit:
-
-```shell
-git clone --depth=1 --branch v9.5.2 https://github.com/grafana/grafana.git
-```
-
-2. switched to `go mod vendor` to avoid long-running builds
-
-3. following files and directories were added:
-
-- `op.Dockerfile` (modified version of original Dockerfile to build both custom grafana backend/frontend)
-- `op.mk` and `/op-develop` (files required to launch developer environment)
-
-4. files that were modified:
-
-- removed `/vendor` from `.gitignore` (to use it in Dockerfile)
-- removed `/.github` to disable github actions and dependencies check
-- added `/vendor` to `.gitattributes` (hide diffs in gitlab mrs)
-
-### <a name="july2023Code"></a> Code
-
-Added `op-pkg` package to both codebase and `op.Dockerfile` (require to be copied) with:
-
-- service and store (to mimic internal logic with custom implementations)
-- sdk (http sdk with client libraries and middlewares)
-- opstorage (opstorage client library made with sdk)
-
-#### Grafana internal codebase changes
-
-API:
-
-- `/pkg/api/http_server.go` (added authentication middlewares from `op-pkg/sdk`)
-- `/pkg/api/accesscontrol.go` (added required rights for all dashboards and folders for `Viewer` and `Editor` by default)
-- `/pkg/server/wire.go` (replaced original services requirements and stores with modified ones from `op-pkg`)
-
-Services and service stores:
-
-- `/pkg/services/datasources/service/datasource.go` (initial datasource Store implementation replacement)
-- `/pkg/services/dashboards/database/database.go` (initial dashboard Store implementation replacement)
-- `/pkg/services/folder/folderImpl/dashboard_folder_store.go` (initial dashboard Store implementation replacement)
-- `/pkg/services/secrets/manager.go` (changes to use modified version of encryption service from `op-pkg` only)
-
-Frontend:
-
-- `/pkg/api/frontendsettings.go` (override `appURL` and `appSubURL` to use dynamic url sub-paths like `localhost:3000/sub1/sub2.../dashboards`)
-- `/pkg/api/index.go` (override `appURL` and `appSubURL` to use dynamic url sub-paths like `localhost:3000/sub1/sub2.../dashboards`
-- `packages/grafana-data/src/themes/palette.ts` (add new color `lightGray`)
-- `packages/grafana-data/src/themes/createColors.ts` (override `background.canvas` color with `lightGray`)
-- `public/app/core/components/AppChrome/AppChrome.tsx` (override `searchBarHidden` with `true`, replace `NavToolbar` with actions, add css for actions, disable `NavToolbar` and `MegaMenu`)
-- `public/app/core/components/PageNew/SectionNav.tsx` (disable `SectionNavToggle`)
-- `public/app/features/dashboard/components/DashNav/DashNav.tsx` (override `canStar`, `canShare` and `isStarred` with `false`)
-- `public/app/core/components/PageNew/Page.tsx` (disable `padding` in css)
-
-### <a name="july2023Architecture"></a> Architecture
+## Architecture
 
 User being authenticated through custom auth proxy, which then adds the following headers:
 
@@ -96,7 +39,7 @@ location ~ ^/{X-REQUEST-CONTEXT}/(.*) {
                     +------------------- http://grafana.uri/{X-REQUEST-CONTEXT}/
 ```
 
-### <a name="july2023DevEnv"></a> Dev Env
+### Dev Env
 
 Quick start:
 
@@ -108,7 +51,7 @@ OPSTORAGE_BASEURL="opStorageBaseURL" \
 REQUEST_CONTEXT="{X-REQUEST-CONTEXT}" \
 USER_ROLE="Editor" \
 USER_SESSION="{X-USER-SESSION}" \
-make -f op.mk
+make op-develop -f op.mk
 ```
 
 3. Open http://localhost:8080/{X-REQUEST-CONTEXT}/ page in your browser
@@ -143,3 +86,115 @@ Used environmental variables:
 | GRAFANA_PORT  | Service and Reverse proxy config (Files `docker-compose.yml`, nginx.conf`) | Grafana local port     | 3030                           | `3030`               |
 
 *required to support `host.docker.internal` for reverse proxy on both linux and macOS
+
+## Updating Grafana from upstream
+
+1. Lookup required grafana release at: https://github.com/grafana/grafana/releases
+2. Clone required grafana release into custom directory `grafana-upstream`
+```shell
+git clone --depth=1 --branch $releaseTag https://github.com/grafana/grafana.git grafana-upstream
+```
+3. Remove .git related files from `grafana-upstream`
+```shell
+cd grafana-upstream
+rm -rf .git
+rm -f .gitignore
+```
+4. Checkout <this> grafana branch into two custom directories `grafana-update-branch-for-push` and `grafana-update-branch-for-diff`
+5. Remove all codebase-related files and directories from directory `grafana-update-branch-for-push` (everything except `.helm`, ci, and `op`-prefixed files and directories from root)
+6. Copy all `grafana-upstream` codebase-related files to `grafana-update-branch-for-push`
+```shell
+cp -a ~/Documents/grafana-upstream/. ~/Documents/grafana-update-branch-for-push/
+```
+7. List files that were previously affected inside `grafana-update-branch-for-diff` and apply corresponding changes inside: `grafana-update-branch-for-push`
+```shell
+make op-list-changes -f op.mk
+```
+```
+./packages/grafana-data/src/themes/palette.ts:20:       // OP_CHANGES.md: added new color lightGray
+./packages/grafana-data/src/themes/createColors.ts:217: // OP_CHANGES.md: added new color lightGray
+./pkg/api/index.go:73:                                  // OP_CHANGES.md: override appURL and appSubURL for frontend
+./pkg/api/frontendsettings.go:97:                       // OP_CHANGES.md: override appURL and appSubURL for frontend
+...
+```
+8. Run `go mod vendor` inside `grafana-update-branch-for-push`
+9. Launch Dev Env to check nothing is broken at the moment
+10. Push applied changes to `grafana-update-branch-for-push` and create the merge request:
+```shell
+git add -f .
+git commit -m "update version"
+git push
+```
+
+## July 2023
+
+### <a name="july2023Builds"></a> Builds
+
+1. initial repository was cloned with currently running version commit:
+
+```shell
+git clone --depth=1 --branch v9.5.2 https://github.com/grafana/grafana.git
+```
+
+2. switched to `go mod vendor` to avoid long-running builds
+
+3. following files and directories were added:
+
+- `op.Dockerfile` (modified version of original Dockerfile to build both custom grafana backend/frontend)
+- `op.mk` and `/op-develop` (files required to launch developer environment)
+
+4. files that were modified:
+
+- removed `/vendor` from `.gitignore` (to use it in Dockerfile)
+- removed `/.github` to disable github actions and dependencies check
+- added `/vendor` to `.gitattributes` (hide diffs in gitlab mrs)
+
+### <a name="july2023Code"></a> Code
+
+Added `op-pkg` package to both codebase and `op.Dockerfile` (require to be copied) with:
+
+- service and store (to mimic internal logic with custom implementations)
+- sdk (http sdk with client libraries and middlewares)
+- opstorage (opstorage client library made with sdk)
+
+#### Grafana internal codebase changes
+
+API:
+
+- `/pkg/api/http_server.go` (added authentication middlewares from `op-pkg/sdk`)
+- `/pkg/api/accesscontrol.go` (added required rights for all dashboards and folders for `Viewer` and `Editor` by default)
+- `/pkg/server/wire.go` (replaced original services requirements and stores with modified ones from `op-pkg`)
+
+Services and service stores:
+
+- `/pkg/services/datasources/service/datasource.go` (initial datasource Store implementation replacement)
+- `/pkg/services/dashboards/database/database.go` (initial dashboard Store implementation replacement)
+- `/pkg/services/folder/folderImpl/dashboard_folder_store.go` (initial dashboard Store implementation replacement)
+- `/pkg/services/secrets/manager.go` (changes to use modified version of encryption service from `op-pkg` only)
+- `/pkg/services/ngalert/api/util.go` (use op middlewares in alerting service)
+
+Frontend:
+
+- `/pkg/api/frontendsettings.go` (override `appURL` and `appSubURL` to use dynamic url sub-paths like `localhost:3000/sub1/sub2.../dashboards`)
+- `/pkg/api/index.go` (override `appURL` and `appSubURL` to use dynamic url sub-paths like `localhost:3000/sub1/sub2.../dashboards`
+- `/pkg/services/navtree/navtreeimpl/navtree.go` (disable navigation on alerting page)
+- `/packages/grafana-data/src/themes/palette.ts` (add new color `lightGray`)
+- `/packages/grafana-data/src/themes/createColors.ts` (override `background.canvas` color with `lightGray`)
+- `/public/app/core/components/AppChrome/AppChrome.tsx` (override `searchBarHidden` with `true`, replace `NavToolbar` with actions, add css for actions, disable `NavToolbar` and `MegaMenu`)
+- `/public/app/core/components/PageNew/Page.tsx` (disable `padding` in css)
+- `/public/app/features/dashboard/components/DashNav/DashNav.tsx` (override `canStar`, `canShare` and `isStarred` with `false`)
+- `/public/app/features/dashboard/components/SaveDashboard/forms/SaveDashboardAsForm.tsx` (disable `showRoot` in `FolderPicker`)
+- `/public/app/features/alerting/routes.tsx` (removed unnecessary routes: `/alerting/ng/list`, `/alerting/notifications`, `/alerting/notifications/*`, `/alerting/notification/*`, `/alerting`, `/alerting/home`, `/alerting/routes*`, `/alerting/groups/`)
+- `/public/app/features/alerting/unified/RuleList.tsx` (removed button export `url:/api/v1/provisioning/alert-rules/export`)
+- `/public/app/features/alerting/unified/components/rule-editor/AlertRuleForm.tsx` (set type: `RuleFormType.cloudAlerting`)
+- `/public/app/features/alerting/unified/components/rule-editor/query-and-alert-condition/AlertType.tsx` (set `RuleFormType.cloudAlerting` as defaultRuleType)
+- `/public/app/features/alerting/unified/components/rule-editor/rule-types/RuleTypePicker.tsx` (disable selected `GrafanaManagedRuleType`)
+- `/public/app/features/alerting/unified/components/rule-viewer/RuleViewerVisualization.tsx` (removed `view in Explore` button)
+- `/public/app/features/alerting/unified/components/rules/RuleActionsButtons.tsx` (removed `copy to clipboard` button)
+- `/public/app/features/alerting/unified/components/rules/RuleDetailsActionButtons.tsx` (removed `copy to clipboard` button)
+- `/public/app/features/alerting/unified/hooks/useAlertManagerSourceName.ts` (use first available alert manager name)
+- `/public/app/features/alerting/unified/utils/datasource.ts` (remove `grafanaAlertManagerDataSource` from available datasources)
+- `/public/app/features/alerting/unified/utils/rule-form.ts` (removed logic related to `RuleFormType.grafana`)
+- `/public/app/features/alerting/unified/Home.tsx` (removed `WelcomeCTABox`es)
+- `/public/app/features/explore/ExploreToolbar.tsx` (hide `<AppChromeUpdate actions={[shareButton}}`)
+- `/public/app/features/explore/RichHistory/RichHistoryCard.tsx` (`hide onCreateShortLink`)

op-develop/nginx/nginx.conf

@@ -1,7 +1,7 @@
 server {
     listen  0.0.0.0:8080 default;
-    location ~ ^/[0-9a-z-]+/[0-9a-z-]+/(.*) {
-        proxy_pass                 http://$GRAFANA_IP:$GRAFANA_PORT/$1;
+    location ~ ^/(?:metrics|logs|usage)/[0-9a-z-]+/[0-9a-z-]+/[0-9a-z-]+/(.*) {
+        proxy_pass                 http://$GRAFANA_IP:$GRAFANA_PORT/$1$is_args$args;
         proxy_set_header           X-WEBAUTH-USER "$USER_ROLE";
         proxy_set_header           X-WEBAUTH-ROLE "$USER_ROLE";
         proxy_set_header           X-REQUEST-CONTEXT "$REQUEST_CONTEXT";

op-pkg/opstorage/client.go

@@ -12,6 +12,8 @@ var (
 	// ErrNotFound is a custom error to make it easier to differ proxy misconfiguration (default 404 response)
 	// from missing an actual item (query by id failed)
 	ErrNotFound = errors.New("not found")
+	// ErrEmptyUserSession is a custom error to identify empty user session
+	ErrEmptyUserSession = errors.New("empty user session")
 )
 
 type Storage struct {

op-pkg/opstorage/dashboard.go

@@ -82,6 +82,10 @@ func (s *dashboardStorage) SaveDashboard(ctx context.Context, query *SaveDashboa
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	payload, err := json.Marshal(query)
 	if err != nil {
 		return nil, err
@@ -118,6 +122,10 @@ func (s *dashboardStorage) GetDashboard(ctx context.Context, query *GetDashboard
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.ID != 0 {
 		params.Set("id", strconv.FormatInt(query.ID, 10))
@@ -175,6 +183,10 @@ func (s *dashboardStorage) GetDashboardRef(ctx context.Context, query *GetDashbo
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 
 	if query.ID != 0 {
@@ -217,6 +229,10 @@ func (s *dashboardStorage) GetDashboardTags(ctx context.Context, query *GetDashb
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	params.Set("org_id", strconv.FormatInt(query.OrgID, 10))
 
@@ -256,6 +272,10 @@ func (s *dashboardStorage) FindDashboards(ctx context.Context, query *FindDashbo
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.Title != "" {
 		params.Set("title", query.Title)
@@ -313,6 +333,10 @@ func (s *dashboardStorage) GetDashboards(ctx context.Context, query *GetDashboar
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	for _, dashboardID := range query.DashboardIDs {
 		params.Add("dashboard_ids[]", strconv.FormatInt(dashboardID, 10))
@@ -351,6 +375,10 @@ func (s *dashboardStorage) GetDashboardsByPluginID(ctx context.Context, query *G
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.PluginID != "" {
 		params.Set("plugin_id", query.PluginID)
@@ -386,6 +414,10 @@ func (s *dashboardStorage) CountDashboardsInFolder(ctx context.Context, query *C
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return 0, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	params.Set("folder_id", strconv.FormatInt(query.FolderID, 10))
 	params.Set("org_id", strconv.FormatInt(query.OrgID, 10))
@@ -419,6 +451,10 @@ func (s *dashboardStorage) Count(ctx context.Context, query *CountDashboardsQuer
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return 0, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.UserID != 0 {
 		params.Set("user_id", strconv.FormatInt(query.UserID, 10))
@@ -454,6 +490,10 @@ func (s *dashboardStorage) DeleteDashboard(ctx context.Context, query *DeleteDas
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.ID != 0 {
 		params.Set("id", strconv.FormatInt(query.ID, 10))

op-pkg/opstorage/datasource.go

@@ -84,6 +84,10 @@ func (s *datasourceStorage) GetDatasource(ctx context.Context, query *GetDataSou
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.ID > 0 {
 		params.Set("id", strconv.FormatInt(query.ID, 10))
@@ -127,6 +131,10 @@ func (s *datasourceStorage) GetDefaultDatasource(ctx context.Context, query *Get
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	params.Set("org_id", strconv.FormatInt(query.OrgID, 10))
 
@@ -157,6 +165,10 @@ func (s *datasourceStorage) GetAllDatasources(ctx context.Context) ([]*Datasourc
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	data, err := s.client.Get(ctx, "datasource/getAllDatasources",
 		interceptor.WithRequestHeader("X-REQUEST-CONTEXT", requestContextData),
 		interceptor.WithRequestCookie(&http.Cookie{Name: "user_session", Value: userSessionData}),
@@ -185,6 +197,10 @@ func (s *datasourceStorage) GetDatasources(ctx context.Context, query *GetDataso
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.Limit > 0 {
 		params.Set("limit", strconv.Itoa(query.Limit))
@@ -220,6 +236,10 @@ func (s *datasourceStorage) GetDatasourcesByType(ctx context.Context, query *Get
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return nil, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.Type != "" {
 		params.Set("type", query.Type)
@@ -255,6 +275,10 @@ func (s *datasourceStorage) Count(ctx context.Context, query *CountDatasourceQue
 		userSessionData    = middleware.GetUserSessionData(ctx)
 	)
 
+	if userSessionData == "" {
+		return 0, ErrEmptyUserSession
+	}
+
 	params := url.Values{}
 	if query.UserID != 0 {
 		params.Set("user_id", strconv.FormatInt(query.UserID, 10))