Skip to content

Commit

Permalink
Merge remote-tracking branch 'grafana/master' into annotations-created
Browse files Browse the repository at this point in the history
* grafana/master: (30 commits)
  changelog: adds note about closing grafana#11278
  docs: spelling
  docs: add intro paragraph to provisioning page
  Cleanup CircleCI V2 Conversion
  changelog: notes for grafana#1271 and grafana#2740
  graph: minor fixes to y-axes alignment feature
  added save icon to save buttons
  removed trash can icon from save buttons
  Return actual user ID in UserProfileDTO
  dashboard version cleanup: more tests and refactor
  minor refactor of dashboard version cleanup
  refactor: dashboard version cleanup
  limit number of rows deleted by dashboard version cleanup
  fix dashboard version cleanup on large datasets
  Allocated to a separate alignment block. Replaced the attribute of the second axis by the attribute of the axes.
  Fixed unit test.
  Changed the way this feature was activated. And changed tolltip.
  Added validation of input parameters.
  Resolved conflict
  Corrected work for graphs created before this feature.
  ...
  • Loading branch information
ryantxu committed Mar 23, 2018
2 parents db92a96 + 09df38d commit 4ed1ec5
Show file tree
Hide file tree
Showing 14 changed files with 481 additions and 63 deletions.
File renamed without changes.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* **MSSQL**: New Microsoft SQL Server data source [#10093](https://github.com/grafana/grafana/pull/10093), [#11298](https://github.com/grafana/grafana/pull/11298), thx [@linuxchips](https://github.com/linuxchips)
* **Prometheus**: The heatmap panel now support Prometheus histograms [#10009](https://github.com/grafana/grafana/issues/10009)
* **Postgres/MySQL**: Ability to insert 0s or nulls for missing intervals [#9487](https://github.com/grafana/grafana/issues/9487), thanks [@svenklemm](https://github.com/svenklemm)
* **Graph**: Align left and right Y-axes to one level [#1271](https://github.com/grafana/grafana/issues/1271) & [#2740](https://github.com/grafana/grafana/issues/2740) thx [@ilgizar](https://github.com/ilgizar)
* **Graph**: Thresholds for Right Y axis [#7107](https://github.com/grafana/grafana/issues/7107), thx [@ilgizar](https://github.com/ilgizar)
* **Graph**: Support multiple series stacking in histogram mode [#8151](https://github.com/grafana/grafana/issues/8151), thx [@mtanda](https://github.com/mtanda)
* **Alerting**: Pausing/un alerts now updates new_state_date [#10942](https://github.com/grafana/grafana/pull/10942)
Expand All @@ -17,6 +18,7 @@
* **Cloudwatch**: Add dimension filtering to CloudWatch `dimension_values()` [#10029](https://github.com/grafana/grafana/issues/10029), thx [@willyhutw](https://github.com/willyhutw)
* **Units**: Second to HH:mm:ss formatter [#11107](https://github.com/grafana/grafana/issues/11107), thx [@gladdiologist](https://github.com/gladdiologist)
* **Singlestat**: Add color to prefix and postfix in singlestat panel [#11143](https://github.com/grafana/grafana/pull/11143), thx [@ApsOps](https://github.com/ApsOps)
* **Dashboards**: Version cleanup fails on old databases with many entries [#11278](https://github.com/grafana/grafana/issues/11278)

# 5.0.4 (unreleased)
* **Dashboard** Fixed bug where collapsed panels could not be directly linked to/renderer [#11114](https://github.com/grafana/grafana/issues/11114) & [#11086](https://github.com/grafana/grafana/issues/11086)
Expand Down
9 changes: 6 additions & 3 deletions docs/sources/administration/provisioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ weight = 8

# Provisioning Grafana

In previous versions of Grafana, you could only use the API for provisioning data sources and dashboards. But that required the service to be running before you started creating dashboards and you also needed to set up credentials for the HTTP API. In v5.0 we decided to improve this experience by adding a new active provisioning system that uses config files. This will make GitOps more natural as data sources and dashboards can be defined via files that can be version controlled. We hope to extend this system to later add support for users, orgs and alerts as well.

## Config file

Checkout the [configuration](/installation/configuration) page for more information on what you can configure in `grafana.ini`
Expand Down Expand Up @@ -77,9 +79,11 @@ Saltstack | [https://github.com/salt-formulas/salt-formula-grafana](https://gith
It's possible to manage datasources in Grafana by adding one or more yaml config files in the [`provisioning/datasources`](/installation/configuration/#provisioning) directory. Each config file can contain a list of `datasources` that will be added or updated during start up. If the datasource already exists, Grafana will update it to match the configuration file. The config file can also contain a list of datasources that should be deleted. That list is called `delete_datasources`. Grafana will delete datasources listed in `delete_datasources` before inserting/updating those in the `datasource` list.

### Running multiple Grafana instances.

If you are running multiple instances of Grafana you might run into problems if they have different versions of the `datasource.yaml` configuration file. The best way to solve this problem is to add a version number to each datasource in the configuration and increase it when you update the config. Grafana will only update datasources with the same or lower version number than specified in the config. That way, old configs cannot overwrite newer configs if they restart at the same time.

### Example datasource config file

```yaml
# config file version
apiVersion: 1
Expand Down Expand Up @@ -137,7 +141,7 @@ datasources:
| Datasource | Misc |
| ---- | ---- |
| Elasticserach | Elasticsearch uses the `database` property to configure the index for a datasource |
| Elasticsearch | Elasticsearch uses the `database` property to configure the index for a datasource |

#### Json data

Expand All @@ -146,7 +150,7 @@ Since not all datasources have the same configuration settings we only have the
| Name | Type | Datasource | Description |
| ---- | ---- | ---- | ---- |
| tlsAuth | boolean | *All* | Enable TLS authentication using client cert configured in secure json data |
| tlsAuthWithCACert | boolean | *All* | Enable TLS authtication using CA cert |
| tlsAuthWithCACert | boolean | *All* | Enable TLS authentication using CA cert |
| tlsSkipVerify | boolean | *All* | Controls whether a client verifies the server's certificate chain and host name. |
| graphiteVersion | string | Graphite | Graphite version |
| timeInterval | string | Elastic, Influxdb & Prometheus | Lowest interval/step value that should be used for this data source |
Expand All @@ -161,7 +165,6 @@ Since not all datasources have the same configuration settings we only have the
| tsdbResolution | string | OpenTsdb | Resolution |
| sslmode | string | Postgre | SSLmode. 'disable', 'require', 'verify-ca' or 'verify-full' |


#### Secure Json data

`{"authType":"keys","defaultRegion":"us-west-2","timeField":"@timestamp"}`
Expand Down
66 changes: 22 additions & 44 deletions pkg/services/sqlstore/dashboard_version.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,30 +67,39 @@ func GetDashboardVersions(query *m.GetDashboardVersionsQuery) error {
return nil
}

const MAX_VERSIONS_TO_DELETE = 100

func DeleteExpiredVersions(cmd *m.DeleteExpiredVersionsCommand) error {
return inTransaction(func(sess *DBSession) error {
versions := []DashboardVersionExp{}
versionsToKeep := setting.DashboardVersionsToKeep

if versionsToKeep < 1 {
versionsToKeep = 1
}

err := sess.Table("dashboard_version").
Select("dashboard_version.id, dashboard_version.version, dashboard_version.dashboard_id").
Where(`dashboard_id IN (
SELECT dashboard_id FROM dashboard_version
GROUP BY dashboard_id HAVING COUNT(dashboard_version.id) > ?
)`, versionsToKeep).
Desc("dashboard_version.dashboard_id", "dashboard_version.version").
Find(&versions)

// Idea of this query is finding version IDs to delete based on formula:
// min_version_to_keep = min_version + (versions_count - versions_to_keep)
// where version stats is processed for each dashboard. This guarantees that we keep at least versions_to_keep
// versions, but in some cases (when versions are sparse) this number may be more.
versionIdsToDeleteQuery := `SELECT id
FROM dashboard_version, (
SELECT dashboard_id, count(version) as count, min(version) as min
FROM dashboard_version
GROUP BY dashboard_id
) AS vtd
WHERE dashboard_version.dashboard_id=vtd.dashboard_id
AND version < vtd.min + vtd.count - ?`

var versionIdsToDelete []interface{}
err := sess.SQL(versionIdsToDeleteQuery, versionsToKeep).Find(&versionIdsToDelete)
if err != nil {
return err
}

// Keep last versionsToKeep versions and delete other
versionIdsToDelete := getVersionIDsToDelete(versions, versionsToKeep)
// Don't delete more than MAX_VERSIONS_TO_DELETE version per time
if len(versionIdsToDelete) > MAX_VERSIONS_TO_DELETE {
versionIdsToDelete = versionIdsToDelete[:MAX_VERSIONS_TO_DELETE]
}

if len(versionIdsToDelete) > 0 {
deleteExpiredSql := `DELETE FROM dashboard_version WHERE id IN (?` + strings.Repeat(",?", len(versionIdsToDelete)-1) + `)`
expiredResponse, err := sess.Exec(deleteExpiredSql, versionIdsToDelete...)
Expand All @@ -103,34 +112,3 @@ func DeleteExpiredVersions(cmd *m.DeleteExpiredVersionsCommand) error {
return nil
})
}

// Short version of DashboardVersion for getting expired versions
type DashboardVersionExp struct {
Id int64 `json:"id"`
DashboardId int64 `json:"dashboardId"`
Version int `json:"version"`
}

func getVersionIDsToDelete(versions []DashboardVersionExp, versionsToKeep int) []interface{} {
versionIds := make([]interface{}, 0)

if len(versions) == 0 {
return versionIds
}

currentDashboard := versions[0].DashboardId
count := 0
for _, v := range versions {
if v.DashboardId == currentDashboard {
count++
} else {
count = 1
currentDashboard = v.DashboardId
}
if count > versionsToKeep {
versionIds = append(versionIds, v.Id)
}
}

return versionIds
}
22 changes: 21 additions & 1 deletion pkg/services/sqlstore/dashboard_version_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,30 @@ func TestDeleteExpiredVersions(t *testing.T) {
err := DeleteExpiredVersions(&m.DeleteExpiredVersionsCommand{})
So(err, ShouldBeNil)

query := m.GetDashboardVersionsQuery{DashboardId: savedDash.Id, OrgId: 1}
query := m.GetDashboardVersionsQuery{DashboardId: savedDash.Id, OrgId: 1, Limit: versionsToWrite}
GetDashboardVersions(&query)

So(len(query.Result), ShouldEqual, versionsToWrite)
})

Convey("Don't delete more than MAX_VERSIONS_TO_DELETE per iteration", func() {
versionsToWriteBigNumber := MAX_VERSIONS_TO_DELETE + versionsToWrite
for i := 0; i < versionsToWriteBigNumber-versionsToWrite; i++ {
updateTestDashboard(savedDash, map[string]interface{}{
"tags": "different-tag",
})
}

err := DeleteExpiredVersions(&m.DeleteExpiredVersionsCommand{})
So(err, ShouldBeNil)

query := m.GetDashboardVersionsQuery{DashboardId: savedDash.Id, OrgId: 1, Limit: versionsToWriteBigNumber}
GetDashboardVersions(&query)

// Ensure we have at least versionsToKeep versions
So(len(query.Result), ShouldBeGreaterThanOrEqualTo, versionsToKeep)
// Ensure we haven't deleted more than MAX_VERSIONS_TO_DELETE rows
So(versionsToWriteBigNumber-len(query.Result), ShouldBeLessThanOrEqualTo, MAX_VERSIONS_TO_DELETE)
})
})
}
1 change: 1 addition & 0 deletions pkg/services/sqlstore/user.go
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ func GetUserProfile(query *m.GetUserProfileQuery) error {
}

query.Result = m.UserProfileDTO{
Id: user.Id,
Name: user.Name,
Email: user.Email,
Login: user.Login,
Expand Down
2 changes: 1 addition & 1 deletion public/app/containers/ManageDashboards/FolderSettings.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ export class FolderSettings extends React.Component<IContainerProps, any> {
className="btn btn-success"
disabled={!folder.folder.canSave || !folder.folder.hasChanged}
>
<i className="fa fa-trash" /> Save
<i className="fa fa-save" /> Save
</button>
<button className="btn btn-danger" onClick={this.delete.bind(this)} disabled={!folder.folder.canSave}>
<i className="fa fa-trash" /> Delete
Expand Down
3 changes: 1 addition & 2 deletions public/app/features/dashboard/partials/folder_settings.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ <h2 class="page-sub-heading">Folder Settings</h2>
</div>
<div class="gf-form-button-row">
<button type="submit" class="btn btn-success" ng-disabled="!ctrl.canSave || !ctrl.hasChanged">
<i class="fa fa-trash"></i>
Save
<i class="fa fa-save"></i>Save
</button>
<button class="btn btn-danger" ng-click="ctrl.delete($event)" ng-disabled="!ctrl.canSave">
<i class="fa fa-trash"></i>
Expand Down
154 changes: 154 additions & 0 deletions public/app/plugins/panel/graph/align_yaxes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
import _ from 'lodash';

/**
* To align two Y axes by Y level
* @param yAxes data [{min: min_y1, min: max_y1}, {min: min_y2, max: max_y2}]
* @param level Y level
*/
export function alignYLevel(yAxes, level) {
if (isNaN(level) || !checkCorrectAxis(yAxes)) {
return;
}

var [yLeft, yRight] = yAxes;
moveLevelToZero(yLeft, yRight, level);

expandStuckValues(yLeft, yRight);

// one of graphs on zero
var zero = yLeft.min === 0 || yRight.min === 0 || yLeft.max === 0 || yRight.max === 0;

var oneSide = checkOneSide(yLeft, yRight);

if (zero && oneSide) {
yLeft.min = yLeft.max > 0 ? 0 : yLeft.min;
yLeft.max = yLeft.max > 0 ? yLeft.max : 0;
yRight.min = yRight.max > 0 ? 0 : yRight.min;
yRight.max = yRight.max > 0 ? yRight.max : 0;
} else {
if (checkOppositeSides(yLeft, yRight)) {
if (yLeft.min >= 0) {
yLeft.min = -yLeft.max;
yRight.max = -yRight.min;
} else {
yLeft.max = -yLeft.min;
yRight.min = -yRight.max;
}
} else {
var rate = getRate(yLeft, yRight);

if (oneSide) {
// all graphs above the Y level
if (yLeft.min > 0) {
yLeft.min = yLeft.max / rate;
yRight.min = yRight.max / rate;
} else {
yLeft.max = yLeft.min / rate;
yRight.max = yRight.min / rate;
}
} else {
if (checkTwoCross(yLeft, yRight)) {
yLeft.min = yRight.min ? yRight.min * rate : yLeft.min;
yRight.min = yLeft.min ? yLeft.min / rate : yRight.min;
yLeft.max = yRight.max ? yRight.max * rate : yLeft.max;
yRight.max = yLeft.max ? yLeft.max / rate : yRight.max;
} else {
yLeft.min = yLeft.min > 0 ? yRight.min * rate : yLeft.min;
yRight.min = yRight.min > 0 ? yLeft.min / rate : yRight.min;
yLeft.max = yLeft.max < 0 ? yRight.max * rate : yLeft.max;
yRight.max = yRight.max < 0 ? yLeft.max / rate : yRight.max;
}
}
}
}

restoreLevelFromZero(yLeft, yRight, level);
}

function expandStuckValues(yLeft, yRight) {
// wide Y min and max using increased wideFactor
var wideFactor = 0.25;
if (yLeft.max === yLeft.min) {
yLeft.min -= wideFactor;
yLeft.max += wideFactor;
}
if (yRight.max === yRight.min) {
yRight.min -= wideFactor;
yRight.max += wideFactor;
}
}

function moveLevelToZero(yLeft, yRight, level) {
if (level !== 0) {
yLeft.min -= level;
yLeft.max -= level;
yRight.min -= level;
yRight.max -= level;
}
}

function restoreLevelFromZero(yLeft, yRight, level) {
if (level !== 0) {
yLeft.min += level;
yLeft.max += level;
yRight.min += level;
yRight.max += level;
}
}

function checkCorrectAxis(axis) {
return axis.length === 2 && checkCorrectAxes(axis[0]) && checkCorrectAxes(axis[1]);
}

function checkCorrectAxes(axes) {
return 'min' in axes && 'max' in axes;
}

function checkOneSide(yLeft, yRight) {
// on the one hand with respect to zero
return (yLeft.min >= 0 && yRight.min >= 0) || (yLeft.max <= 0 && yRight.max <= 0);
}

function checkTwoCross(yLeft, yRight) {
// both across zero
return yLeft.min <= 0 && yLeft.max >= 0 && yRight.min <= 0 && yRight.max >= 0;
}

function checkOppositeSides(yLeft, yRight) {
// on the opposite sides with respect to zero
return (yLeft.min >= 0 && yRight.max <= 0) || (yLeft.max <= 0 && yRight.min >= 0);
}

function getRate(yLeft, yRight) {
var rateLeft, rateRight, rate;
if (checkTwoCross(yLeft, yRight)) {
rateLeft = yRight.min ? yLeft.min / yRight.min : 0;
rateRight = yRight.max ? yLeft.max / yRight.max : 0;
} else {
if (checkOneSide(yLeft, yRight)) {
var absLeftMin = Math.abs(yLeft.min);
var absLeftMax = Math.abs(yLeft.max);
var absRightMin = Math.abs(yRight.min);
var absRightMax = Math.abs(yRight.max);
var upLeft = _.max([absLeftMin, absLeftMax]);
var downLeft = _.min([absLeftMin, absLeftMax]);
var upRight = _.max([absRightMin, absRightMax]);
var downRight = _.min([absRightMin, absRightMax]);

rateLeft = downLeft ? upLeft / downLeft : upLeft;
rateRight = downRight ? upRight / downRight : upRight;
} else {
if (yLeft.min > 0 || yRight.min > 0) {
rateLeft = yLeft.max / yRight.max;
rateRight = 0;
} else {
rateLeft = 0;
rateRight = yLeft.min / yRight.min;
}
}
}

rate = rateLeft > rateRight ? rateLeft : rateRight;

return rate;
}
Loading

0 comments on commit 4ed1ec5

Please sign in to comment.