Merge remote-tracking branch 'grafana/master' into jwt

* grafana/master:
  Usage Stats: Update known datasource plugins (grafana#17787)
  Docs: Adds section on Querying Logs for Elasticsearch (grafana#17730)
  Docs: Adds section on Querying Logs for InfluxDB (grafana#17726)
  Devenv: makes the grafana users default for saml. (grafana#17782)
  Explore: Displays only one Time column as configured in TimeZone settings (grafana#17775)
  Markdown: Replace rendering library (grafana#17686)
  ApiKeys: Fix check for UTC timezone (grafana#17776)
  Prometheus: Minor style fix (grafana#17773)
  Docs: fixed notifications table
  Auth: Allow expiration of API keys (grafana#17678)
  17278 prometheus step align utc (grafana#17477)
  Docs: Update release guide  (grafana#17759)

conf/defaults.ini

@@ -287,6 +287,9 @@ signout_redirect_url =
 # This setting is ignored if multiple OAuth providers are configured.
 oauth_auto_login = false
 
+# limit of api_key seconds to live before expiration
+api_key_max_seconds_to_live = -1
+
 #################################### Anonymous Auth ######################
 [auth.anonymous]
 # enable anonymous access

devenv/docker/blocks/saml/users.php

@@ -3,7 +3,7 @@
     'admin' => array(
         'core:AdminPassword',
     ),
-    'grafana-userpass' => array(
+    'example-userpass' => array(
         'exampleauth:UserPass',
         'saml-admin:grafana' => array(
             'groups' => array('admins'),

docs/sources/administration/provisioning.md

@@ -149,6 +149,8 @@ Since not all datasources have the same configuration settings we only have the
 | esVersion | number | Elasticsearch | Elasticsearch version as a number (2/5/56/60/70) |
 | timeField | string | Elasticsearch | Which field that should be used as timestamp |
 | interval | string | Elasticsearch | Index date time format. nil(No Pattern), 'Hourly', 'Daily', 'Weekly', 'Monthly' or 'Yearly' |
+| logMessageField | string | Elasticsearch | Which field should be used as the log message |
+| logLevelField | string | Elasticsearch | Which field should be used to indicate the priority of the log message |
 | authType | string | Cloudwatch | Auth provider. keys/credentials/arn |
 | assumeRoleArn | string | Cloudwatch | ARN of Assume Role |
 | defaultRegion | string | Cloudwatch | AWS region |

docs/sources/alerting/notifications.md

@@ -167,8 +167,8 @@ Notifications can be sent by setting up an incoming webhook in Google Hangouts c
 
 ### All supported notifiers
 
-Name | Type | Supports images |Support alert rule tags
------|------------ | ------
+Name | Type | Supports images | Support alert rule tags
+-----|------|---------------- | -----------------------
 DingDing | `dingding` | yes, external only | no
 Discord | `discord` | yes | no
 Email | `email` | yes | no

docs/sources/auth/overview.md

@@ -63,6 +63,9 @@ login_maximum_lifetime_days = 30
 
 # How often should auth tokens be rotated for authenticated users when being active. The default is each 10 minutes.
 token_rotation_interval_minutes = 10
+
+# The maximum lifetime (seconds) an api key can be used. If it is set all the api keys should have limited lifetime that is lower than this value.
+api_key_max_seconds_to_live = -1
 ```
 
 ### Anonymous authentication

docs/sources/features/datasources/elasticsearch.md

@@ -79,6 +79,18 @@ Identifier | Description
 `s`   | second
 `ms`  | millisecond
 
+### Logs (BETA)
+
+> Only available in Grafana v6.3+.
+
+There are two parameters, `Message field name` and `Level field name`, that can optionally be configured from the data source settings page that determine
+which fields will be used for log messages and log levels when visualizing logs in [Explore](/features/explore).
+
+For example, if you're using a default setup of Filebeat for shipping logs to Elasticsearch the following configuration should work:
+
+- **Message field name:**  message
+- **Level field name:** fields.level
+
 ## Metric Query editor
 
 ![Elasticsearch Query Editor](/img/docs/elasticsearch/query_editor.png)
@@ -162,6 +174,28 @@ Time | The name of the time field, needs to be date field.
 Text | Event description field.
 Tags | Optional field name to use for event tags (can be an array or a CSV string).
 
+## Querying Logs (BETA)
+
+> Only available in Grafana v6.3+.
+
+Querying and displaying log data from Elasticsearch is available via [Explore](/features/explore).
+
+![](/img/docs/v63/elasticsearch_explore_logs.png)
+
+Select the Elasticsearch data source, change to Logs using the Metrics/Logs switcher, and then optionally enter a lucene query into the query field to filter the log messages.
+
+Finally, press the `Enter` key or the `Run Query` button to display your logs.
+
+### Log Queries
+
+Once the result is returned, the log panel shows a list of log rows and a bar chart where the x-axis shows the time and the y-axis shows the frequency/count.
+
+Note that the fields used for log message and level is based on an [optional datasource configuration](#logs-beta).
+
+### Filter Log Messages
+
+Optionally enter a lucene query into the query field to filter the log messages. For example, using a default Filebeat setup you should be able to use `fields.level:error` to only show error log messages.
+
 ## Configure the Datasource with Provisioning
 
 It's now possible to configure datasources using config files with Grafana's provisioning system. You can read more about how it works and all the settings you can set for datasources on the [provisioning docs page](/administration/provisioning/#datasources)
@@ -181,3 +215,22 @@ datasources:
       interval: Daily
       timeField: "@timestamp"
 ```
+
+or, for logs:
+
+```yaml
+apiVersion: 1
+
+datasources:
+  - name: elasticsearch-v7-filebeat
+    type: elasticsearch
+    access: proxy
+    database: "[filebeat-]YYYY.MM.DD"
+    url: http://localhost:9200
+    jsonData:
+      interval: Daily
+      timeField: "@timestamp"
+      esVersion: 70
+      logMessageField: message
+      logLevelField: fields.level
+```

docs/sources/features/datasources/influxdb.md

@@ -117,6 +117,26 @@ You can switch to raw query mode by clicking hamburger icon and then `Switch edi
 You can remove the group by time by clicking on the `time` part and then the `x` icon. You can
 change the option `Format As` to `Table` if you want to show raw data in the `Table` panel.
 
+## Querying Logs (BETA)
+
+> Only available in Grafana v6.3+.
+
+Querying and displaying log data from InfluxDB is available via [Explore](/features/explore).
+
+![](/img/docs/v63/influxdb_explore_logs.png)
+
+Select the InfluxDB data source, change to Logs using the Metrics/Logs switcher,
+and then use the `Measurements/Fields` button to display your logs.
+
+### Log Queries
+
+The Logs Explorer (the `Measurements/Fields` button) next to the query field shows a list of measurements and fields. Choose the desired measurement that contains your log data and then choose which field Explore should use to display the log message.
+
+Once the result is returned, the log panel shows a list of log rows and a bar chart where the x-axis shows the time and the y-axis shows the frequency/count.
+
+### Filter search
+
+To add a filter click the plus icon to the right of the `Measurements/Fields` button or a condition. You can remove tag filters by clicking on the first select and choosing `--remove filter--`.
 
 ## Templating
 

docs/sources/http_api/auth.md

@@ -82,7 +82,8 @@ Content-Type: application/json
   {
     "id": 1,
     "name": "TestAdmin",
-    "role": "Admin"
+    "role": "Admin",
+    "expiration": "2019-06-26T10:52:03+03:00"
   }
 ]
 ```
@@ -101,14 +102,21 @@ Authorization: Bearer eyJrIjoiT0tTcG1pUlY2RnVKZTFVaDFsNFZXdE9ZWmNrMkZYbk
 
 {
   "name": "mykey",
-  "role": "Admin"
+  "role": "Admin",
+  "secondsToLive": 86400
 }
 ```
 
 JSON Body schema:
 
 - **name** – The key name
 - **role** – Sets the access level/Grafana Role for the key. Can be one of the following values: `Viewer`, `Editor` or `Admin`.
+- **secondsToLive** – Sets the key expiration in seconds. It is optional. If it is a positive number an expiration date for the key is set. If it is null, zero or is omitted completely (unless `api_key_max_seconds_to_live` configuration option is set) the key will never expire.
+
+Error statuses:
+
+- **400** – `api_key_max_seconds_to_live` is set but no `secondsToLive` is specified or `secondsToLive` is greater than this value.
+- **500** – The key was unable to be stored in the database.
 
 **Example Response**:
 

package.json

@@ -42,7 +42,6 @@
     "@types/react-transition-group": "2.0.16",
     "@types/react-virtualized": "9.18.12",
     "@types/react-window": "1.7.0",
-    "@types/remarkable": "1.7.4",
     "angular-mocks": "1.6.6",
     "autoprefixer": "9.5.0",
     "axios": "0.19.0",
@@ -192,6 +191,7 @@
     "@types/angular-route": "1.7.0",
     "@types/d3-scale-chromatic": "1.3.1",
     "@types/enzyme-adapter-react-16": "1.0.5",
+    "@types/marked": "0.6.5",
     "@types/react-redux": "^7.0.8",
     "@types/redux-logger": "3.0.7",
     "@types/reselect": "2.2.0",
@@ -214,6 +214,7 @@
     "immutable": "3.8.2",
     "jquery": "3.4.1",
     "lodash": "4.17.11",
+    "marked": "0.6.2",
     "moment": "2.24.0",
     "mousetrap": "1.6.3",
     "mousetrap-global-bind": "1.1.0",
@@ -238,7 +239,6 @@
     "redux-logger": "3.0.6",
     "redux-observable": "1.1.0",
     "redux-thunk": "2.3.0",
-    "remarkable": "1.7.1",
     "reselect": "4.0.0",
     "rst2html": "github:thoward/rst2html#990cb89",
     "rxjs": "6.4.0",

packages/grafana-data/src/utils/index.ts

@@ -1 +1,2 @@
 export * from './string';
+export * from './markdown';

packages/grafana-data/src/utils/markdown.ts

@@ -0,0 +1,20 @@
+import marked, { MarkedOptions } from 'marked';
+
+const defaultMarkedOptions: MarkedOptions = {
+  renderer: new marked.Renderer(),
+  pedantic: false,
+  gfm: true,
+  tables: true,
+  sanitize: true,
+  smartLists: true,
+  smartypants: false,
+  xhtml: false,
+};
+
+export function setMarkdownOptions(optionsOverride?: MarkedOptions) {
+  marked.setOptions({ ...defaultMarkedOptions, ...optionsOverride });
+}
+
+export function renderMarkdown(str: string): string {
+  return marked(str);
+}

packages/grafana-ui/README.md

@@ -17,31 +17,35 @@ See [package source](https://github.com/grafana/grafana/tree/master/packages/gra
 For development purposes we suggest using `yarn link` that will create symlink to @grafana/ui lib. To do so navigate to `packages/grafana-ui` and run `yarn link`. Then, navigate to your project and run `yarn link @grafana/ui` to use the linked version of the lib. To unlink follow the same procedure, but use `yarn unlink` instead.
 
 ## Building @grafana/ui
-To build @grafana/ui run `npm run gui:build` script *from Grafana repository root*. The build will be created in `packages/grafana-ui/dist` directory. Following steps from [Development](#development) you can test built package.
+
+To build @grafana/ui run `npm run gui:build` script _from Grafana repository root_. The build will be created in `packages/grafana-ui/dist` directory. Following steps from [Development](#development) you can test built package.
 
 ## Releasing new version
-To release new version run `npm run gui:release` script *from Grafana repository root*. The script will prepare the distribution package as well as prompt you to bump library version and publish it to the NPM registry.
+
+To release new version run `npm run gui:release` script _from Grafana repository root_. This has to be done on the master branch. The script will prepare the distribution package as well as prompt you to bump library version and publish it to the NPM registry. When the new package is published, create a PR with the bumped version in package.json.
 
 ### Automatic version bump
+
 When running `npm run gui:release` package.json file will be automatically updated. Also, package.json file will be commited and pushed to upstream branch.
 
 ### Manual version bump
-To use `package.json` defined version run `npm run gui:release --usePackageJsonVersion` *from Grafana repository root*.
+
+Manually update the version in `package.json` and then run `npm run gui:release --usePackageJsonVersion` _from Grafana repository root_.
 
 ### Preparing release package without publishing to NPM registry
+
 For testing purposes there is `npm run gui:releasePrepare` task that prepares distribution package without publishing it to the NPM registry.
 
 ### V1 release process overview
+
 1. Package is compiled with TSC. Typings are created in `/dist` directory, and the compiled js lands in `/compiled` dir
 2. Rollup creates a CommonJS package based on compiled sources, and outputs it to `/dist` directory
 3. Readme, changelog and index.js files are moved to `/dist` directory
 4. Package version is bumped in both `@grafana/ui` package dir and in dist directory.
 5. Version commit is created and pushed to master branch
-5. Package is published to npm
-
+6. Package is published to npm
 
 ## Versioning
+
 To limit the confusion related to @grafana/ui and Grafana versioning we decided to keep the major version in sync between those two.
 This means, that first version of @grafana/ui is taged with 6.0.0-alpha.0 to keep version in sync with Grafana 6.0 release.
-
-

packages/grafana-ui/rollup.config.ts

@@ -27,6 +27,8 @@ const buildCjsPackage = ({ env }) => {
     plugins: [
       commonjs({
         include: /node_modules/,
+        // When 'rollup-plugin-commonjs' fails to properly convert the CommonJS modules to ES6 one has to manually name the exports
+        // https://github.com/rollup/rollup-plugin-commonjs#custom-named-exports
         namedExports: {
           '../../node_modules/lodash/lodash.js': [
             'flatten',

packages/grafana-ui/src/types/logs.ts

@@ -46,6 +46,7 @@ export interface LogRowModel {
   timeFromNow: string;
   timeEpochMs: number;
   timeLocal: string;
+  timeUtc: string;
   uniqueLabels?: Labels;
 }
 

packages/grafana-ui/src/utils/moment_wrapper.ts

@@ -46,6 +46,7 @@ export interface DateTimeDuration {
   hours: () => number;
   minutes: () => number;
   seconds: () => number;
+  asSeconds: () => number;
 }
 
 export interface DateTime extends Object {
@@ -68,6 +69,7 @@ export interface DateTime extends Object {
   valueOf: () => number;
   unix: () => number;
   utc: () => DateTime;
+  utcOffset: () => number;
   hour?: () => number;
 }