Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converts usage collection README to .mdx #92982

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/developer/plugin-list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ which also contains the timelion APIs and backend, look at the vis_type_timelion
|This plugins contains helpers to redirect legacy URLs. It can be used to forward old URLs to their new counterparts.


|{kib-repo}blob/{branch}/src/plugins/usage_collection/README.md[usageCollection]
|{kib-repo}blob/{branch}/src/plugins/usage_collection/README.mdx[usageCollection]
|The Usage Collection Service defines a set of APIs for other plugins to report the usage of their features. At the same time, it provides necessary the APIs for other services (i.e.: telemetry, monitoring, ...) to consume that usage data.


Expand Down
2 changes: 1 addition & 1 deletion src/core/CONVENTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ export class Plugin {

#### Usage Collection

For creating and registering a Usage Collector. Collectors should be defined in a separate directory `server/collectors/`. You can read more about usage collectors on `src/plugins/usage_collection/README.md`.
For creating and registering a Usage Collector. Collectors should be defined in a separate directory `server/collectors/`. You can read more about usage collectors on `src/plugins/usage_collection/README.mdx`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the links to the README


```ts
// server/collectors/register.ts
Expand Down
2 changes: 1 addition & 1 deletion src/core/server/saved_objects/service/lib/repository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1640,7 +1640,7 @@ export class SavedObjectsRepository {
*
* When using incrementCounter for collecting usage data, you need to ensure
* that usage collection happens on a best-effort basis and doesn't
* negatively affect your plugin or users. See https://github.com/elastic/kibana/blob/master/src/plugins/usage_collection/README.md#tracking-interactions-with-incrementcounter)
* negatively affect your plugin or users. See https://github.com/elastic/kibana/blob/master/src/plugins/usage_collection/README.mdx#tracking-interactions-with-incrementcounter)
*
* @example
* ```ts
Expand Down
2 changes: 1 addition & 1 deletion src/plugins/telemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Telemetry allows Kibana features to have usage tracked in the wild. The general
2. Sending a payload of usage data up to Elastic's telemetry cluster.
3. Viewing usage data in the Kibana instance of the telemetry cluster (Viewing).

This plugin is responsible for sending usage data to the telemetry cluster. For collecting usage data, use the [`usageCollection` plugin](../usage_collection/README.md)
This plugin is responsible for sending usage data to the telemetry cluster. For collecting usage data, use the [`usageCollection` plugin](../usage_collection/README.mdx)

## Telemetry Plugin public API

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
---
id: kibUsageCollectionPlugin
slug: /kibana-dev-docs/services/usage-collection-plugin
title: Usage collection service
summary: The Usage Collection Service defines a set of APIs for other plugins to report the usage of their features.
date: 2021-02-24
tags: ['kibana','dev', 'contributor', 'api docs']
---

# Kibana Usage Collection Service

The Usage Collection Service defines a set of APIs for other plugins to report the usage of their features. At the same time, it provides necessary the APIs for other services (i.e.: telemetry, monitoring, ...) to consume that usage data.
Expand Down Expand Up @@ -43,7 +52,7 @@ import { Plugin, CoreStart } from '../../../core/public';

export class MyPlugin implements Plugin {
public start(
core: CoreStart,
core: CoreStart,
{ usageCollection }: { usageCollection?: UsageCollectionSetup }
) {
// Call the following method as many times as you want to report an increase in the count for this event
Expand Down Expand Up @@ -109,9 +118,9 @@ In many cases, plugins need to report the custom usage of a feature. In this cas

##### Creating and Registering Usage Collector

1. To create the usage collector, the API `usageCollection.makeUsageCollector` expects:
- `type`: the key under which to nest all the usage reported by the `fetch` method.
- `schema`: field to define the expected output of the `fetch` method.
1. To create the usage collector, the API `usageCollection.makeUsageCollector` expects:
- `type`: the key under which to nest all the usage reported by the `fetch` method.
- `schema`: field to define the expected output of the `fetch` method.
- `isReady`: async method (that returns true or false) for letting the usage collection consumers know if they need to wait for any asynchronous action (initialization of clients or other services) before calling the `fetch` method.
- `fetch`: async method for returning the usage collector's data.

Expand Down Expand Up @@ -161,7 +170,7 @@ In many cases, plugins need to report the custom usage of a feature. In this cas
},
},
isReady: () => isCollectorFetchReady, // Method to return `true`/`false` or Promise(`true`/`false`) to confirm if the collector is ready for the `fetch` method to be called.

fetch: async (collectorFetchContext: CollectorFetchContext) => {

// query ES or saved objects and get some data
Expand All @@ -181,13 +190,13 @@ In many cases, plugins need to report the custom usage of a feature. In this cas
}
```

Some background:
Some background:

- `MY_USAGE_TYPE` can be any string. It usually matches the plugin name. As a safety mechanism, we double check there are no duplicates at the moment of registering the collector.

- `isReady` (added in v7.2.0 and v6.8.4) is a way for a usage collector to announce that some async process must finish first before it can return data in the `fetch` method (e.g. a client needs to ne initialized, or the task manager needs to run a task first). If any collector reports that it is not ready when we call its `fetch` method, we reset a flag to try again and, after a set amount of time, collect data from those collectors that are ready and skip any that are not. This means that if a collector returns `true` for `isReady` and it actually isn't ready to return data, there won't be telemetry data from that collector in that telemetry report (usually once per day). You should consider what it means if your collector doesn't return data in the first few documents when Kibana starts or, if we should wait for any other reason (e.g. the task manager needs to run your task first). If you need to tell telemetry collection to wait, you should implement this function with custom logic. If your `fetch` method can run without the need of any previous dependencies, then you can return true for `isReady` as shown in the example below.

- The `fetch` method needs to support multiple contexts in which it is called. For example, when a user requests the example of what we collect in the **Kibana>Advanced Settings>Usage data** section, the clients provided in the context of the function (`CollectorFetchContext`) are scoped to that user's privileges. The reason is to avoid exposing via telemetry any data that user should not have access to (i.e.: if the user does not have access to certain indices, they shouldn't be allowed to see the number of documents that exists in it). In this case, the `fetch` method receives the clients `esClient` and `soClient` scoped to the user who performed the HTTP API request. Alternatively, when requesting the usage data to be reported to the Remote Telemetry Service, the clients are scoped to the internal Kibana user (`kibana_system`). Please, mind it might have lower-level access than the default super-admin `elastic` test user.
- The `fetch` method needs to support multiple contexts in which it is called. For example, when a user requests the example of what we collect in the **Kibana>Advanced Settings>Usage data** section, the clients provided in the context of the function (`CollectorFetchContext`) are scoped to that user's privileges. The reason is to avoid exposing via telemetry any data that user should not have access to (i.e.: if the user does not have access to certain indices, they shouldn't be allowed to see the number of documents that exists in it). In this case, the `fetch` method receives the clients `esClient` and `soClient` scoped to the user who performed the HTTP API request. Alternatively, when requesting the usage data to be reported to the Remote Telemetry Service, the clients are scoped to the internal Kibana user (`kibana_system`). Please, mind it might have lower-level access than the default super-admin `elastic` test user.
In some scenarios, your collector might need to maintain its own client. An example of that is the `monitoring` plugin, that maintains a connection to the Remote Monitoring Cluster to push its monitoring data. If that's the case, your plugin can opt-in to receive the additional `kibanaRequest` parameter by adding `extendFetchContext.kibanaRequest: true` to the collector's config: it will be appended to the context of the `fetch` method only if the request needs to be scoped to a user other than Kibana Internal, so beware that your collector will need to work for both scenarios (especially for the scenario when `kibanaRequest` is missing).

Note: there will be many cases where you won't need to use the `esClient` or `soClient` function that gets passed in to your `fetch` method at all. Your feature might have an accumulating value in server memory, or read something from the OS.
Expand Down Expand Up @@ -241,15 +250,15 @@ export const myCollector = makeUsageCollector<Usage>({
},
some_array: {
type: 'array',
items: { type: 'keyword' }
items: { type: 'keyword' }
},
some_array_of_obj: {
type: 'array',
items: {
items: {
total: {
type: 'long',
},
},
},
},
},
});
Expand All @@ -263,7 +272,7 @@ use your plugin or specific features. For tracking user interactions the
method which can be used to increment one or more counter fields in a
document. Examples of interactions include tracking:
- the number of API calls
- the number of times users installed and uninstalled the sample datasets
- the number of times users installed and uninstalled the sample datasets

When using `incrementCounter` for collecting usage data, you need to ensure
that usage collection happens on a best-effort basis and doesn't
Expand Down Expand Up @@ -341,7 +350,7 @@ export class DashboardPlugin implements Plugin {
counter
], {refresh: false})
.catch((e) => (this.isDevEnvironement ? this.logger.error(e) : e));

return response.ok();
}
);
Expand Down
2 changes: 1 addition & 1 deletion x-pack/plugins/apm/dev_docs/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Telemetry data is uploaded to the "xpack-phone-home" indices.

Information that can be derived from a cluster's APM indices is queried and sent
to the telemetry cluster using the
[Usage Collection plugin](../../../../src/plugins/usage_collection/README.md).
[Usage Collection plugin](../../../../src/plugins/usage_collection/README.mdx).

During the APM server-side plugin's setup phase a
[Saved Object](https://www.elastic.co/guide/en/kibana/master/managing-saved-objects.html)
Expand Down