-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' into move_popularize_field_to_discover
- Loading branch information
Showing
318 changed files
with
9,058 additions
and
2,720 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
311 changes: 311 additions & 0 deletions
311
docs/developer/architecture/development-plugin-saved-objects.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,311 @@ | ||
| [[development-plugin-saved-objects]] | ||
| == Using Saved Objects | ||
|
|
||
| Saved Objects allow {kib} plugins to use {es} like a primary | ||
| database. Think of it as an Object Document Mapper for {es}. Once a | ||
| plugin has registered one or more Saved Object types, the Saved Objects client | ||
| can be used to query or perform create, read, update and delete operations on | ||
| each type. | ||
|
|
||
| By using Saved Objects your plugin can take advantage of the following | ||
| features: | ||
|
|
||
| * Migrations can evolve your document's schema by transforming documents and | ||
| ensuring that the field mappings on the index are always up to date. | ||
| * a <<saved-objects-api,HTTP API>> is automatically exposed for each type (unless | ||
| `hidden=true` is specified). | ||
| * a Saved Objects client that can be used from both the server and the browser. | ||
| * Users can import or export Saved Objects using the Saved Objects management | ||
| UI or the Saved Objects import/export API. | ||
| * By declaring `references`, an object's entire reference graph will be | ||
| exported. This makes it easy for users to export e.g. a `dashboard` object and | ||
| have all the `visualization` objects required to display the dashboard | ||
| included in the export. | ||
| * When the X-Pack security and spaces plugins are enabled these transparently | ||
| provide RBAC access control and the ability to organize Saved Objects into | ||
| spaces. | ||
|
|
||
| This document contains developer guidelines and best-practices for plugins | ||
| wanting to use Saved Objects. | ||
|
|
||
| === Registering a Saved Object type | ||
| Saved object type definitions should be defined in their own `my_plugin/server/saved_objects` directory. | ||
|
|
||
| The folder should contain a file per type, named after the snake_case name of the type, and an `index.ts` file exporting all the types. | ||
|
|
||
| .src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts | ||
| [source,typescript] | ||
| ---- | ||
| import { SavedObjectsType } from 'src/core/server'; | ||
| export const dashboardVisualization: SavedObjectsType = { | ||
| name: 'dashboard_visualization', // <1> | ||
| hidden: false, | ||
| namespaceType: 'single', | ||
| mappings: { | ||
| dynamic: false, | ||
| properties: { | ||
| description: { | ||
| type: 'text', | ||
| }, | ||
| hits: { | ||
| type: 'integer', | ||
| }, | ||
| }, | ||
| }, | ||
| migrations: { | ||
| '1.0.0': migratedashboardVisualizationToV1, | ||
| '2.0.0': migratedashboardVisualizationToV2, | ||
| }, | ||
| }; | ||
| ---- | ||
| <1> Since the name of a Saved Object type forms part of the url path for the | ||
| public Saved Objects HTTP API, these should follow our API URL path convention | ||
| and always be written as snake case. | ||
|
|
||
| .src/plugins/my_plugin/server/saved_objects/index.ts | ||
| [source,typescript] | ||
| ---- | ||
| export { dashboardVisualization } from './dashboard_visualization'; | ||
| export { dashboard } from './dashboard'; | ||
| ---- | ||
|
|
||
| .src/plugins/my_plugin/server/plugin.ts | ||
| [source,typescript] | ||
| ---- | ||
| import { dashboard, dashboardVisualization } from './saved_objects'; | ||
| export class MyPlugin implements Plugin { | ||
| setup({ savedObjects }) { | ||
| savedObjects.registerType(dashboard); | ||
| savedObjects.registerType(dashboardVisualization); | ||
| } | ||
| } | ||
| ---- | ||
|
|
||
| === Mappings | ||
| Each Saved Object type can define it's own {es} field mappings. | ||
| Because multiple Saved Object types can share the same index, mappings defined | ||
| by a type will be nested under a top-level field that matches the type name. | ||
|
|
||
| For example, the mappings defined by the `dashboard_visualization` Saved | ||
| Object type: | ||
|
|
||
| .src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts | ||
| [source,typescript] | ||
| ---- | ||
| import { SavedObjectsType } from 'src/core/server'; | ||
| export const dashboardVisualization: SavedObjectsType = { | ||
| name: 'dashboard_visualization', | ||
| ... | ||
| mappings: { | ||
| properties: { | ||
| dynamic: false, | ||
| description: { | ||
| type: 'text', | ||
| }, | ||
| hits: { | ||
| type: 'integer', | ||
| }, | ||
| }, | ||
| }, | ||
| migrations: { ... }, | ||
| }; | ||
| ---- | ||
|
|
||
| Will result in the following mappings being applied to the `.kibana` index: | ||
| [source,json] | ||
| ---- | ||
| { | ||
| "mappings": { | ||
| "dynamic": "strict", | ||
| "properties": { | ||
| ... | ||
| "dashboard_vizualization": { | ||
| "dynamic": false, | ||
| "properties": { | ||
| "description": { | ||
| "type": "text", | ||
| }, | ||
| "hits": { | ||
| "type": "integer", | ||
| }, | ||
| }, | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ---- | ||
|
|
||
| Do not use field mappings like you would use data types for the columns of a | ||
| SQL database. Instead, field mappings are analogous to a SQL index. Only | ||
| specify field mappings for the fields you wish to search on or query. By | ||
| specifying `dynamic: false` in any level of your mappings, {es} will | ||
| accept and store any other fields even if they are not specified in your mappings. | ||
|
|
||
| Since {es} has a default limit of 1000 fields per index, plugins | ||
| should carefully consider the fields they add to the mappings. Similarly, | ||
| Saved Object types should never use `dynamic: true` as this can cause an | ||
| arbitrary amount of fields to be added to the `.kibana` index. | ||
|
|
||
| === References | ||
| When a Saved Object declares `references` to other Saved Objects, the | ||
| Saved Objects Export API will automatically export the target object with all | ||
| of it's references. This makes it easy for users to export the entire | ||
| reference graph of an object. | ||
|
|
||
| If a Saved Object can't be used on it's own, that is, it needs other objects | ||
| to exist for a feature to function correctly, that Saved Object should declare | ||
| references to all the objects it requires. For example, a `dashboard` | ||
| object might have panels for several `visualization` objects. When these | ||
| `visualization` objects don't exist, the dashboard cannot be rendered | ||
| correctly. The `dashboard` object should declare references to all it's | ||
| visualizations. | ||
|
|
||
| However, `visualization` objects can continue to be rendered or embedded into | ||
| other dashboards even if the `dashboard` it was originally embedded into | ||
| doesn't exist. As a result, `visualization` objects should not declare | ||
| references to `dashboard` objects. | ||
|
|
||
| For each referenced object, an `id`, `type` and `name` are added to the | ||
| `references` array: | ||
|
|
||
| [source, typescript] | ||
| ---- | ||
| router.get( | ||
| { path: '/some-path', validate: false }, | ||
| async (context, req, res) => { | ||
| const object = await context.core.savedObjects.client.create( | ||
| 'dashboard', | ||
| { | ||
| title: 'my dashboard', | ||
| panels: [ | ||
| { visualization: 'vis1' }, // <1> | ||
| ], | ||
| indexPattern: 'indexPattern1' | ||
| }, | ||
| { references: [ | ||
| { id: '...', type: 'visualization', name: 'vis1' }, | ||
| { id: '...', type: 'index_pattern', name: 'indexPattern1' }, | ||
| ] | ||
| } | ||
| ) | ||
| ... | ||
| } | ||
| ); | ||
| ---- | ||
| <1> Note how `dashboard.panels[0].visualization` stores the `name` property of | ||
| the reference (not the `id` directly) to be able to uniquely identify this | ||
| reference. This guarantees that the id the reference points to always remains | ||
| up to date. If a visualization `id` was directly stored in | ||
| `dashboard.panels[0].visualization` there is a risk that this `id` gets | ||
| updated without updating the reference in the references array. | ||
|
|
||
| ==== Writing Migrations | ||
|
|
||
| Saved Objects support schema changes between Kibana versions, which we call | ||
| migrations. Migrations are applied when a Kibana installation is upgraded from | ||
| one version to the next, when exports are imported via the Saved Objects | ||
| Management UI, or when a new object is created via the HTTP API. | ||
|
|
||
| Each Saved Object type may define migrations for its schema. Migrations are | ||
| specified by the Kibana version number, receive an input document, and must | ||
| return the fully migrated document to be persisted to Elasticsearch. | ||
|
|
||
| Let's say we want to define two migrations: | ||
| - In version 1.1.0, we want to drop the `subtitle` field and append it to the | ||
| title | ||
| - In version 1.4.0, we want to add a new `id` field to every panel with a newly | ||
| generated UUID. | ||
|
|
||
| First, the current `mappings` should always reflect the latest or "target" | ||
| schema. Next, we should define a migration function for each step in the schema | ||
| evolution: | ||
|
|
||
| src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts | ||
| [source,typescript] | ||
| ---- | ||
| import { SavedObjectsType, SavedObjectMigrationFn } from 'src/core/server'; | ||
| import uuid from 'uuid'; | ||
| interface DashboardVisualizationPre110 { | ||
| title: string; | ||
| subtitle: string; | ||
| panels: Array<{}>; | ||
| } | ||
| interface DashboardVisualization110 { | ||
| title: string; | ||
| panels: Array<{}>; | ||
| } | ||
| interface DashboardVisualization140 { | ||
| title: string; | ||
| panels: Array<{ id: string }>; | ||
| } | ||
| const migrateDashboardVisualization110: SavedObjectMigrationFn< | ||
| DashboardVisualizationPre110, // <1> | ||
| DashboardVisualization110 | ||
| > = (doc) => { | ||
| const { subtitle, ...attributesWithoutSubtitle } = doc.attributes; | ||
| return { | ||
| ...doc, // <2> | ||
| attributes: { | ||
| ...attributesWithoutSubtitle, | ||
| title: `${doc.attributes.title} - ${doc.attributes.subtitle}`, | ||
| }, | ||
| }; | ||
| }; | ||
| const migrateDashboardVisualization140: SavedObjectMigrationFn< | ||
| DashboardVisualization110, | ||
| DashboardVisualization140 | ||
| > = (doc) => { | ||
| const outPanels = doc.attributes.panels?.map((panel) => { | ||
| return { ...panel, id: uuid.v4() }; | ||
| }); | ||
| return { | ||
| ...doc, | ||
| attributes: { | ||
| ...doc.attributes, | ||
| panels: outPanels, | ||
| }, | ||
| }; | ||
| }; | ||
| export const dashboardVisualization: SavedObjectsType = { | ||
| name: 'dashboard_visualization', // <1> | ||
| /** ... */ | ||
| migrations: { | ||
| // Takes a pre 1.1.0 doc, and converts it to 1.1.0 | ||
| '1.1.0': migrateDashboardVisualization110, | ||
| // Takes a 1.1.0 doc, and converts it to 1.4.0 | ||
| '1.4.0': migrateDashboardVisualization140, // <3> | ||
| }, | ||
| }; | ||
| ---- | ||
| <1> It is useful to define an interface for each version of the schema. This | ||
| allows TypeScript to ensure that you are properly handling the input and output | ||
| types correctly as the schema evolves. | ||
| <2> Returning a shallow copy is necessary to avoid type errors when using | ||
| different types for the input and output shape. | ||
| <3> Migrations do not have to be defined for every version. The version number | ||
| of a migration must always be the earliest Kibana version in which this | ||
| migration was released. So if you are creating a migration which will be | ||
| part of the v7.10.0 release, but will also be backported and released as | ||
| v7.9.3, the migration version should be: 7.9.3. | ||
|
|
||
| Migrations should be written defensively, an exception in a migration function | ||
| will prevent a Kibana upgrade from succeeding and will cause downtime for our | ||
| users. Having said that, if a document is encountered that is not in the | ||
| expected shape, migrations are encouraged to throw an exception to abort the | ||
| upgrade. In most scenarios, it is better to fail an upgrade than to silently | ||
| ignore a corrupt document which can cause unexpected behaviour at some future | ||
| point in time. | ||
|
|
||
| It is critical that you have extensive tests to ensure that migrations behave | ||
| as expected with all possible input documents. Given how simple it is to test | ||
| all the branch conditions in a migration function and the high impact of a bug | ||
| in this code, there's really no reason not to aim for 100% test code coverage. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 changes: 0 additions & 15 deletions
15
...ta/public/kibana-plugin-plugins-data-public.indexpattern.istimebasedwildcard.md
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.