Allow routes to define some payload config values (elastic#50783)

* Allow routes to define some payload config values

* Documentation typo

* Move hapi `payload` config under `body` + additional validations

* Update API docs

* Amend explanation in API docs

* Add stream and buffer types to @kbn/config-schema

* Fixes based on PR feedback:
- Add 'patch' and 'options' to valid RouteMethod
- Add tests for all the new flags
- Allow `stream` and `buffer` schema in the body validations (findings from tests)

* API documentation update

* Fix type definitions

* Fix the NITs in the PR comments + better typing inheritance

* API docs update

* Fix APM-legacy wrapper's types

* Fix KibanaRequest.from type exposure of hapi in API docs

* Move RouterRoute interface back to private + Expose some public docs

* Update @kbn/config-schema docs

docs/development/core/server/kibana-plugin-server.basepath.get.md

@@ -9,5 +9,5 @@ returns `basePath` value, specific for an incoming request.
 <b>Signature:</b>
 
 ```typescript
-get: (request: KibanaRequest<unknown, unknown, unknown> | LegacyRequest) => string;
+get: (request: KibanaRequest<unknown, unknown, unknown, any> | LegacyRequest) => string;
 ```

docs/development/core/server/kibana-plugin-server.basepath.md

@@ -16,11 +16,11 @@ export declare class BasePath
 
 |  Property | Modifiers | Type | Description |
 |  --- | --- | --- | --- |
-|  [get](./kibana-plugin-server.basepath.get.md) |  | <code>(request: KibanaRequest&lt;unknown, unknown, unknown&gt; &#124; LegacyRequest) =&gt; string</code> | returns <code>basePath</code> value, specific for an incoming request. |
+|  [get](./kibana-plugin-server.basepath.get.md) |  | <code>(request: KibanaRequest&lt;unknown, unknown, unknown, any&gt; &#124; LegacyRequest) =&gt; string</code> | returns <code>basePath</code> value, specific for an incoming request. |
 |  [prepend](./kibana-plugin-server.basepath.prepend.md) |  | <code>(path: string) =&gt; string</code> | Prepends <code>path</code> with the basePath. |
 |  [remove](./kibana-plugin-server.basepath.remove.md) |  | <code>(path: string) =&gt; string</code> | Removes the prepended basePath from the <code>path</code>. |
 |  [serverBasePath](./kibana-plugin-server.basepath.serverbasepath.md) |  | <code>string</code> | returns the server's basePath<!-- -->See [BasePath.get](./kibana-plugin-server.basepath.get.md) for getting the basePath value for a specific request |
-|  [set](./kibana-plugin-server.basepath.set.md) |  | <code>(request: KibanaRequest&lt;unknown, unknown, unknown&gt; &#124; LegacyRequest, requestSpecificBasePath: string) =&gt; void</code> | sets <code>basePath</code> value, specific for an incoming request. |
+|  [set](./kibana-plugin-server.basepath.set.md) |  | <code>(request: KibanaRequest&lt;unknown, unknown, unknown, any&gt; &#124; LegacyRequest, requestSpecificBasePath: string) =&gt; void</code> | sets <code>basePath</code> value, specific for an incoming request. |
 
 ## Remarks
 

docs/development/core/server/kibana-plugin-server.basepath.set.md

@@ -9,5 +9,5 @@ sets `basePath` value, specific for an incoming request.
 <b>Signature:</b>
 
 ```typescript
-set: (request: KibanaRequest<unknown, unknown, unknown> | LegacyRequest, requestSpecificBasePath: string) => void;
+set: (request: KibanaRequest<unknown, unknown, unknown, any> | LegacyRequest, requestSpecificBasePath: string) => void;
 ```

docs/development/core/server/kibana-plugin-server.irouter.delete.md

@@ -9,5 +9,5 @@ Register a route handler for `DELETE` request.
 <b>Signature:</b>
 
 ```typescript
-delete: RouteRegistrar;
+delete: RouteRegistrar<'delete'>;
 ```

docs/development/core/server/kibana-plugin-server.irouter.get.md

@@ -9,5 +9,5 @@ Register a route handler for `GET` request.
 <b>Signature:</b>
 
 ```typescript
-get: RouteRegistrar;
+get: RouteRegistrar<'get'>;
 ```

docs/development/core/server/kibana-plugin-server.irouter.md

@@ -16,10 +16,11 @@ export interface IRouter
 
 |  Property | Type | Description |
 |  --- | --- | --- |
-|  [delete](./kibana-plugin-server.irouter.delete.md) | <code>RouteRegistrar</code> | Register a route handler for <code>DELETE</code> request. |
-|  [get](./kibana-plugin-server.irouter.get.md) | <code>RouteRegistrar</code> | Register a route handler for <code>GET</code> request. |
+|  [delete](./kibana-plugin-server.irouter.delete.md) | <code>RouteRegistrar&lt;'delete'&gt;</code> | Register a route handler for <code>DELETE</code> request. |
+|  [get](./kibana-plugin-server.irouter.get.md) | <code>RouteRegistrar&lt;'get'&gt;</code> | Register a route handler for <code>GET</code> request. |
 |  [handleLegacyErrors](./kibana-plugin-server.irouter.handlelegacyerrors.md) | <code>&lt;P extends ObjectType, Q extends ObjectType, B extends ObjectType&gt;(handler: RequestHandler&lt;P, Q, B&gt;) =&gt; RequestHandler&lt;P, Q, B&gt;</code> | Wrap a router handler to catch and converts legacy boom errors to proper custom errors. |
-|  [post](./kibana-plugin-server.irouter.post.md) | <code>RouteRegistrar</code> | Register a route handler for <code>POST</code> request. |
-|  [put](./kibana-plugin-server.irouter.put.md) | <code>RouteRegistrar</code> | Register a route handler for <code>PUT</code> request. |
+|  [patch](./kibana-plugin-server.irouter.patch.md) | <code>RouteRegistrar&lt;'patch'&gt;</code> | Register a route handler for <code>PATCH</code> request. |
+|  [post](./kibana-plugin-server.irouter.post.md) | <code>RouteRegistrar&lt;'post'&gt;</code> | Register a route handler for <code>POST</code> request. |
+|  [put](./kibana-plugin-server.irouter.put.md) | <code>RouteRegistrar&lt;'put'&gt;</code> | Register a route handler for <code>PUT</code> request. |
 |  [routerPath](./kibana-plugin-server.irouter.routerpath.md) | <code>string</code> | Resulted path |
 

docs/development/core/server/kibana-plugin-server.irouter.patch.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [IRouter](./kibana-plugin-server.irouter.md) &gt; [patch](./kibana-plugin-server.irouter.patch.md)
+
+## IRouter.patch property
+
+Register a route handler for `PATCH` request.
+
+<b>Signature:</b>
+
+```typescript
+patch: RouteRegistrar<'patch'>;
+```

docs/development/core/server/kibana-plugin-server.irouter.post.md

@@ -9,5 +9,5 @@ Register a route handler for `POST` request.
 <b>Signature:</b>
 
 ```typescript
-post: RouteRegistrar;
+post: RouteRegistrar<'post'>;
 ```

docs/development/core/server/kibana-plugin-server.irouter.put.md

@@ -9,5 +9,5 @@ Register a route handler for `PUT` request.
 <b>Signature:</b>
 
 ```typescript
-put: RouteRegistrar;
+put: RouteRegistrar<'put'>;
 ```

docs/development/core/server/kibana-plugin-server.kibanarequest.md

@@ -9,7 +9,7 @@ Kibana specific abstraction for an incoming request.
 <b>Signature:</b>
 
 ```typescript
-export declare class KibanaRequest<Params = unknown, Query = unknown, Body = unknown> 
+export declare class KibanaRequest<Params = unknown, Query = unknown, Body = unknown, Method extends RouteMethod = any> 
 ```
 
 ## Constructors
@@ -26,7 +26,7 @@ export declare class KibanaRequest<Params = unknown, Query = unknown, Body = unk
 |  [headers](./kibana-plugin-server.kibanarequest.headers.md) |  | <code>Headers</code> | Readonly copy of incoming request headers. |
 |  [params](./kibana-plugin-server.kibanarequest.params.md) |  | <code>Params</code> |  |
 |  [query](./kibana-plugin-server.kibanarequest.query.md) |  | <code>Query</code> |  |
-|  [route](./kibana-plugin-server.kibanarequest.route.md) |  | <code>RecursiveReadonly&lt;KibanaRequestRoute&gt;</code> | matched route details |
+|  [route](./kibana-plugin-server.kibanarequest.route.md) |  | <code>RecursiveReadonly&lt;KibanaRequestRoute&lt;Method&gt;&gt;</code> | matched route details |
 |  [socket](./kibana-plugin-server.kibanarequest.socket.md) |  | <code>IKibanaSocket</code> |  |
 |  [url](./kibana-plugin-server.kibanarequest.url.md) |  | <code>Url</code> | a WHATWG URL standard object. |
 

docs/development/core/server/kibana-plugin-server.kibanarequest.route.md

@@ -9,5 +9,5 @@ matched route details
 <b>Signature:</b>
 
 ```typescript
-readonly route: RecursiveReadonly<KibanaRequestRoute>;
+readonly route: RecursiveReadonly<KibanaRequestRoute<Method>>;
 ```

docs/development/core/server/kibana-plugin-server.kibanarequestroute.md

@@ -9,14 +9,14 @@ Request specific route information exposed to a handler.
 <b>Signature:</b>
 
 ```typescript
-export interface KibanaRequestRoute 
+export interface KibanaRequestRoute<Method extends RouteMethod> 
 ```
 
 ## Properties
 
 |  Property | Type | Description |
 |  --- | --- | --- |
-|  [method](./kibana-plugin-server.kibanarequestroute.method.md) | <code>RouteMethod &#124; 'patch' &#124; 'options'</code> |  |
-|  [options](./kibana-plugin-server.kibanarequestroute.options.md) | <code>Required&lt;RouteConfigOptions&gt;</code> |  |
+|  [method](./kibana-plugin-server.kibanarequestroute.method.md) | <code>Method</code> |  |
+|  [options](./kibana-plugin-server.kibanarequestroute.options.md) | <code>KibanaRequestRouteOptions&lt;Method&gt;</code> |  |
 |  [path](./kibana-plugin-server.kibanarequestroute.path.md) | <code>string</code> |  |
 

docs/development/core/server/kibana-plugin-server.kibanarequestroute.method.md

@@ -7,5 +7,5 @@
 <b>Signature:</b>
 
 ```typescript
-method: RouteMethod | 'patch' | 'options';
+method: Method;
 ```

docs/development/core/server/kibana-plugin-server.kibanarequestroute.options.md

@@ -7,5 +7,5 @@
 <b>Signature:</b>
 
 ```typescript
-options: Required<RouteConfigOptions>;
+options: KibanaRequestRouteOptions<Method>;
 ```

docs/development/core/server/kibana-plugin-server.kibanarequestrouteoptions.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [KibanaRequestRouteOptions](./kibana-plugin-server.kibanarequestrouteoptions.md)
+
+## KibanaRequestRouteOptions type
+
+Route options: If 'GET' or 'OPTIONS' method, body options won't be returned.
+
+<b>Signature:</b>
+
+```typescript
+export declare type KibanaRequestRouteOptions<Method extends RouteMethod> = Method extends 'get' | 'options' ? Required<Omit<RouteConfigOptions<Method>, 'body'>> : Required<RouteConfigOptions<Method>>;
+```

docs/development/core/server/kibana-plugin-server.md

@@ -84,6 +84,8 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  [RequestHandlerContext](./kibana-plugin-server.requesthandlercontext.md) | Plugin specific context passed to a route handler.<!-- -->Provides the following clients: - [savedObjects.client](./kibana-plugin-server.savedobjectsclient.md) - Saved Objects client which uses the credentials of the incoming request - [elasticsearch.dataClient](./kibana-plugin-server.scopedclusterclient.md) - Elasticsearch data client which uses the credentials of the incoming request - [elasticsearch.adminClient](./kibana-plugin-server.scopedclusterclient.md) - Elasticsearch admin client which uses the credentials of the incoming request |
 |  [RouteConfig](./kibana-plugin-server.routeconfig.md) | Route specific configuration. |
 |  [RouteConfigOptions](./kibana-plugin-server.routeconfigoptions.md) | Additional route options. |
+|  [RouteConfigOptionsBody](./kibana-plugin-server.routeconfigoptionsbody.md) | Additional body options for a route |
+|  [RouteSchemas](./kibana-plugin-server.routeschemas.md) | RouteSchemas contains the schemas for validating the different parts of a request. |
 |  [SavedObject](./kibana-plugin-server.savedobject.md) |  |
 |  [SavedObjectAttributes](./kibana-plugin-server.savedobjectattributes.md) | The data for a Saved Object is stored as an object in the <code>attributes</code> property. |
 |  [SavedObjectReference](./kibana-plugin-server.savedobjectreference.md) | A reference to another saved object. |
@@ -133,6 +135,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  Variable | Description |
 |  --- | --- |
 |  [kibanaResponseFactory](./kibana-plugin-server.kibanaresponsefactory.md) | Set of helpers used to create <code>KibanaResponse</code> to form HTTP response on an incoming request. Should be returned as a result of [RequestHandler](./kibana-plugin-server.requesthandler.md) execution. |
+|  [validBodyOutput](./kibana-plugin-server.validbodyoutput.md) | The set of valid body.output |
 
 ## Type Aliases
 
@@ -156,6 +159,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  [IsAuthenticated](./kibana-plugin-server.isauthenticated.md) | Return authentication status for a request. |
 |  [ISavedObjectsRepository](./kibana-plugin-server.isavedobjectsrepository.md) | See [SavedObjectsRepository](./kibana-plugin-server.savedobjectsrepository.md) |
 |  [IScopedClusterClient](./kibana-plugin-server.iscopedclusterclient.md) | Serves the same purpose as "normal" <code>ClusterClient</code> but exposes additional <code>callAsCurrentUser</code> method that doesn't use credentials of the Kibana internal user (as <code>callAsInternalUser</code> does) to request Elasticsearch API, but rather passes HTTP headers extracted from the current user request to the API.<!-- -->See [ScopedClusterClient](./kibana-plugin-server.scopedclusterclient.md)<!-- -->. |
+|  [KibanaRequestRouteOptions](./kibana-plugin-server.kibanarequestrouteoptions.md) | Route options: If 'GET' or 'OPTIONS' method, body options won't be returned. |
 |  [KibanaResponseFactory](./kibana-plugin-server.kibanaresponsefactory.md) | Creates an object containing request response payload, HTTP headers, error details, and other data transmitted to the client. |
 |  [KnownHeaders](./kibana-plugin-server.knownheaders.md) | Set of well-known HTTP headers. |
 |  [LifecycleResponseFactory](./kibana-plugin-server.lifecycleresponsefactory.md) | Creates an object containing redirection or error response with error details, HTTP headers, and other data transmitted to the client. |
@@ -176,8 +180,9 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  [ResponseError](./kibana-plugin-server.responseerror.md) | Error message and optional data send to the client in case of error. |
 |  [ResponseErrorAttributes](./kibana-plugin-server.responseerrorattributes.md) | Additional data to provide error details. |
 |  [ResponseHeaders](./kibana-plugin-server.responseheaders.md) | Http response headers to set. |
+|  [RouteContentType](./kibana-plugin-server.routecontenttype.md) | The set of supported parseable Content-Types |
 |  [RouteMethod](./kibana-plugin-server.routemethod.md) | The set of common HTTP methods supported by Kibana routing. |
-|  [RouteRegistrar](./kibana-plugin-server.routeregistrar.md) | Handler to declare a route. |
+|  [RouteRegistrar](./kibana-plugin-server.routeregistrar.md) | Route handler common definition |
 |  [SavedObjectAttribute](./kibana-plugin-server.savedobjectattribute.md) | Type definition for a Saved Object attribute value |
 |  [SavedObjectAttributeSingle](./kibana-plugin-server.savedobjectattributesingle.md) | Don't use this type, it's simply a helper type for [SavedObjectAttribute](./kibana-plugin-server.savedobjectattribute.md) |
 |  [SavedObjectsClientContract](./kibana-plugin-server.savedobjectsclientcontract.md) | Saved Objects is Kibana's data persisentence mechanism allowing plugins to use Elasticsearch for storing plugin state.<!-- -->\#\# SavedObjectsClient errors<!-- -->Since the SavedObjectsClient has its hands in everything we are a little paranoid about the way we present errors back to to application code. Ideally, all errors will be either:<!-- -->1. Caused by bad implementation (ie. undefined is not a function) and as such unpredictable 2. An error that has been classified and decorated appropriately by the decorators in [SavedObjectsErrorHelpers](./kibana-plugin-server.savedobjectserrorhelpers.md)<!-- -->Type 1 errors are inevitable, but since all expected/handle-able errors should be Type 2 the <code>isXYZError()</code> helpers exposed at <code>SavedObjectsErrorHelpers</code> should be used to understand and manage error responses from the <code>SavedObjectsClient</code>.<!-- -->Type 2 errors are decorated versions of the source error, so if the elasticsearch client threw an error it will be decorated based on its type. That means that rather than looking for <code>error.body.error.type</code> or doing substring checks on <code>error.body.error.reason</code>, just use the helpers to understand the meaning of the error:<!-- -->\`\`\`<!-- -->js if (SavedObjectsErrorHelpers.isNotFoundError(error)) { // handle 404 }<!-- -->if (SavedObjectsErrorHelpers.isNotAuthorizedError(error)) { // 401 handling should be automatic, but in case you wanted to know }<!-- -->// always rethrow the error unless you handle it throw error; \`\`\`<!-- -->\#\#\# 404s from missing index<!-- -->From the perspective of application code and APIs the SavedObjectsClient is a black box that persists objects. One of the internal details that users have no control over is that we use an elasticsearch index for persistance and that index might be missing.<!-- -->At the time of writing we are in the process of transitioning away from the operating assumption that the SavedObjects index is always available. Part of this transition is handling errors resulting from an index missing. These used to trigger a 500 error in most cases, and in others cause 404s with different error messages.<!-- -->From my (Spencer) perspective, a 404 from the SavedObjectsApi is a 404; The object the request/call was targeting could not be found. This is why \#14141 takes special care to ensure that 404 errors are generic and don't distinguish between index missing or document missing.<!-- -->\#\#\# 503s from missing index<!-- -->Unlike all other methods, create requests are supposed to succeed even when the Kibana index does not exist because it will be automatically created by elasticsearch. When that is not the case it is because Elasticsearch's <code>action.auto_create_index</code> setting prevents it from being created automatically so we throw a special 503 with the intention of informing the user that their Elasticsearch settings need to be updated.<!-- -->See [SavedObjectsClient](./kibana-plugin-server.savedobjectsclient.md) See [SavedObjectsErrorHelpers](./kibana-plugin-server.savedobjectserrorhelpers.md) |

docs/development/core/server/kibana-plugin-server.requesthandler.md

@@ -9,7 +9,7 @@ A function executed when route path matched requested resource path. Request han
 <b>Signature:</b>
 
 ```typescript
-export declare type RequestHandler<P extends ObjectType, Q extends ObjectType, B extends ObjectType> = (context: RequestHandlerContext, request: KibanaRequest<TypeOf<P>, TypeOf<Q>, TypeOf<B>>, response: KibanaResponseFactory) => IKibanaResponse<any> | Promise<IKibanaResponse<any>>;
+export declare type RequestHandler<P extends ObjectType, Q extends ObjectType, B extends ObjectType | Type<Buffer> | Type<Stream>, Method extends RouteMethod = any> = (context: RequestHandlerContext, request: KibanaRequest<TypeOf<P>, TypeOf<Q>, TypeOf<B>, Method>, response: KibanaResponseFactory) => IKibanaResponse<any> | Promise<IKibanaResponse<any>>;
 ```
 
 ## Example

docs/development/core/server/kibana-plugin-server.routeconfig.md

@@ -9,14 +9,14 @@ Route specific configuration.
 <b>Signature:</b>
 
 ```typescript
-export interface RouteConfig<P extends ObjectType, Q extends ObjectType, B extends ObjectType> 
+export interface RouteConfig<P extends ObjectType, Q extends ObjectType, B extends ObjectType | Type<Buffer> | Type<Stream>, Method extends RouteMethod> 
 ```
 
 ## Properties
 
 |  Property | Type | Description |
 |  --- | --- | --- |
-|  [options](./kibana-plugin-server.routeconfig.options.md) | <code>RouteConfigOptions</code> | Additional route options [RouteConfigOptions](./kibana-plugin-server.routeconfigoptions.md)<!-- -->. |
+|  [options](./kibana-plugin-server.routeconfig.options.md) | <code>RouteConfigOptions&lt;Method&gt;</code> | Additional route options [RouteConfigOptions](./kibana-plugin-server.routeconfigoptions.md)<!-- -->. |
 |  [path](./kibana-plugin-server.routeconfig.path.md) | <code>string</code> | The endpoint \_within\_ the router path to register the route. |
 |  [validate](./kibana-plugin-server.routeconfig.validate.md) | <code>RouteSchemas&lt;P, Q, B&gt; &#124; false</code> | A schema created with <code>@kbn/config-schema</code> that every request will be validated against. |
 

docs/development/core/server/kibana-plugin-server.routeconfig.options.md

@@ -9,5 +9,5 @@ Additional route options [RouteConfigOptions](./kibana-plugin-server.routeconfig
 <b>Signature:</b>
 
 ```typescript
-options?: RouteConfigOptions;
+options?: RouteConfigOptions<Method>;
 ```

docs/development/core/server/kibana-plugin-server.routeconfigoptions.body.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [RouteConfigOptions](./kibana-plugin-server.routeconfigoptions.md) &gt; [body](./kibana-plugin-server.routeconfigoptions.body.md)
+
+## RouteConfigOptions.body property
+
+Additional body options [RouteConfigOptionsBody](./kibana-plugin-server.routeconfigoptionsbody.md)<!-- -->.
+
+<b>Signature:</b>
+
+```typescript
+body?: Method extends 'get' | 'options' ? undefined : RouteConfigOptionsBody;
+```