Import / Export API for Dashboards

[simianhacker]

This PR creates an import / export API for dashboards. The current implementation under Mangement > Saved Objects allows you to only export everything or individual dashboards (with out the supporting saved objects like searches, visualizations or index patterns). This creates 2 API endpoints at /api/kibana/dashboards/export and /api/kibana/dashboards/import. The user can POST to the export end point an array of dashboard IDs. The response will contain ALL (searches, index patterns, visualizations, etc) the objects needed for that dashboard in an object with a version. To import a dashboard (an all of it's supporting objects) the user can POST the same object returned from the export endpoint to the import endpoint. The import endpoint will check the version and try and upgrade the objects if they are from a previous version; and import them into the .kibana index; skipping existing objects.

[epixa]

Does the export request actually result in any changes internally? If not, why not just do a GET instead?

[simianhacker]

No reason other then I was just trying to provide the maximum options so you could either use a GET or POST depending on which HTTP religion you subscribed to :D

[xycloud]

mark

[simianhacker]

Hamill

[epixa]

@xycloud If you just want to see updates for a given issue, please use the "subscribe" link on the right side of posts rather than leaving a generic comment

[simianhacker]

@monicasarbu I added two query arguments to the import api:

• ?exclude=<type> - For excluding certain types from being imported (aka index-patterns) and an option for
• ?force=true - To force the import to overwrite existing objects.

[epixa]

There's no documentation for these endpoints yet as they are still experimental, but if you like to live on the bleeding edge, you can feel free to give them a whirl. We are using these APIs for beats dashboard setup in 6.0 onward, so they are probably going to remain pretty stable despite being experimental.

In the absence of documentation, I recommend setting up a Kibana dashboard and then using the export API (GET /api/kibana/dashboards/export?dashboard=<id of dashboard>). The response body of the export API should be a valid request body for the import API (POST /api/kibana/dashboards/import). @simianhacker Please correct me if I'm wrong...

[hogarthj]

@epixa it is valid ... it's that response body I'm grabbing with the ansible above (from my dev instance) and then import directly to the test and prod instances.

But you can't just /api/kibana/dashboards/export AFAIK ... you need the id of the dashboard you want to grab or it errors stating it's missing a dashboard.id

[epixa]

@hogarthj Thanks for the clarification, and yes, you're correct. You must specify dashboard as a query parameter, as you did in your ansible script. I've updated my comment.

[simianhacker]

@epixa and @hogarthj Yes... the response from the export is the body for the import. You are required to provide dashboard arguments for each dashboard id you're trying to export. Here is a what a typical dashboard export will look like (with X-Pack):

curl --user <username>:<password> -XGET localhost:5601/api/kibana/dashboards/export?dashboard=<some-dashboard-uuid> > my-dashboards.json

To import it looks like

curl --user <username>:<password> -XPOST localhost:5601/api/kibana/dashboards/import -H 'kbn-xsrf:true' -H 'Content-type:application/json' -d @./my-dashboards.json

If you are NOT using x-pack you can omit the --user <username>:<password> portion.

[dkirrane]

curl -XGET http://localhost:5601/app/kibana/dashboards/export?dashboard=33797130-97ae-11e7-aa10-79a79f2e360b
{"statusCode":404,"error":"Not Found"}

The dashboard _id I used is the one returned in id field by this API call
curl -s http://localhost:9200/.kibana/dashboard/_search?pretty

...
      {
        "_index" : ".kibana",
        "_type" : "dashboard",
        "_id" : "33797130-97ae-11e7-aa10-79a79f2e360b",
...

[tylersmalley]

@dkirrane, alternatively you can use the Saved Objects API to capture the ID's:

$ curl -s http://localhost:5601/api/saved_objects/dashboard | python -m json.tool
{
    "page": 1,
    "per_page": 20,
    "saved_objects": [
        {
            "attributes": {
                "description": "",
                "hits": 0,
                "kibanaSavedObjectMeta": {
                    "searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"lucene\"},\"filter\":[],\"highlightAll\":true,\"version\":true}"
                },
                "optionsJSON": "{\"darkTheme\":false}",
                "panelsJSON": "[]",
                "timeRestore": false,
                "title": "New Dashboard",
                "uiStateJSON": "{}",
                "version": 1
            },
            "id": "9a8bc880-9805-11e7-aaf0-9134624698e1",
            "type": "dashboard",
            "updated_at": "2017-09-12T21:59:09.319Z",
            "version": 1
        }
    ],
    "total": 1
}

[dkirrane]

@tylersmalley tried that. Got the id but still returning 404. Tried with both Kibana 5.5.1 and 5.5.2
curl -XGET http://localhost:5601/app/kibana/dashboards/export?dashboard=f2f2a960-97df-11e7-9226-8163619d402b

{
    "error": "Not Found",
    "statusCode": 404
}

[epixa]

Use api in the url rather than app.

[cjauvin]

This API is very handy, but there's something I'm not sure I understand: assuming that the only way to export a dashboard with the UI is to use Management -> Saved Objects -> Export Everything.. then it means that it's not possible to import a UI-exported dashboard via your API, because the two formats are not compatible (in particular, the UI exports a JSON list, while the API requires an object).

Shouldn't the API be accompanied by a UI update which unifies the import/export processes, both from their API and UI perspectives?

[epixa]

@cjauvin I suspect the two outputs will converge at some point, but they don't do the same thing at the moment, and we're not yet ready to say for sure whether they will converge or not. "Export everything" is a raw dump of certain types of saved objects, whereas this new import/export API is a more intelligent mechanism to export all related saved objects for a given dashboard.

Out of curiosity, do you have a specific workflow in mind that would benefit from a shared format?

[cjauvin]

@epixa Thanks for the quick answer! I do have a specific workflow in mind: we are currently provisioning an ES + Kibana stack running in a Kubernetes cluster, and in this context your API is very handy as a way to automatically provision required dashboards at the cluster creation and setup stage (with the Helm tool, specifically). However, we would like it also to be extra easy for a cluster operator to be able to contribute any dashboard update (say a title change) back into the setup scripts.. and for this simply exporting (from the UI) + copy/pasting would probably be the path of least friction.

[simianhacker]

@epixa I've always envision doing an export from the dashboard listing page, click the checkboxes next to the dashboards you want to export and click the new export button that shows up next to the delete button. And it just redirects you the the API URL with the query arguments for the dashboards. BTW... the export response already has the content-disposition header for treating it as a download. So really all that is left is the UI work. :D

[epixa]

@simianhacker That experience sounds great to me. We can't just jump into it since import needs to accommodate existing json exports, but it sounds like a good goal

[yangwong9664]

Great API, really useful for our project, I have no problems with exporting. Do we have any examples of the request body for an import POST?

[simianhacker]

@yangwong9664 it’s the same as the response from the export.

[yangwong9664]

@simianhacker Thanks.

[gingerwizard]

Is there a way to simply dump all objects through this API? I see the import takes a list of objects.

[gingerwizard]

ah i see #11632