Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kbn-es] Add support to kbn-es and kbn-test for data archives #28723

Merged
merged 4 commits into from
Jan 23, 2019

Conversation

joshdover
Copy link
Contributor

Summary

This adds support to both kbn-es and kbn-test to unpack a zip or tarball of an Elasticsearch data directory to seed a cluster with specific data. This allows us to test opening ES state from a previous version of Elasticsearch and is required to support some of the testing needed for the Upgrade Assistant in #27457.

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations

@elasticmachine

This comment has been minimized.

@joshdover joshdover force-pushed the kbn-es-data-archive branch from 8d9cd2c to a81da7f Compare January 14, 2019 23:42
@elasticmachine

This comment has been minimized.

@joshdover

This comment has been minimized.

@elasticmachine

This comment has been minimized.

@joshdover joshdover force-pushed the kbn-es-data-archive branch from a81da7f to e3d57d1 Compare January 15, 2019 17:38
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Contributor

@spalger spalger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried making a data directory and using the --data-archive, but when I started up Elasticsearch it didn't have any data.

This is the data directory that I used: data-dir.tar.gz

packages/kbn-es/src/cluster.js Outdated Show resolved Hide resolved
@joshdover
Copy link
Contributor Author

@spalger How was your zip structured?

The one I have that works (upgrade_assistant.zip) is like this:

upgrade_assistant
└── data
    └── nodes
        └── 0

So the root directory is the name of the zip. We could change this to have the data directory as the root, and either way I'll add some documentation somewhere. Thoughts?

@elasticmachine
Copy link
Contributor

💔 Build Failed

@joshdover
Copy link
Contributor Author

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Co-Authored-By: joshdover <me@joshdover.com>
@joshdover joshdover force-pushed the kbn-es-data-archive branch from 6808600 to e237646 Compare January 23, 2019 21:03
@joshdover
Copy link
Contributor Author

Updated to accept data zips that only contain a data directory

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@joshdover joshdover merged commit 45a4d3b into elastic:master Jan 23, 2019
@joshdover joshdover deleted the kbn-es-data-archive branch January 23, 2019 22:45
@joshdover joshdover removed the review label Jan 23, 2019
joshdover added a commit to joshdover/kibana that referenced this pull request Jan 23, 2019
…c#28723)

* Add support to kbn-es and kbn-test for data archives

* Fix log indent

* Decompress directly to data directory

Co-Authored-By: joshdover <me@joshdover.com>

* Improve logs
joshdover added a commit that referenced this pull request Jan 24, 2019
#29221)

* Add support to kbn-es and kbn-test for data archives

* Fix log indent

* Decompress directly to data directory

Co-Authored-By: joshdover <me@joshdover.com>

* Improve logs
@joshdover joshdover added the dev label Mar 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev Team:Operations Team label for Operations Team v6.7.0 v7.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants