Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[test] java tests for archive packaging #30734

Merged

Conversation

andyb-elastic
Copy link
Contributor

Ports the first couple tests for archive distributions from the old bats
project to the new java project that includes windows platforms,
consolidating them into one test method that tests that the
distributions can be extracted and their contents verified. Includes the
zip distributions which were not tested in the bats project.

For #26741

This takes into account @rjernst's changes in #30596

Ports the first couple tests for archive distributions from the old bats
project to the new java project that includes windows platforms,
consolidating them into one test method that tests that the
distributions can be extracted and their contents verified. Includes the
zip distributions which were not tested in the bats project.

For elastic#26741
@andyb-elastic andyb-elastic added >test Issues or PRs that are addressing/adding tests :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts v7.0.0 v6.4.0 labels May 18, 2018
@andyb-elastic andyb-elastic requested a review from rjernst May 18, 2018 17:54
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

* override {@link ArchiveTestCase#distribution()}. These tests should be the same across all archive distributions
*/
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public abstract class ArchiveTestCase {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally I used parameterized tests to serve this purpose but it seemed like this was a more easily understandable way to achieve the same thing

@@ -237,6 +237,7 @@ def linux_common(config,

config.vm.provision 'markerfile', type: 'shell', inline: <<-SHELL
touch /etc/is_vagrant_vm
touch /is_vagrant_vm # for consistency between linux and windows
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make a follow up changing the bats tests to expect it in this location

assertThat("distribution file must exist", Files.exists(distributionFile), is(true));
assertThat("elasticsearch must not already be installed", lsGlob(baseInstallPath, "elasticsearch*"), empty());

if (distribution.packaging == Distribution.Packaging.TAR) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this kind of thing should be moved to ArchiveTestCase and you subclass it for each time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like you move the entire installArchive method to the abstract superclass.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean this kind of thing

// ArchiveTestCase.java

abstract void extractArchive(Path install);

Installation installArchive(Distribution distribution) {
  // first half of Archives#installArchive
  extractArchive(installLocation, distributionFile);
  // rest of Archives#installArchive
}

// DefaultTarTests.java
// ditto for OssTarTests.java

@Override
void extractArchive(Path install, Path distributionFile) {
  // run the untar command
}

I feel like there's some value from not coupling the install logic to the test class too tightly (although maybe that's a misplaced idea). Maybe something like this?

// ArchiveTestCase.java

abstract void extractArchive(Path install, Path distributionFile);

@Test
public void test10Install() {
  installation = Archives.installArchive(distribution, this::extractArchive);
  Archives.verify(installation, distribution());
}

// Archives.java

public Installation installArchive(Distribution d, BiConsumer<Path, Path> extractor) {
  // first half of Archives.installArchive
  extractor.accept(installPath, distributionFile);
  // rest of Archives.installArchive
}

// DefaultTarTests.java
// ditto for OssTarTests.java

@Override
void extractArchive(Path install, Path distributionFile) {
  // run the untar command
}

That said, the reason I ended up doing it this way was that I figured it was easier to follow if all this logic was in one place, even if all the if-branching was kind of gross, because there's only two archive types (and it seems unlikely there would ever be significantly more than that). What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine to do it as you are doing it. We could change it to my way later on if we figure it is better. I don't think it is a blocker.

I just look at your way and think "this made a ton of sense in bash scripts but I want to OO all the things in Java because I'm a Java developer." I admit that is a bad habit some times. I don't think it'd be too weird to keep the install logic in the superclass but you certainly are right that there aren't that many variants.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this made a ton of sense in bash scripts but I want to OO all the things in Java because I'm a Java developer

Yeah this was exactly what I did at first and then walked it back to here since it seemed simpler :)

I'll take a look at the other places in Archives' methods where this kind of stuff happens and see if I can find a happy medium

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried a couple variations of what we discussed. I still feel like the benefit of being able to answer the question "how are the archive distros installed" by looking in one place outweighs that place being a little gross - not in general, but in this instance because we don't expect to have many, if any additional cases.

I agree that if it does change it would be better to do something like your suggestion. I'll also keep thinking about other ways to do it as I revise this

"NOTICE.txt",
"LICENSE.txt",
"README.textile"
).forEach(doc -> assertThat(es.home.resolve(doc), file(File, owner, owner, p644)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't passing for me:

Expected: "file/directory: "<File> with owner "elasticsearch" with group "elasticsearch" with posix permissions [<OWNER_READ>,<OWNER_WRITE>,<GROUP_READ>,<OTHERS_READ>]
     but: path </tmp/elasticsearch/NOTICE.txt>Has permissions [GROUP_WRITE, OWNER_READ, GROUP_READ, OWNER_WRITE, OTHERS_READ]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird, it passes locally for me and in CI. Which box and test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually if you got a full reproduce line that would be best - I'm pretty sure the randomization doesn't affect anything about the current version installs, but best to check anyway

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran:

./gradlew -p qa/vagrant/ vagrantCentos7#javaPackagingTest

I'm pulling and trying again.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. That command fails for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird, still not seeing it. If you still have it mind sharing the test seed it used

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got this:

REPRODUCE WITH: ./gradlew :qa:vagrant:vagrantCentos7#javaPackagingTest -Dtests.seed=924BDF23E0249263 

Though I'm not sure how useful that is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andyb-elastic @nik9000 I recommend comparing umasks.

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. I think it would be good to have some type of documentation on how to write/extend these tests, as there are a number of classes now that one must understand to change these tests. But that can be in followups.

private static void purgePackagesLinux() {
final Shell sh = new Shell();

if (isRPM()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these all mutually exclusive?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They should be (at least deb vs rpm). That said I think the logic here does make sense (if any of these package managers exist on the system, remove the packages from them). It's the same in the bats tests

this.workingDirectory = workingDirectory;
}

public Result runAndAssertSuccess(String... command) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is far more often we want to assert success. Can we flip this around have have bare run assert, and have a runIgnoreFailure for the other case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good point, it's also probably more intuitive that way

The use of java.io.File seems unavoidable because that's the type that
processbuilder expects when setting a working directory, there doesn't
appear to be an alternative that uses Path

We can't know ahead of time what character set subprocesses' output will
use. It seems best to use the jvm's default character set, which is
based on the platform's character set. For all linux platforms we test
that's utf8, and for the windows platforms we test that's iso-8859-1 aka
windows codepage 1252; in all cases this is reflected correctly in the
jvm's default character set
@andyb-elastic
Copy link
Contributor Author

Talked to Nik and he said he's ✅ on this

@andyb-elastic andyb-elastic merged commit a1b5381 into elastic:master May 23, 2018
@andyb-elastic
Copy link
Contributor Author

@rjernst sure thing can do a follow up with some docs about writing tests for this project. Where's the right place for those to go? Readme in qa/vagrant?

andyb-elastic added a commit to andyb-elastic/elasticsearch that referenced this pull request May 23, 2018
Ports the first couple tests for archive distributions from the old bats
project to the new java project that includes windows platforms,
consolidating them into one test method that tests that the
distributions can be extracted and their contents verified. Includes the
zip distributions which were not tested in the bats project.
@rjernst
Copy link
Member

rjernst commented May 23, 2018

Where's the right place for those to go? Readme in qa/vagrant?

That's a good place to start.

andyb-elastic added a commit that referenced this pull request May 23, 2018
Ports the first couple tests for archive distributions from the old bats
project to the new java project that includes windows platforms,
consolidating them into one test method that tests that the
distributions can be extracted and their contents verified. Includes the
zip distributions which were not tested in the bats project.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request May 23, 2018
* master:
  [DOCS] Splits auditing.asciidoc into smaller files
  Reintroduce mandatory http pipelining support (elastic#30820)
  Painless: Types Section Clean Up (elastic#30283)
  Add support for indexed shape routing in geo_shape query (elastic#30760)
  [test] java tests for archive packaging (elastic#30734)
  Revert "Make http pipelining support mandatory (elastic#30695)" (elastic#30813)
  [DOCS] Fix more edit URLs in Stack Overview (elastic#30704)
  Use correct cluster state version for node fault detection (elastic#30810)
  Change serialization version of doc-value fields.
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (elastic#30797)
  [TEST] Don't expect acks when isolating nodes
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request May 23, 2018
* master: (25 commits)
  [DOCS] Splits auditing.asciidoc into smaller files
  Reintroduce mandatory http pipelining support (elastic#30820)
  Painless: Types Section Clean Up (elastic#30283)
  Add support for indexed shape routing in geo_shape query (elastic#30760)
  [test] java tests for archive packaging (elastic#30734)
  Revert "Make http pipelining support mandatory (elastic#30695)" (elastic#30813)
  [DOCS] Fix more edit URLs in Stack Overview (elastic#30704)
  Use correct cluster state version for node fault detection (elastic#30810)
  Change serialization version of doc-value fields.
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (elastic#30797)
  [TEST] Don't expect acks when isolating nodes
  Add a `format` option to `docvalue_fields`. (elastic#29639)
  Fixes UpdateSettingsRequestStreamableTests mutate bug
  Mustes {p0=snapshot.get_repository/10_basic/*} YAML test
  Revert "Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled"
  Only allow x-pack metadata if all nodes are ready (elastic#30743)
  Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled
  Use original settings on full-cluster restart (elastic#30780)
  Only ack cluster state updates successfully applied on all nodes (elastic#30672)
  ...
dnhatn added a commit that referenced this pull request May 24, 2018
* 6.x:
  [DOCS] Fixes typos in security settings
  Add support for indexed shape routing in geo_shape query (#30760)
  [DOCS] Splits auditing.asciidoc into smaller files
  Painless: Types Section Clean Up (#30283)
  [test] java tests for archive packaging (#30734)
  Deprecate http.pipelining setting (#30786)
  [DOCS] Fix more edit URLs in Stack Overview (#30704)
  Use correct cluster state version for node fault detection (#30810)
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (#30797)
  Change serialization version of doc-value fields.
  Add a `format` option to `docvalue_fields`. (#29639)
  [TEST] Don't expect acks when isolating nodes
  Fixes UpdateSettingsRequestStreamableTests mutate bug
  Revert "Add more yaml tests for get alias API (#29513)"
  Revert "Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled"
  Only allow x-pack metadata if all nodes are ready (#30743)
  Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled
  Use original settings on full-cluster restart (#30780)
  Only ack cluster state updates successfully applied on all nodes (#30672)
  Replace Request#setHeaders with addHeader (#30588)
  [TEST] remove endless wait in RestClientTests (#30776)
  QA: Add xpack tests to rolling upgrade (#30795)
  Add support for search templates to the high-level REST client. (#30473)
  Reduce CLI scripts to one-liners on Windows (#30772)
  Fold RestGetAllSettingsAction in RestGetSettingsAction (#30561)
  Add more yaml tests for get alias API (#29513)
  [Docs] Fix script-fields snippet execution (#30693)
  Convert FieldCapabilitiesResponse to a ToXContentObject. (#30182)
  Remove assert statements from field caps documentation. (#30601)
  Fix a bug in FieldCapabilitiesRequest#equals and hashCode. (#30181)
  Add support for field capabilities to the high-level REST client. (#29664)
  [DOCS] Add SAML configuration information (#30548)
  [DOCS] Remove X-Pack references from SQL CLI (#30694)
  [Docs] Fix typo in circuit breaker docs (#29659)
  [Feature] Adding a char_group tokenizer (#24186)
  Increase the maximum number of filters that may be in the cache. (#30655)
  [Docs] Fix broken cross link in documentation
  Test: wait for netty threads in a JUnit ClassRule (#30763)
  [Security] Include an empty json object in an json array when FLS filters out all fields (#30709)
  [DOCS] fixed incorrect default
  [TEST] Wait for CS to be fully applied in testDeleteCreateInOneBulk
  Enable installing plugins from snapshots.elastic.co (#30765)
  Ignore empty completion input (#30713)
  Fix docs failure on language analyzers (#30722)
  [Docs] Fix inconsistencies in snapshot/restore doc (#30480)
  Add Delete Repository High Level REST API (#30666)
  Reduce CLI scripts to one-liners (#30759)
dnhatn added a commit that referenced this pull request May 24, 2018
* master:
  [DOCS] Fixes typos in security settings
  Fix GeoShapeQueryBuilder serialization after backport
  [DOCS] Splits auditing.asciidoc into smaller files
  Reintroduce mandatory http pipelining support (#30820)
  Painless: Types Section Clean Up (#30283)
  Add support for indexed shape routing in geo_shape query (#30760)
  [test] java tests for archive packaging (#30734)
  Revert "Make http pipelining support mandatory (#30695)" (#30813)
  [DOCS] Fix more edit URLs in Stack Overview (#30704)
  Use correct cluster state version for node fault detection (#30810)
  Change serialization version of doc-value fields.
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (#30797)
  [TEST] Don't expect acks when isolating nodes
  Add a `format` option to `docvalue_fields`. (#29639)
  Fixes UpdateSettingsRequestStreamableTests mutate bug
  Mustes {p0=snapshot.get_repository/10_basic/*} YAML test
  Revert "Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled"
  Only allow x-pack metadata if all nodes are ready (#30743)
  Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled
  Use original settings on full-cluster restart (#30780)
  Only ack cluster state updates successfully applied on all nodes (#30672)
  Expose Lucene's FeatureField. (#30618)
  Fix a grammatical error in the 'search types' documentation.
  Remove http pipelining from integration test case (#30788)
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request May 25, 2018
* es/ccr: (55 commits)
  [DOCS] Fixes typos in security settings
  Fix GeoShapeQueryBuilder serialization after backport
  [DOCS] Splits auditing.asciidoc into smaller files
  Reintroduce mandatory http pipelining support (elastic#30820)
  Painless: Types Section Clean Up (elastic#30283)
  Add support for indexed shape routing in geo_shape query (elastic#30760)
  [test] java tests for archive packaging (elastic#30734)
  Revert "Make http pipelining support mandatory (elastic#30695)" (elastic#30813)
  [DOCS] Fix more edit URLs in Stack Overview (elastic#30704)
  Use correct cluster state version for node fault detection (elastic#30810)
  Change serialization version of doc-value fields.
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (elastic#30797)
  [TEST] Don't expect acks when isolating nodes
  Mute CorruptedFileIT in CCR
  Add a `format` option to `docvalue_fields`. (elastic#29639)
  Fixes UpdateSettingsRequestStreamableTests mutate bug
  Mustes {p0=snapshot.get_repository/10_basic/*} YAML test
  Revert "Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled"
  Only allow x-pack metadata if all nodes are ready (elastic#30743)
  ...
@andyb-elastic andyb-elastic mentioned this pull request Jul 17, 2018
24 tasks
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts Team:Delivery Meta label for Delivery team >test Issues or PRs that are addressing/adding tests v6.4.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants