Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging: Remove windows bin files from the tar distribution #30596

Merged
merged 4 commits into from
Jun 18, 2018

Conversation

rjernst
Copy link
Member

@rjernst rjernst commented May 15, 2018

This commit removes windows specific files from the tar distribution.
Windows users use the zip, linux users use the tar.

This commit removes windows specific files from the tar distribution.
Windows users use the zip, linux users use the tar.
@rjernst rjernst added >breaking :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts v7.0.0 labels May 15, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@rjernst rjernst requested a review from jasontedor May 15, 2018 00:08
@rjernst rjernst changed the title Packaging: Remove windows bat files from the tar distribution Packaging: Remove windows bin files from the tar distribution May 15, 2018
@andyb-elastic
Copy link
Contributor

My two cents: it would be surprising to me to find that the tgz and zip distributions had different contents. I do see how this makes sense though when considering file permission/attribute retention between the two formats.

@rjernst
Copy link
Member Author

rjernst commented May 15, 2018

@andyb-elastic why is it not surprising then when rpm and deb don't also have these same contents?

@jasontedor
Copy link
Member

jasontedor commented May 15, 2018

@rjernst It would not be reasonable for a Windows user to think that the RPM or Deb package is compatible with Windows. With the Linux subsystem and Git Bash, there are ways for a Windows user to easily consume a .tar.gz. Our download page does not give any indication that the .tar.gz is not compatible with Windows (and today it is compatible). Today we display the downloads as:

screen shot 2018-05-15 at 5 44 59 pm

This is how the Kibana downloads appear:

screen shot 2018-05-15 at 5 46 23 pm

And while Kibana has architecture-specific downloads (because of the Node binary), there is no confusion on Windows which artifact to download whereas there is possibility of confusion with our artifacts.

I can support this change if we change our download page to indicate that the zip distribution is for Windows only and the .tar.gz is for macOS/Linux (and it needs to be macOS/Linux as we support these OS and no other Unix-like OS).

@andyb-elastic
Copy link
Contributor

Right, on Windows even without WSL, extracting the tgz with 7zip and running it seems to work fine and would be a pretty normal thing to do (at least I think).

That said, I agree that making it explicit which platform these distributions are intended for is most helpful. Is the idea to take the bash scripts out of the zip distribution at some point as well?

@rjernst
Copy link
Member Author

rjernst commented May 16, 2018

Is the idea to take the bash scripts out of the zip distribution at some point as well?

I would not be opposed to that. The only caveat is we would then need to select which distribution to use based on the system we are running tests on. And eg -Dtests.distribution would have to be different by selecting whether the oss or default is used, but not the actual distribution.

@rjernst
Copy link
Member Author

rjernst commented May 16, 2018

I can support this change if we change our download page to indicate that the zip distribution is for Windows only and the .tar.gz is for macOS/Linux (and it needs to be macOS/Linux as we support these OS and no other Unix-like OS).

With each release we add the new download artifacts and their link contents, so this would need to be done with the 6.4.0 release (assuming this is merged to 6.x before 6.4 is cut). Is that sufficient?

@jasontedor
Copy link
Member

I am supportive of the proposed changes to the .tar.gz distribution.

With each release we add the new download artifacts and their link contents, so this would need to be done with the 6.4.0 release (assuming this is merged to 6.x before 6.4 is cut). Is that sufficient?

Yes.

@droberts195
Copy link
Contributor

Are there some major benefits in making the .zip distribution not work on Linux/Mac?

It will cause a certain amount of rework in @dolaru's QA test framework for ML, as currently this uses the .zip distribution on all platforms. Obviously it's not impossible to change this, but it does show that this change is going to cause people to have to spend time that could instead be spent on new features. So if there are benefits to making this change that outweigh the costs downstream then I think it would be good to list them on the PR.

@dolaru
Copy link
Member

dolaru commented May 17, 2018

just to be clear on what we're trying to achieve with this PR:

  • .zip distribution contains Windows only binaries
  • .tar.gz contains macOS/Linux only binaries

Also, when i say binaries i mean not just what's in bin/ but nested module binaries too.

is this right?

@jasontedor
Copy link
Member

jasontedor commented May 17, 2018

This PR only changes the packaging content of the tar.gz distribution. It does not change the content of the zip distribution (there is discussion of it here), and it does not change the content of what executables are distributed under the ML module.

@droberts195
Copy link
Contributor

Thanks for clarifying @jasontedor. I think making the .tar.gz distribution not work on Windows is much less of a problem than making the .zip distribution not work on Mac/Linux.

@sophiec20
Copy link
Contributor

sophiec20 commented May 17, 2018

Why are we making this change to the .tar.gz? It doesn't take us all the way to named OS specific distributions, which would align with Kibana and be more intuitive. Am curious if we are aiming towards this?

@rjernst
Copy link
Member Author

rjernst commented May 17, 2018

@sophiec20 There are multiple motivations here:

  • Remove inconsistency across packages (we ship .bat files in rpm/deb right now, but only for xpack scripts)
  • Make it more clear tar.gz is for linux. I want to add (in a followup) our systemd service files to the tar.gz, but I don't think we should add them to the zip too.

It will cause a certain amount of rework in @dolaru's QA test framework for ML, as currently this uses the .zip distribution on all platforms

As Jason mentioned, this PR only removes the windows files from the tar.gz. To me, it is less important to remove the bash scripts from windows, since they will run on windows. That would not preclude labeling the zip and tar.gz files as windows and macos/linux though.

Copy link
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@rjernst rjernst merged commit f3297ed into elastic:master Jun 18, 2018
@rjernst rjernst deleted the no_tar_bat_files branch June 18, 2018 17:02
dnhatn added a commit that referenced this pull request Jun 19, 2018
* master:
  Add get stored script and delete stored script to high level REST API - post backport fix
  Add get stored script and delete stored script to high level REST API (#31355)
  Core: Combine Action and GenericAction (#31405)
  Fix reference to XContentBuilder.string() (#31337)
  Avoid sending duplicate remote failed shard requests (#31313)
  Fix defaults in GeoShapeFieldMapper output (#31302)
  RestAPI: Reject forcemerge requests with a body (#30792)
  Packaging: Remove windows bin files from the tar distribution (#30596)
  Docs: Use the default distribution to test docs (#31251)
  [DOCS] Adds testing for security APIs (#31345)
  Clarify that IP range data can be specified in CIDR notation. (#31374)
  Use system context for cluster state update tasks (#31241)
  Percentile/Ranks should return null instead of NaN when empty (#30460)
  REST high-level client: add validate query API (#31077)
  Move language analyzers from server to analysis-common module. (#31300)
  [Test] Fix :example-plugins:rest-handler on Windows
  Expose lucene's RemoveDuplicatesTokenFilter (#31275)
  Reload secure settings for plugins (#31383)
  Remove some cases in FieldTypeLookupTests that are no longer relevant. (#31381)
  Ensure we don't use a remote profile if cluster name matches (#31331)
  [TEST] Double write alias fault (#30942)
  [DOCS] Fix version in SQL JDBC Maven template
  [DOCS] Improve install and setup section for SQL JDBC
  SQL: Fix rest endpoint names in node stats (#31371)
  Support for remote path in reindex api - post backport fix Closes #22913
  [ML] Put ML filter API response should contain the filter (#31362)
  Support for remote path in reindex api (#31290)
  Add byte array pooling to nio http transport (#31349)
  Remove trial status info from start trial doc (#31365)
  [DOCS] Adds links to release notes and highlights
  add is-write-index flag to aliases (#30942)
  Add rollover-creation-date setting to rolled over index (#31144)
  [ML] Hold ML filter items in sorted set (#31338)
  [Tests] Fix edge case in ScriptedMetricAggregatorTests (#31357)
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>breaking :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts Team:Delivery Meta label for Delivery team v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants