Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download platform/architecture specific binaries #33

Merged
merged 30 commits into from
Jan 27, 2020

Conversation

jperl
Copy link
Contributor

@jperl jperl commented Jan 14, 2020

Resolves #20
Resolves https://github.com/qawolf/qawolf/issues/316

To Do

  • Create GitHub action to download & release binaries on v* tags
  • Create install.js to download binary from release
  • Use separate binary_release for the version
  • Clean up install.js
  • Use non-deprecated github action
  • Alert user in install.js if they have an invalid platform
  • Compress to reduce download size

@jperl jperl changed the title Release binaries Download platform/architecture specific binaries Jan 14, 2020
@jperl
Copy link
Contributor Author

jperl commented Jan 14, 2020

Instead of rely on the package version I will create a separate binary tag / release and include it in the package.json.

  • Now the binaries github action can be triggered and released separately from the package version.
  • This makes it possible to run the tests against different binary releases. Otherwise you would not be able to run the tests until after a release.

.travis.yml Outdated Show resolved Hide resolved
index.js Outdated
@@ -1,24 +1,22 @@
var os = require('os')
var path = require('path')
var os = require("os");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert these changes. They make it really hard to find out later when & how something was changed.

install.js Outdated
var URL = require("url");
var pkg = require("./package");

function httpRequest(url, method, response) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a slightly-higher-level lib like simple-get for this. Unfortunately we'll still disrepect other important parts of the HTTP ecosystem, e.g. caching & proxy support.

@derhuerst
Copy link
Collaborator

Now the binaries github action can be triggered and released separately from the package version.

I think that makes a lot of sense because it decouples ffmpeg releases from ffmpeg-static releases (in both ways).

This makes it possible to run the tests against different binary releases. Otherwise you would not be able to run the tests until after a release.

The tests don't assert the functionality of the ffmpeg binaries anyways, so we don't need that.

@jperl jperl marked this pull request as ready for review January 15, 2020 18:18
@jperl
Copy link
Contributor Author

jperl commented Jan 15, 2020

@derhuerst Sorry for the delay on wrapping this up I am taking a vacation this week and working on this when I can.

The PR should be ready for review now, I made the changes requested. Overall I think this approach ended up being simpler than the multi-package approach.

@jperl
Copy link
Contributor Author

jperl commented Jan 15, 2020

Note: The last commit tests are failing because that commit changes the download url to this repo, and there is not a binary release here yet.

After this PR is merged you should create a b3.0.0 tag so a binary release is created. Any time you want to update which binary release is downloaded by default, you just change ffmpeg-static.binary_release in the package.json.

You can also override it when you install the package with an environment variable ex FFMPEG_BINARY_RELEASE=b4.0.0 npm install

@jperl
Copy link
Contributor Author

jperl commented Jan 15, 2020

I released this as @qawolf/ffmpeg-static and used it in qawolf to check it works. The tests pass and the install works properly.

@jperl jperl force-pushed the release-binaries branch 4 times, most recently from d1cde19 to 11b43e9 Compare January 20, 2020 18:55
@jperl
Copy link
Contributor Author

jperl commented Jan 20, 2020

Leaving the url as qawolf so tests pass. Will change it back before merging. I updated it to compress the binaries so the download size is much smaller.

Copy link
Collaborator

@derhuerst derhuerst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already looking very good!

Given that we only download one executable on install, having the bin structure and the excludes in package.json#files feel overly complicated. Let's

  • let install.js just download & index.js just resolve to /ffmpeg//ffmpeg.exe,
  • let build/index.sh just put the executables in bin (without subdirectories), with the same file names as in the GitHub release.

Thanks for your patience!

.travis.yml Outdated Show resolved Hide resolved
build/index.sh Outdated Show resolved Hide resolved
install.js Outdated Show resolved Hide resolved
install.js Outdated Show resolved Hide resolved
test/index.js Outdated Show resolved Hide resolved
@jperl
Copy link
Contributor Author

jperl commented Jan 27, 2020

@derhuerst Thanks for the review 🙏. I made the changes you requested except #33 (comment)

@jperl
Copy link
Contributor Author

jperl commented Jan 27, 2020

@derhuerst made the deltaBytes change. It should be good to go 👍

install.js Show resolved Hide resolved
install.js Show resolved Hide resolved
@jperl
Copy link
Contributor Author

jperl commented Jan 27, 2020

Good catches. Updated it 👍

@derhuerst derhuerst merged commit 503b7ed into eugeneware:master Jan 27, 2020
@jperl jperl deleted the release-binaries branch January 27, 2020 18:07
@jperl
Copy link
Contributor Author

jperl commented Jan 27, 2020

@derhuerst to wrap this up

  1. create a b* tag to start a release
  2. change the download url from qawolf to eugeneware
  3. update the binary_release in package.json to the tag in step 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🤏Slim down ffmpeg-static split package to reduce install size
2 participants