Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove bundler version in Gemfile.lock #928

Merged
merged 3 commits into from
Nov 5, 2019
Merged

Remove bundler version in Gemfile.lock #928

merged 3 commits into from
Nov 5, 2019

Conversation

schneems
Copy link
Contributor

@schneems schneems commented Oct 4, 2019 •
edited
Loading

Backstory: Bundler introduced the BUNDLED WITH directive into the Gemfile.lock to record the version of bundler that was used to generate the Gemfile.lock. For a long time this information was only a FYI.

In Bundler 2.x originally there were plans for new behavior. Rubygems became "bundler aware" and when you tried to work in a project that had a Gemfile.lock with a BUNDLED WITH then it would automatically try to find that version and use it. For example

BUNDLED WITH
  1.15.2

In this scenario, RubyGems would try to find bundler 1.15.2 and use it even if there was a more recent version available. Otherwise, it would fail.

This is "bounce" behavior was problematic for a few reasons, when the version switching happened it was opaque to the user, it's not guaranteed that everyone on the team and every environment (such as production/CI/etc.) will have that EXACT version of bundler on the system, and the errors were pre-emptive. What I mean by this is that there was no guarantee that a project generated with bundler 1.15.2 could not run with bundler 1.15.3.

This plan for "bounce" behavior was decided to be removed from the bundler codebase. Instead, it was decided to only look at the major version and then find a major version that satisfies these criteria:

BUNDLED WITH
  1.15.2

When executed with bundler 2.x would look on the system for any valid 1.x version of bundler. Such as 1.17.3 or 1.0.0. Any 1.x version would be valid. If multiple are present then the latest is used (which is similar to how rubygems selects bundler to be loaded prior to these changes).

If the Gemfile.lock contains:

BUNDLED WITH
  2.0.0

Then when executed with bundler 2.x then the system looks for any version of bundler 2.x. If it can't find one then it would error.

This is somewhat better behavior than the originally proposed "bounce" because it preserves existing 1.x interactions with rubygems (choosing the latest version of bundler on the platform) and it allows for a wide range of versions to execute the Gemfile.lock (making the "platform" problem i.e. heroku/CI/etc. better). But it still has two flaws: The silent version switching code is still present (though more granular) and the error behavior is still present.

Originally Bundler 2.x was intended to be backwards incompatible with 1.x. Eventually many of these incompatibilities were dropped. To my knowledge, the only one that stayed in was officially dropping support for REALLY old versions of Ruby. Even with this incompatibility if you can install a version of Bundler on your system, then the behavior between 1.x and 2.x should be identical.

What this means is that 100% of the time that bundler raises an error because it could not find a "valid version of bundler" on the platform, it would still be executable with ANY (1.x or 2.x) version of bundler. Granted there are still bugs on individual versions of bundler released, but that's a different story.

So now we've got this error behavior in bundler that is also coupled to RubyGems that is not needed.

Now is where it starts to go bad

Because Rubygems has been integrating with this bundler logic for a long time, the logic still lives inside of older versions of Rubygems. If there's a bug, it's not enough to rev the bundler version you also have to get a newer rubygems version.

All of this code was behind a flag that only triggers when bundler >= 2.0.0 is on a system so it was not widely tested or used until bundler 2.0.0 was officially released. After that time many bugs were found. Bugs continue to be found.

So we're in a situation where there is a matrix of failure modes (bundler versions on one axis, rubygems versions on another axis, and BUNDLED WITH declarations on the third axis). This surface area has exploded the number of cases that have to be handled. Another axis is that with Ruby 2.6 bundler became a default gem that cannot be uninstalled.

Heroku Bundler support

Heroku originally worked with the bundler team to understand behavior of Bundler 2.0 on release and we decided that we would inspect the Gemfile.lock and install an appropriate "blessed" version of bundler 1.x or 2.x based on the version. This would have been a fine approach except older versions of RubyGems still contain bugs built in for Bundler 2.0.

Our version of RubyGems is tightly coupled to the Ruby version. Each version of Ruby ships with a Ruby Gems version. Since bundler 2.0 came out, we've gone so far as to apply backports of patches to older rubygems versions. This is a difficult process and could result in introducing incompatibilities for people's applications that were otherwise running. There are also cases where there are no patches to apply or version updates that will fix issues.

To the Future

This PR deletes the information in the BUNDLED WITH section after a bundler version is detected and installed. This means if you're deploying with 1.17.3 you'll get our blessed version of bundler 1.x. If you're deploying with bundler 2.0.1 you'll get our blessed version of bundler 2.x. After this, we discard the BUNDLED WITH information so that no version of RubyGems or Bundler can use this information against your application to falsely raise an exception.

While this might sound drastic, we are still honoring the intent of the value by using it to inform the bundler version that you get. Also this is not much different than if the feature had been shipped with some kind of an "escape valve" to disable the errors when we are in a context we know should be valid. For example if they had switched this error behavior off when an environment variable was detected BUNDLER_DISABLE_VERSION_ERRORS=1 then we would have enabled that escape valve months ago.

For fun

Twitter poll: https://twitter.com/schneems/status/1179413794583392257

@schneems
Remove bundler version in Gemfile.lock
e07ffff 
https://twitter.com/schneems/status/1179413794583392257
@schneems
# This is a combination of 2 commits.
41ed0b6 
# This is the 1st commit message:

Failing test for Bundler 2 and Ruby 2.5.5

When you deploy an app with Bundler 2.0.1 in the Gemfile.lock with Bundler 2.0.2 on the system, it works the first time, but on the second deploy it fails:

```
  1) Bundler deploys with version 2.x
     Failure/Error: app.push!

     Hatchet::App::FailedDeploy:
       Could not deploy 'hatchet-t-31aafb8954' (default_ruby) using 'Hatchet::GitApp' at path: './repos/rack/default_ruby'
        if this was expected add `allow_failure: true` to your deploy hash.
       output:
       Buildpack: nil
       Repo: https://git.heroku.com/hatchet-t-31aafb8954.git
       remote: Compressing source files... done.
       remote: Building source:
       remote:
       remote: -----> Ruby app detected
       remote: -----> Compiling Ruby/Rack
       remote: -----> Using Ruby version: ruby-2.5.5
       remote: -----> Installing dependencies using bundler 2.0.2
       remote:        Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       remote:        Activating bundler (2.0.1) failed:
       remote:        Could not find 'bundler' (2.0.1) required by your /tmp/build_a9c801af0c0fc42d984cfda6569e532c/Gemfile.lock.
       remote:        To update to the latest version installed on your system, run `bundle update --bundler`.
       remote:        To install the missing version, run `gem install bundler:2.0.1`
       remote:        Checked in 'GEM_PATH=vendor/bundle/ruby/2.5.0', execute `gem env` for more information
       remote:
       remote:        To install the version of bundler this project requires, run `gem install bundler -v '2.0.1'`
       remote:        Bundler Output: Activating bundler (2.0.1) failed:
       remote:        Could not find 'bundler' (2.0.1) required by your /tmp/build_a9c801af0c0fc42d984cfda6569e532c/Gemfile.lock.
       remote:        To update to the latest version installed on your system, run `bundle update --bundler`.
       remote:        To install the missing version, run `gem install bundler:2.0.1`
       remote:        Checked in 'GEM_PATH=vendor/bundle/ruby/2.5.0', execute `gem env` for more information
       remote:
       remote:        To install the version of bundler this project requires, run `gem install bundler -v '2.0.1'`
       remote:
       remote:  !
       remote:  !     Failed to install gems via Bundler.
       remote:  !
       remote:  !     Push rejected, failed to compile Ruby app.
       remote:
       remote:  !     Push failed
       remote: Verifying deploy...
       remote:
       remote: !  Push rejected to hatchet-t-31aafb8954.
       remote:
       To https://git.heroku.com/hatchet-t-31aafb8954.git
        ! [remote rejected] master -> master (pre-receive hook declined)
       error: failed to push some refs to 'https://git.heroku.com/hatchet-t-31aafb8954.git'
```

The first deploy succeeds because there is no cache. The second deploy fails because the cache exists and `which bundler` points to a file generated by this bundler template (in `vendor/bundle/bin/bundle`): https://github.com/bundler/bundler/blob/905dce42705d0e92fa5c74ce4b9133c7d77a6fb1/lib/bundler/templates/Executable.bundler

It fails due to this code which is run:

```
  def activate_bundler(bundler_version)
    if Gem::Version.correct?(bundler_version) && Gem::Version.new(bundler_version).release < Gem::Version.new("2.0")
      bundler_version = "< 2"
    end
    gem_error = activation_error_handling do
      gem "bundler", bundler_version
    end
```

The `bundler_version` here will be 2.0.1 but it sees that it's own version is 2.0.2.

This can potentially be mitigated by manually setting `BUNDLER_VERSION=2.0.2`, however I'm not in love with that solution as it doesn't seem like it should be required.

Another solution is to not call this `vendor/bundle/bin/bundle` executable directly and instead go through a shim such as:

```
#!/usr/bin/env ruby
require 'rubygems'

version = "#{bundler.version}"

if ARGV.first
  str = ARGV.first
  str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
  if str =~ /\A_(.*)_\z/ and Gem::Version.correct?($1) then
    version = $1
    ARGV.shift
  end
end

if Gem.respond_to?(:activate_bin_path)
load Gem.activate_bin_path('bundler', 'bundle', version)
else
gem "bundler", version
load Gem.bin_path("bundler", "bundle", version)
end
```

This is similar (if not identical) to the shim that rubygems would install and use on a "normal" system. On Heroku we do not use this system since we do not actually install bundler through rubygems, but rather we pre-package it as a tarball, put it on S3 and then download it. We invoke bundler, not through rubygems, but by placing it's executable on the PATH and then letting the OS find it directly.

There are two fixes listed here:

- Use BUNDLE_VERSION env var
  - Pros: Fixes the problem. A relatively small change.
  - Cons: Requires us to propagate this env var into the dyno and/or remember to set it any time we invoke bundler. This will be very hard to catch incorrect uses as failure to include it does not always make things fail right away, but they may fail in the future. There's also no guarantee that this will continue to work into the future. I.e. this might be a fix for today
- Write our own shim file and pretend to be rubygems
  - Pros: Fixes the problem
  - Cons: We're effectively forking Rubygems shim logic with this method, any bugfixes or upstream changes to Rubygems would need to be mirrored in the buildpack, not great. Possibly brittle. A large change.

I'm interested in looking for a third solution, one where Heroku will behave more like a "normal" install of Ruby w/ Bundler. Ideally we would find a way for all `bundle` calls when the client's codebase does not include a `bin/bundle` binstub, to go through whatever version of Rubygems is on the system for all invocations of `bundle` as this is how it will eventually be executed on the Dyno.
- Pros: Fixes the problem. No need to worry about following upstream fixes or changes to Rubygems shim logic.
- Cons: There might be additional Rubygems bugs that we're successfully avoiding today due to our current (accidental?) logic. This would be the largest change.

# This is the commit message #2:

Update to 2.5.7
@schneems
Changelog for deleting BUNDLED WITH
21bb1a9
@schneems schneems merged commit 1a30c5a into master Nov 5, 2019
@schneems schneems deleted the schneems/bust-bundler branch November 5, 2019 18:48
@schneems schneems mentioned this pull request Nov 5, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danielleadams danielleadams danielleadams approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@schneems @danielleadams