bundle install should force installation of yanked gems in the Gemfile.lock

[sferik]

Let me know if there's anything you need from me on the RubyGems.org side. I'll have some free time on Sunday to hack on this.

[indirect]

This needs:

• Bundler to add a command=install or command=update to dependency-api requests (fetcher.rb)
• rubygems/bundler-api to to include yanked gems in the SQL query if the command is install
• There is no step three

[nathany]

I ran into issues while updating some gems today, so here are some use cases for the proposed solution.

New Relic released a gem update that they then yanked, 3.5.6.46. This installed locally with a bundle update (bundler 1.2.3). I'm not sure when it was yanked, but I had some successful pushes to Heroku today (bundler 1.3.0.pre.5) with this gem. On one project I ran into an error: Could not find newrelic_rpm-3.5.6.46 in any of the sources. Noticing it was yanked, I went and reverted all my projects to the last stable release. To do this, I had to specify '3.5.5.38' in my Gemfile, otherwise Bundler (1.2.3) still held on to the yanked version.

I also did an update from fog 1.8 to 1.9, with bundle update fog. It specifies a dependency on net-scp ~> 1.0.4, but bundler (1.2.3) brought in the yanked 1.0.6 gem. Again, I don't know when it was yanked, but these gems were released several days ago. That resulted in the same sort of deploy error (Heroku using bundler 1.3.0.pre.5), and having to add net-scp to my Gemfile so that I could force it back to 1.0.4. Presumably these versions of net-scp were yanked because they introduced breaking changes that warranted a 1.1.0 release?

The proposed solution would allow the yanked versions to be deployed. Since whatever test suite/QA process was passed with those versions, it seems like an okay idea (though newrelic_rpm wouldn't be covered by my test suite).

Obviously the current handling of yanked versions leaves something to be desired, and the problem is further exacerbated by my using of a different version of bundler than Heroku. Something needs to be done to make this work better, but I still can't say I'm sold on the idea of installing yanked gems based on Gemfile.lock, at least not as the default scenario. Presumably they were yanked for a reason.

What about replacing Could not find newrelic_rpm-3.5.6.46 in any of the sources with The newrelic_rpm-3.5.6.46 gem was yanked. Use -f to force install.?

Personally, I don't want bundler installing yanked gems/dependencies in the first place. If a gem is yanked after added to my Gemfile.lock, I want an easy way to revert back without having to add additional dependencies (eg. net-scp) to my Gemfile.

[xaviershay]

This is still relevant.

[hone]

I can take this on, but it won't make it until post 1.6.

[myronmarston]

I just opened an issue for rubygems about this:

rubygems/rubygems#858

Didn't realize there was already an issue here (thanks for directing me to this, @sferik!).

If it's easier I can close that issue and bring my suggestions over here.

[sferik]

IMHO, the deployment problem is better addressed by Bundler than RubyGems. Even if RubyGems “fixes” it, it would still need to be addressed in Bundler.

[myronmarston]

Sure, bundler definitely needs to solve the problem with Gemfile.lock gems being yanked. I still think that being able to differentiate between "this gem is dangerous to install" and "this release is buggy" is useful (particularly with a reason attached) is useful. But my proposed solution involves a lot more coordination among teams and would take more effort and time to implement, so I can understand going with this simpler solution.

[indirect]

My personal opinion is that yanking should be removed, and 'dangerous to install' gems get removed completely, while buggy gems simply get newer, higher versions.

In the meantime, I think that installing locked gems even if they were yanked after the bundle was locked is the best thing we can do.

On Fri, Mar 21, 2014 at 11:25 AM, Myron Marston notifications@github.com
wrote:

Sure, bundler definitely needs to solve the problem with Gemfile.lock gems being yanked. I still think that being able to differentiate between "this gem is dangerous to install" and "this release is buggy" is useful (particularly with a reason attached) is useful. But my proposed solution involves a lot more coordination among teams and would take more effort and time to implement, so I can understand going with this simpler solution.

Reply to this email directly or view it on GitHub:
#2277 (comment)

[nathany]

@indirect Thats a good point. A gem maintainer can always revert some commits and release a higher version instead of yanking.

Or failing that, end-users can lock a previous version in a Gemfile (I imagine yank predates bundler?).

[sferik]

@indirect It would be great if maintainers used the current yanking functionality as you’ve percribed but there are three problems:

1. It’s not always possible to immediately release a bug fix. Some bugs take several days to investigate. In the mean time, the maintainer(s) should have the ability to stop the spread of the bug. This is especially true for new major versions, where shipping an earlier version of the gem with a higher number is not a good option (e.g. if there is a serious bug in version 2.0.0, it doesn’t make sense to re-release 1.9.0 as 2.0.1).
2. Yanking serves various other purposes (e.g. surrendering a squatted gem name). Removing it would create a bigger maintenance burden for RubyGems.org volunteers, who already have enough trouble keeping up with the support queue.
3. 

[sferik]

As of http://blog.rubygems.org/2015/04/13/permadelete-on-yank.html, this feature is no longer possible.