refactor pip.req_set.prepare_files

[qwcode]

opening an issue to state the obvious, that pip.req_set.prepare_files is too large and complicated. It needs to be redone.

It's full of hairy conditonal logic that's based on:

1. whether it's sdist vs wheel vs bundle vs editable
2. whether it's a download vs an install
3. whether it's an upgrade

for #1, I think the answer is to subclass InstallRequirement (for sdist/wheel/editable), or maybe create our own hierarchy of Distribution classes, and move the logic to those classes. bundle is going away soon, so that will help simplify as well.

for #2, I think clearly separating "unpacking" vs "dependency resolution" vs "installing" vs "downloading" would help a lot. download and install both need unpacking and resolution, as independent steps. Note that in the future, we won't need unpacking when using PyPI (but will still need dependency resolution), due to PyPI likely providing metadata outside of the archive.

[rbtcollins]

Having been poking around heavily at this, I thought I'd capture a couple of thoughts here.

InstallRequirement is stateful - it builds up state, and we can't tell until we've done a bunch of work with it whether we have an sdist/wheel/editable/$newshiny. That makes using InstallRequirement as a subclasssable thing hard. I think the variation along the sdist/wheel/editable etc dimension should be a separate set of classes.

[dstufft]

Closing this, I don't think that we're going to forget that this code is horrible and needs to die.