[New custom build] Implement the new custom build command

[tomaka]

Does include:

• The build script is compiled in target/build/$pkg, its output is written in the same directory in the output file, and it has target/build/$pkg/out available to write temporary files
• The output files are later loaded by each rustc invocation and other custom build command, in order to retreive the rustc-flags or DEP_ variables.
• Internal rename from plugin to for_host
• A different system for passing command descriptions to the shell (internal change)
• Deprecation of the old custom build script system with a warning if it is being used

Does not include:

• Support for links, I plan to add this after this PR is merged
• DEP_<links>_<key>: You just need to uncomment a block of code, but this can't be implemented without support for links
• Build dependencies, I plan to add this after this PR is merged ; this probably conflicts a lot with [New custom build] Implement platform-specific dependencies #735, if I implement this now, I'll need to rewrite half of the code when rebasing
• Tests to check that rustc-flags are correctly passed to rustc, because rustc doesn't support the -l flag yet. I'll add this to the PR once rust supports it (or maybe even before if I compile the fork of rustc that has it).
• The test supposed to check whether a custom build script failure triggers a compilation failure for the whole project is commented out, because I didn't find a way to get the name of the package to match the exact error message. This will need to be fixed before merging.
• There's no feedback of the new system in the Compilation struct
• The build directory is not used when determining the fingerprint, something that was apparently done for the native directory

I recommend reviewing commit by commit. I implemented each feature one by one (tests should pass for each individual commit).

cc #610

[alexcrichton]

s/sender/tx/

[alexcrichton]

Ok, I've looked this over and added some comments here and there, thanks @tomaka!

Could you also add some tests along these lines:

• build commands and cargo {test,run,bench,doc}
• propagating of -L throughout the entire dependency graph when one package requests it
• If I change a file in a package with a build command the build command is re-run

The list of things you plan on adding after this gets merged looks great, I hope to merge this soon! Some specific comments:

There's no feedback of the new system in the Compilation struct

This is currently mainly used for feeding information into cargo {test,bench,run} which uses the output of compilation to actually run some commands. I don't think we'll need to modify them much, but you'll need to be sure that any -L paths are indeed included in LD_LIBRARY_PATH.

The build directory is not used when determining the fingerprint, something that was apparently done for the native directory

Could you elaborate a little more on this?

[alexcrichton]

Ah, and one other recommendation for tests: Could you pass -v wherever possible and match the entire output of all commands? I'd just want to make sure that build commands are printed as being run wherever possible.

[tomaka]

Could you elaborate a little more on this?

I re-read Cargo's source code, and apparently I was mistaken. When I modified Layout, I noticed several references to native() in fingerprint.rs, but on a second look it's just here to fill the Compilation struct.

[alexcrichton]

Ok, sounds good to me!

[tomaka]

For the record, here's what remains to be done before merging:

• Extract code in cargo_rustc::compile to its own function
• Add tests for build commands and cargo {test,run,bench,doc}, don't forget the -v flag
• Add test for "If I change a file in a package with a build command the build command is re-run"
• Add tests for -l and -L flags being passed correctly (including -L between dependencies)

[tomaka]

Closing in favor of #792