Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement infrastructure for the souped-up build command #792

Merged
merged 28 commits into from
Nov 5, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
e3f60a7
Extract custom build tests and prefix with "old_"
tomaka Oct 22, 2014
82e65c3
Deprecate old build system and add rust build script as target
tomaka Oct 22, 2014
019d515
Custom build commands are now being run
tomaka Oct 22, 2014
34ace11
Build command output now being checked and written to filesystem
tomaka Oct 27, 2014
f888b4b
Flags from custom build script are now used
tomaka Oct 27, 2014
15f1040
Jobs now send back their own description when they start
tomaka Oct 27, 2014
5ae2846
Add draft for passing metadata to dependencies
tomaka Oct 27, 2014
9cca8c5
Deprecated a few more things
tomaka Oct 27, 2014
f4790de
Address minor issues
tomaka Oct 28, 2014
d712fdd
Move custom-build-related code in its own module
tomaka Oct 28, 2014
d9066f6
Tweak tests that check whether the build script is being run
tomaka Oct 29, 2014
87ad442
Implement the `links` manifest key
alexcrichton Oct 31, 2014
4358288
Fix lines_match test for matching process output
alexcrichton Oct 31, 2014
6391536
Implement overrides via local cargo configuration
alexcrichton Oct 31, 2014
a0d499e
Refine the dependency graph for build scripts
alexcrichton Oct 31, 2014
3b21f7a
Ensure fresh build commands populate build info
alexcrichton Oct 31, 2014
cde3b1e
Add tests for other cargo cmds + -L propagation
alexcrichton Nov 1, 2014
8960dd1
Implement build-dependencies
alexcrichton Nov 1, 2014
bf30c8c
Add a test for build commands with build commands
alexcrichton Nov 1, 2014
a5c868a
Really fix `cargo test` and fix an OUT_DIR bug
alexcrichton Nov 1, 2014
f62d6c6
Update documentation for new build command system
tomaka Oct 23, 2014
2a17e9e
Beef up documentation about build scripts
alexcrichton Nov 3, 2014
800d9eb
Fix the build command passing -L/-l without links
alexcrichton Nov 4, 2014
92b07cb
Dox fixups
alexcrichton Nov 4, 2014
2833358
Fix some flaky tests
alexcrichton Nov 4, 2014
85a96e4
Follow through on the OUT_DIR promise for rustc compiles
alexcrichton Nov 4, 2014
c05442a
Expand the "link to system libraries" section
alexcrichton Nov 4, 2014
5e29a8b
Fix custom build tests on windows
alexcrichton Nov 5, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ clean:

# === Documentation

DOCS := index faq config guide manifest native-build pkgid-spec
DOCS := index faq config guide manifest build-script pkgid-spec
DOC_DIR := target/doc
DOC_OPTS := --markdown-no-toc \
--markdown-css stylesheets/normalize.css \
Expand Down
25 changes: 20 additions & 5 deletions src/cargo/core/dependency.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ pub struct Dependency {
source_id: SourceId,
req: VersionReq,
specified_req: Option<String>,
transitive: bool,
kind: Kind,
only_match_name: bool,

optional: bool,
Expand All @@ -22,6 +22,13 @@ pub struct Dependency {
only_for_platform: Option<String>,
}

#[deriving(PartialEq, Clone, Show)]
pub enum Kind {
Normal,
Development,
Build,
}

impl Dependency {
/// Attempt to create a `Dependency` from an entry in the manifest.
///
Expand Down Expand Up @@ -55,7 +62,7 @@ impl Dependency {
name: name.to_string(),
source_id: source_id.clone(),
req: VersionReq::any(),
transitive: true,
kind: Normal,
only_match_name: true,
optional: false,
features: Vec::new(),
Expand Down Expand Up @@ -83,8 +90,8 @@ impl Dependency {
&self.source_id
}

pub fn transitive(mut self, transitive: bool) -> Dependency {
self.transitive = transitive;
pub fn kind(mut self, kind: Kind) -> Dependency {
self.kind = kind;
self
}

Expand Down Expand Up @@ -132,7 +139,15 @@ impl Dependency {
}

/// Returns false if the dependency is only used to build the local package.
pub fn is_transitive(&self) -> bool { self.transitive }
pub fn is_transitive(&self) -> bool {
match self.kind {
Normal | Build => true,
Development => false,
}
}
pub fn is_build(&self) -> bool {
match self.kind { Build => true, _ => false }
}
pub fn is_optional(&self) -> bool { self.optional }
/// Returns true if the default features of the dependency are requested.
pub fn uses_default_features(&self) -> bool { self.default_features }
Expand Down
59 changes: 47 additions & 12 deletions src/cargo/core/manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ pub struct Manifest {
targets: Vec<Target>,
target_dir: Path,
doc_dir: Path,
build: Vec<String>,
build: Vec<String>, // TODO: deprecated, remove
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do the stability markers not work here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now these are actually just markers of what to remove once we remove the old system of build cmd, not necessarily for deprecated functionality itself.

links: Option<String>,
warnings: Vec<String>,
exclude: Vec<String>,
metadata: ManifestMetadata,
Expand Down Expand Up @@ -59,7 +60,7 @@ pub struct SerializedManifest {
targets: Vec<Target>,
target_dir: String,
doc_dir: String,
build: Option<Vec<String>>,
build: Option<Vec<String>>, // TODO: deprecated, remove
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

}

impl<E, S: Encoder<E>> Encodable<S, E> for Manifest {
Expand All @@ -73,6 +74,7 @@ impl<E, S: Encoder<E>> Encodable<S, E> for Manifest {
targets: self.targets.clone(),
target_dir: self.target_dir.display().to_string(),
doc_dir: self.doc_dir.display().to_string(),
// TODO: deprecated, remove
build: if self.build.len() == 0 { None } else { Some(self.build.clone()) },
}.encode(s)
}
Expand Down Expand Up @@ -131,8 +133,9 @@ pub struct Profile {
doctest: bool,
doc: bool,
dest: Option<String>,
plugin: bool,
for_host: bool,
harness: bool, // whether to use the test harness (--test)
custom_build: bool,
}

impl Profile {
Expand All @@ -146,8 +149,9 @@ impl Profile {
test: false,
doc: false,
dest: None,
plugin: false,
for_host: false,
doctest: false,
custom_build: false,
harness: true,
}
}
Expand Down Expand Up @@ -219,8 +223,13 @@ impl Profile {
self.doctest
}

pub fn is_plugin(&self) -> bool {
self.plugin
pub fn is_custom_build(&self) -> bool {
self.custom_build
}

/// Returns true if the target must be built for the host instead of the target.
pub fn is_for_host(&self) -> bool {
self.for_host
}

pub fn get_opt_level(&self) -> uint {
Expand Down Expand Up @@ -282,15 +291,22 @@ impl Profile {
self
}

pub fn plugin(mut self, plugin: bool) -> Profile {
self.plugin = plugin;
/// Sets whether the `Target` must be compiled for the host instead of the target platform.
pub fn for_host(mut self, for_host: bool) -> Profile {
self.for_host = for_host;
self
}

pub fn harness(mut self, harness: bool) -> Profile {
self.harness = harness;
self
}

/// Sets whether the `Target` is a custom build script.
pub fn custom_build(mut self, custom_build: bool) -> Profile {
self.custom_build = custom_build;
self
}
}

impl<H: hash::Writer> hash::Hash<H> for Profile {
Expand All @@ -302,7 +318,7 @@ impl<H: hash::Writer> hash::Hash<H> for Profile {
codegen_units,
debug,
rpath,
plugin,
for_host,
ref dest,
harness,

Expand All @@ -313,8 +329,10 @@ impl<H: hash::Writer> hash::Hash<H> for Profile {
env: _,
test: _,
doctest: _,

custom_build: _,
} = *self;
(opt_level, codegen_units, debug, rpath, plugin, dest, harness).hash(into)
(opt_level, codegen_units, debug, rpath, for_host, dest, harness).hash(into)
}
}

Expand Down Expand Up @@ -366,16 +384,17 @@ impl Show for Target {
impl Manifest {
pub fn new(summary: Summary, targets: Vec<Target>,
target_dir: Path, doc_dir: Path,
build: Vec<String>, exclude: Vec<String>,
build: Vec<String>, exclude: Vec<String>, links: Option<String>,
metadata: ManifestMetadata) -> Manifest {
Manifest {
summary: summary,
targets: targets,
target_dir: target_dir,
doc_dir: doc_dir,
build: build,
build: build, // TODO: deprecated, remove
warnings: Vec::new(),
exclude: exclude,
links: links,
metadata: metadata,
}
}
Expand Down Expand Up @@ -416,6 +435,10 @@ impl Manifest {
self.build.as_slice()
}

pub fn get_links(&self) -> Option<&str> {
self.links.as_ref().map(|s| s.as_slice())
}

pub fn add_warning(&mut self, s: String) {
self.warnings.push(s)
}
Expand Down Expand Up @@ -466,6 +489,18 @@ impl Target {
}
}

/// Builds a `Target` corresponding to the `build = "build.rs"` entry.
pub fn custom_build_target(name: &str, src_path: &Path, profile: &Profile,
metadata: Option<Metadata>) -> Target {
Target {
kind: BinTarget,
name: name.to_string(),
src_path: src_path.clone(),
profile: profile.clone(),
metadata: metadata,
}
}

pub fn example_target(name: &str, src_path: &Path, profile: &Profile) -> Target {
Target {
kind: ExampleTarget,
Expand Down
3 changes: 2 additions & 1 deletion src/cargo/ops/cargo_clean.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use std::collections::HashMap;
use std::io::fs::{mod, PathExtensions};

use core::{MultiShell, PackageSet};
Expand Down Expand Up @@ -49,7 +50,7 @@ pub fn clean(manifest_path: &Path, opts: &mut CleanOptions) -> CargoResult<()> {
let pkgs = PackageSet::new([]);
let cx = try!(Context::new("compile", &resolve, &srcs, &pkgs, &mut cfg,
Layout::at(root.get_absolute_target_dir()),
None, &pkg));
None, &pkg, HashMap::new()));

// And finally, clean everything out!
for target in pkg.get_targets().iter() {
Expand Down
89 changes: 58 additions & 31 deletions src/cargo/ops/cargo_compile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ use std::collections::HashMap;
use core::registry::PackageRegistry;
use core::{MultiShell, Source, SourceId, PackageSet, Package, Target, PackageId};
use core::resolver;
use ops;
use ops::{mod, BuildOutput};
use sources::{PathSource};
use util::config::{Config, ConfigValue};
use util::{CargoResult, Wrap, config, internal, human, ChainError, profile};
Expand Down Expand Up @@ -129,7 +129,7 @@ pub fn compile_pkg(package: &Package, options: &mut CompileOptions)
};

let targets = to_build.get_targets().iter().filter(|target| {
match env {
target.get_profile().is_custom_build() || match env {
// doc-all == document everything, so look for doc targets
"doc" | "doc-all" => target.get_profile().get_env() == "doc",
env => target.get_profile().get_env() == env,
Expand All @@ -138,12 +138,12 @@ pub fn compile_pkg(package: &Package, options: &mut CompileOptions)

let ret = {
let _p = profile::start("compiling");
try!(scrape_target_config(&config, &user_configs));
let lib_overrides = try!(scrape_target_config(&config, &user_configs));

try!(ops::compile_targets(env.as_slice(), targets.as_slice(), to_build,
&PackageSet::new(packages.as_slice()),
&resolve_with_overrides, &sources,
&config))
&config, lib_overrides))
};

return Ok(ret);
Expand Down Expand Up @@ -175,41 +175,68 @@ fn source_ids_from_config(configs: &HashMap<String, config::ConfigValue>,

fn scrape_target_config(config: &Config,
configs: &HashMap<String, config::ConfigValue>)
-> CargoResult<()> {
-> CargoResult<HashMap<String, BuildOutput>> {
let target = match configs.find_equiv("target") {
None => return Ok(()),
None => return Ok(HashMap::new()),
Some(target) => try!(target.table().chain_error(|| {
internal("invalid configuration for the key `target`")
})),
};
let target = match config.target() {
None => target,
Some(triple) => match target.find_equiv(triple) {
None => return Ok(()),
Some(target) => try!(target.table().chain_error(|| {
internal(format!("invalid configuration for the key \
`target.{}`", triple))
})),
},
let triple = config.target().unwrap_or(config.rustc_host()).to_string();
let target = match target.find(&triple) {
None => return Ok(HashMap::new()),
Some(target) => try!(target.table().chain_error(|| {
internal(format!("invalid configuration for the key \
`target.{}`", triple))
})),
};

match target.find_equiv("ar") {
None => {}
Some(ar) => {
config.set_ar(try!(ar.string().chain_error(|| {
internal("invalid configuration for key `ar`")
})).ref0().to_string());
}
}

match target.find_equiv("linker") {
None => {}
Some(linker) => {
config.set_linker(try!(linker.string().chain_error(|| {
internal("invalid configuration for key `ar`")
})).ref0().to_string());
let mut ret = HashMap::new();
for (k, v) in target.iter() {
match k.as_slice() {
"ar" | "linker" => {
let v = try!(v.string().chain_error(|| {
internal(format!("invalid configuration for key `{}`", k))
})).ref0().to_string();
if k.as_slice() == "linker" {
config.set_linker(v);
} else {
config.set_ar(v);
}
}
lib_name => {
let table = try!(v.table().chain_error(|| {
internal(format!("invalid configuration for the key \
`target.{}.{}`", triple, lib_name))
}));
let mut output = BuildOutput {
library_paths: Vec::new(),
library_links: Vec::new(),
metadata: Vec::new(),
};
for (k, v) in table.iter() {
let v = try!(v.string().chain_error(|| {
internal(format!("invalid configuration for the key \
`target.{}.{}.{}`", triple, lib_name,
k))
})).val0();
if k.as_slice() == "rustc-flags" {
let whence = format!("in `target.{}.{}.rustc-flags`",
triple, lib_name);
let whence = whence.as_slice();
let (paths, links) = try!(
BuildOutput::parse_rustc_flags(v.as_slice(), whence)
);
output.library_paths.extend(paths.into_iter());
output.library_links.extend(links.into_iter());
} else {
output.metadata.push((k.to_string(), v.to_string()));
}
}
ret.insert(lib_name.to_string(), output);
}
}
}

Ok(())
Ok(ret)
}
3 changes: 2 additions & 1 deletion src/cargo/ops/cargo_run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ pub fn run(manifest_path: &Path,
LibTarget(_) => false,
};
let matches_name = name.as_ref().map_or(true, |n| n.as_slice() == a.get_name());
matches_kind && matches_name && a.get_profile().get_env() == env
matches_kind && matches_name && a.get_profile().get_env() == env &&
!a.get_profile().is_custom_build()
});
let bin = try!(bins.next().require(|| {
human("a bin target must be available for `cargo run`")
Expand Down
3 changes: 2 additions & 1 deletion src/cargo/ops/cargo_rustc/compilation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ pub struct Compilation {
///
/// This is currently used to drive some entries which are added to the
/// LD_LIBRARY_PATH as appropriate.
// TODO: deprecated, remove
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this TODO correct, @alexcrichton? I.e. should I PR the removal of Compilation.native_dirs, or PR the removal of the TODO?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh dear that's a good question, this PR is... quite old though! I vaguely remember placing this on a few things throughout Cargo I've wished I deleted at the time and then never got to, but removing this would be good if it's not actually used in Cargo today! (or if it's actually used just remove the comment)

pub native_dirs: HashMap<PackageId, Path>,

/// Root output directory (for the local package's artifacts)
Expand All @@ -43,7 +44,7 @@ impl Compilation {
pub fn new(pkg: &Package) -> Compilation {
Compilation {
libraries: HashMap::new(),
native_dirs: HashMap::new(),
native_dirs: HashMap::new(), // TODO: deprecated, remove
root_output: Path::new("/"),
deps_output: Path::new("/"),
tests: Vec::new(),
Expand Down
Loading