Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Layout docs and cleanup. #7227

Merged
merged 1 commit into from
Aug 8, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions src/cargo/core/compiler/build_context/target_info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,17 @@ pub struct FileType {
/// The kind of file.
pub flavor: FileFlavor,
/// The suffix for the file (for example, `.rlib`).
/// This is an empty string for executables on Unix-like platforms.
suffix: String,
/// The prefix for the file (for example, `lib`).
/// This is an empty string for things like executables.
prefix: String,
// Wasm bin target will generate two files in deps such as
// "web-stuff.js" and "web_stuff.wasm". Note the different usages of
// "-" and "_". should_replace_hyphens is a flag to indicate that
// we need to convert the stem "web-stuff" to "web_stuff", so we
// won't miss "web_stuff.wasm".
/// Flag to convert hyphen to underscore.
///
/// wasm bin targets will generate two files in deps such as
/// "web-stuff.js" and "web_stuff.wasm". Note the different usages of "-"
/// and "_". This flag indicates that the stem "web-stuff" should be
/// converted to "web_stuff".
should_replace_hyphens: bool,
}

Expand Down
1 change: 1 addition & 0 deletions src/cargo/core/compiler/compilation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ pub struct Doctest {
/// A structure returning the result of a compilation.
pub struct Compilation<'cfg> {
/// An array of all tests created during this compilation.
/// `(package, target, path_to_test_exe)`
pub tests: Vec<(Package, Target, PathBuf)>,

/// An array of all binaries created.
Expand Down
11 changes: 3 additions & 8 deletions src/cargo/core/compiler/context/compilation_files.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ impl<'a, 'cfg: 'a> CompilationFiles<'a, 'cfg> {
/// target.
pub fn out_dir(&self, unit: &Unit<'a>) -> PathBuf {
if unit.mode.is_doc() {
self.layout(unit.kind).root().parent().unwrap().join("doc")
self.layout(unit.kind).doc().to_path_buf()
} else if unit.mode.is_doc_test() {
panic!("doc tests do not have an out dir");
} else if unit.target.is_custom_build() {
Expand All @@ -169,11 +169,6 @@ impl<'a, 'cfg: 'a> CompilationFiles<'a, 'cfg> {
}
}

/// Returns the root of the build output tree for the target
pub fn target_root(&self) -> &Path {
self.target.as_ref().unwrap_or(&self.host).dest()
}

/// Returns the root of the build output tree for the host
pub fn host_root(&self) -> &Path {
self.host.dest()
Expand Down Expand Up @@ -261,8 +256,8 @@ impl<'a, 'cfg: 'a> CompilationFiles<'a, 'cfg> {
/// (eg a dependent lib).
fn link_stem(&self, unit: &Unit<'a>) -> Option<(PathBuf, String)> {
let out_dir = self.out_dir(unit);
let bin_stem = self.bin_stem(unit);
let file_stem = self.file_stem(unit);
let bin_stem = self.bin_stem(unit); // Stem without metadata.
let file_stem = self.file_stem(unit); // Stem with metadata.

// We currently only lift files up from the `deps` directory. If
// it was compiled into something like `example/` or `doc/` then
Expand Down
172 changes: 111 additions & 61 deletions src/cargo/core/compiler/layout.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,71 +8,123 @@
//! # places all of its output here.
//! target/
//!
//! # This is the root directory for all output of *dependencies*
//! deps/
//! # Cache of `rustc -Vv` output for performance.
//! .rustc-info.json
//!
//! # Root directory for all compiled examples
//! examples/
//! # All final artifacts are linked into this directory from `deps`.
//! debug/ # or release/
//!
//! # File used to lock the directory to prevent multiple cargo processes
//! # from using it at the same time.
//! .cargo-lock
//!
//! # Hidden directory that holds all of the fingerprint files for all
//! # packages
//! .fingerprint/
//! # Each package is in a separate directory.
//! $pkgname-$META/
//! # Set of source filenames for this package.
//! dep-lib-$pkgname-$META
//! # Timestamp when this package was last built.
//! invoked.timestamp
//! # The fingerprint hash.
//! lib-$pkgname-$META
//! # Detailed information used for logging the reason why
//! # something is being recompiled.
//! lib-$pkgname-$META.json
//!
//! # This is the root directory for all rustc artifacts except build
//! # scripts, examples, and test and bench executables. Almost every
//! # artifact should have a metadata hash added to its filename to
//! # prevent collisions. One notable exception is dynamic libraries.
//! deps/
//!
//! # Root directory for all compiled examples.
//! examples/
//!
//! # Directory used to store incremental data for the compiler (when
//! # incremental is enabled.
//! incremental/
//!
//! # This is the location at which the output of all custom build
//! # commands are rooted
//! # commands are rooted.
//! build/
//!
//! # Each package gets its own directory where its build script and
//! # script output are placed
//! $pkg1/
//! $pkg2/
//! $pkg3/
//! $pkgname-$META/ # For the build script itself.
//! # The build script executable (name may be changed by user).
//! build-script-build-$META
//! # Hard link to build-script-build-$META.
//! build-script-build
//! # Dependency information generated by rustc.
//! build-script-build-$META.d
//! # Debug information, depending on platform and profile
//! # settings.
//! <debug symbols>
//!
//! # Each directory package has a `out` directory where output
//! # is placed.
//! # The package shows up twice with two different metadata hashes.
//! $pkgname-$META/ # For the output of the build script.
//! # Timestamp when the build script was last executed.
//! invoked.timestamp
//! # Directory where script can output files ($OUT_DIR).
//! out/
//! # Output from the build script.
//! output
//! # Path to `out`, used to help when the target directory is
//! # moved.
//! root-output
//! # Stderr output from the build script.
//! stderr
//!
//! # Output from rustdoc
//! doc/
//!
//! # This is the location at which the output of all old custom build
//! # commands are rooted
//! native/
//!
//! # Each package gets its own directory for where its output is
//! # placed. We can't track exactly what's getting put in here, so
//! # we just assume that all relevant output is in these
//! # directories.
//! $pkg1/
//! $pkg2/
//! $pkg3/
//!
//! # Directory used to store incremental data for the compiler (when
//! # incremental is enabled.
//! incremental/
//!
//! # Hidden directory that holds all of the fingerprint files for all
//! # packages
//! .fingerprint/
//! # Used by `cargo package` and `cargo publish` to build a `.crate` file.
//! package/
//!
//! # Experimental feature for generated build scripts.
//! .metabuild/
//! ```
//!
//! When cross-compiling, the layout is the same, except it appears in
//! `target/$TRIPLE`.

use std::fs;
use std::io;
use std::path::{Path, PathBuf};

use crate::core::Workspace;
use crate::util::{CargoResult, Config, FileLock, Filesystem};
use crate::util::{CargoResult, FileLock};

/// Contains the paths of all target output locations.
///
/// See module docs for more information.
pub struct Layout {
/// The root directory: `/path/to/target`.
/// If cross compiling: `/path/to/target/$TRIPLE`.
root: PathBuf,
/// The final artifact destination: `$root/debug` (or `release`).
dest: PathBuf,
/// The directory with rustc artifacts: `$dest/deps`
deps: PathBuf,
native: PathBuf,
/// The directory for build scripts: `$dest/build`
build: PathBuf,
/// The directory for incremental files: `$dest/incremental`
incremental: PathBuf,
/// The directory for fingerprints: `$dest/.fingerprint`
fingerprint: PathBuf,
/// The directory for examples: `$dest/examples`
examples: PathBuf,
/// The lock file for a build, will be unlocked when this struct is `drop`ped.
/// The directory for rustdoc output: `$root/doc`
doc: PathBuf,
/// The lockfile for a build (`.cargo-lock`). Will be unlocked when this
/// struct is `drop`ped.
_lock: FileLock,
}

pub fn is_bad_artifact_name(name: &str) -> bool {
["deps", "examples", "build", "native", "incremental"]
["deps", "examples", "build", "incremental"]
.iter()
.any(|&reserved| reserved == name)
}
Expand All @@ -82,63 +134,57 @@ impl Layout {
///
/// This function will block if the directory is already locked.
///
/// Differs from `at` in that this calculates the root path from the workspace target directory,
/// adding the target triple and the profile (debug, release, ...).
/// `dest` should be the final artifact directory name. Currently either
/// "debug" or "release".
pub fn new(ws: &Workspace<'_>, triple: Option<&str>, dest: &str) -> CargoResult<Layout> {
let mut path = ws.target_dir();
let mut root = ws.target_dir();
// Flexible target specifications often point at json files, so interpret
// the target triple as a Path and then just use the file stem as the
// component for the directory name in that case.
if let Some(triple) = triple {
let triple = Path::new(triple);
if triple.extension().and_then(|s| s.to_str()) == Some("json") {
path.push(
root.push(
triple
.file_stem()
.ok_or_else(|| failure::format_err!("invalid target"))?,
);
} else {
path.push(triple);
root.push(triple);
}
}
path.push(dest);
Layout::at(ws.config(), path)
}

/// Calculate the paths for build output, lock the build directory, and return as a Layout.
///
/// This function will block if the directory is already locked.
pub fn at(config: &Config, root: Filesystem) -> CargoResult<Layout> {
let dest = root.join(dest);
// If the root directory doesn't already exist go ahead and create it
// here. Use this opportunity to exclude it from backups as well if the
// system supports it since this is a freshly created folder.
if !root.as_path_unlocked().exists() {
root.create_dir()?;
exclude_from_backups(root.as_path_unlocked());
if !dest.as_path_unlocked().exists() {
dest.create_dir()?;
exclude_from_backups(dest.as_path_unlocked());
}

// For now we don't do any more finer-grained locking on the artifact
// directory, so just lock the entire thing for the duration of this
// compile.
let lock = root.open_rw(".cargo-lock", config, "build directory")?;
let lock = dest.open_rw(".cargo-lock", ws.config(), "build directory")?;
let root = root.into_path_unlocked();
let dest = dest.into_path_unlocked();

Ok(Layout {
deps: root.join("deps"),
native: root.join("native"),
build: root.join("build"),
incremental: root.join("incremental"),
fingerprint: root.join(".fingerprint"),
examples: root.join("examples"),
deps: dest.join("deps"),
build: dest.join("build"),
incremental: dest.join("incremental"),
fingerprint: dest.join(".fingerprint"),
examples: dest.join("examples"),
doc: root.join("doc"),
root,
dest,
_lock: lock,
})
}

/// Makes sure all directories stored in the Layout exist on the filesystem.
pub fn prepare(&mut self) -> io::Result<()> {
mkdir(&self.deps)?;
mkdir(&self.native)?;
mkdir(&self.incremental)?;
mkdir(&self.fingerprint)?;
mkdir(&self.examples)?;
Expand All @@ -154,9 +200,9 @@ impl Layout {
}
}

/// Fetch the root path.
/// Fetch the destination path for final artifacts (`/…/target/debug`).
pub fn dest(&self) -> &Path {
&self.root
&self.dest
}
/// Fetch the deps path.
pub fn deps(&self) -> &Path {
Expand All @@ -166,7 +212,11 @@ impl Layout {
pub fn examples(&self) -> &Path {
&self.examples
}
/// Fetch the root path.
/// Fetch the doc path.
pub fn doc(&self) -> &Path {
&self.doc
}
/// Fetch the root path (`/…/target`).
pub fn root(&self) -> &Path {
&self.root
}
Expand All @@ -178,7 +228,7 @@ impl Layout {
pub fn fingerprint(&self) -> &Path {
&self.fingerprint
}
/// Fetch the build path.
/// Fetch the build script path.
pub fn build(&self) -> &Path {
&self.build
}
Expand Down
1 change: 1 addition & 0 deletions src/cargo/core/compiler/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,7 @@ fn link_targets<'a, 'cfg>(
}))
}

/// Hardlink (file) or symlink (dir) src to dst if possible, otherwise copy it.
fn hardlink_or_copy(src: &Path, dst: &Path) -> CargoResult<()> {
debug!("linking {} to {}", src.display(), dst.display());
if is_same_file(src, dst).unwrap_or(false) {
Expand Down
2 changes: 1 addition & 1 deletion src/doc/src/reference/manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -628,7 +628,7 @@ dependencies residing in the workspace directory become members. You can add
additional packages to the workspace by listing them in the `members` key. Note
that members of the workspaces listed explicitly will also have their path
dependencies included in the workspace. Sometimes a package may have a lot of
workspace members and it can be onerous to keep up to date. The path dependency
workspace members and it can be onerous to keep up to date. The `members` list
can also use [globs][globs] to match multiple paths. Finally, the `exclude`
key can be used to blacklist paths from being included in a workspace. This can
be useful if some path dependencies aren't desired to be in the workspace at
Expand Down