Cannot [replace] sub-dependency (package version not found, various other problems)

[AndiDog]

This relates directly to #3191. I'm having a similar use case with ffmpeg: since I want to use a newer FFmpeg library (3.x) than rust-ffmpeg currently depends on (2.8), I must replace that dependency by rust-ffmpeg-sys 3.x or master. Ideally, I'd just clone both repositories as submodule at the desired commits, and use [dependencies.ffmpeg] path = submodules/rust-ffmpeg and [replace] "rust-ffmpeg-sys:..." { path = submodules/rust-ffmpeg-sys } .

Unfortunately all my attempts failed 😢. But the good news is that I have a script for you to reproduce:

#!/usr/bin/env bash
set -e

cd /tmp
[ -e replace_problem ] || mkdir replace_problem
cd replace_problem

[ -e rust-ffmpeg ] || git clone -q https://github.com/meh/rust-ffmpeg.git
(
    cd rust-ffmpeg
    git reset --hard 8aabd018f537934d4520d6dad97d29d6f41baa54
    grep -A1 dependencies.ffmpeg-sys Cargo.toml
)

[ -e rust-ffmpeg-sys ] || git clone -q https://github.com/meh/rust-ffmpeg-sys.git
(
    cd rust-ffmpeg-sys
    echo "ADD GIBBERISH TO CHECK WHETHER THIS DIRECTORY GETS BUILT AT ALL" >> src/lib.rs
    git reset --hard b768211648c7a1063f423feaa6bdea2456af3408
)

rm -rf main_project
cargo new --bin main_project
(
    cd main_project

    tee -a Cargo.toml <<EOF
[dependencies.ffmpeg]
path = "../rust-ffmpeg"

[replace]
# I don't want to build against ffmpeg-sys 2.8 because I have FFmpeg 3.x installed and thus need to replace ffmpeg-sys
# with a later revision. At best, I want to have a Git submodule representing exactly the commit I need.
# Neither of the below attempts work, for different reasons (see problem in comment). You can comment them in
# individually to reproduce.

# error: no matching package for override \`https://github.com/rust-lang/crates.io-index#ffmpeg-sys:2.8.8\` found
# location searched: file:///tmp/replace_problem/rust-ffmpeg-sys
# version required: = 2.8.8
"ffmpeg-sys:2.8.8" = { git = "file:///tmp/replace_problem/rust-ffmpeg-sys" }

# error: no matching package for override \`https://github.com/rust-lang/crates.io-index#ffmpeg-sys:2.8.8\` found
# location searched: file:///private/tmp/replace_problem/rust-ffmpeg-sys
# version required: = 2.8.8
#"ffmpeg-sys:2.8.8" = { path = "../rust-ffmpeg-sys" }

# replacements must specify a valid semver version to replace, but \`ffmpeg-sys:2.8\` does not
#"ffmpeg-sys:2.8" = { git = "file:///tmp/replace_problem/rust-ffmpeg-sys" }

# Parsing errors, just like above
#"ffmpeg-sys:>=2.8.0" = { git = "file:///tmp/replace_problem/rust-ffmpeg-sys" }
#"ffmpeg-sys:^2.8.0" = { git = "file:///tmp/replace_problem/rust-ffmpeg-sys" }

# This doesn't replace the dependency, but builds 2.8.9 as downloaded from the Internet
#"ffmpeg-sys:2.8.0" = { path = "../rust-ffmpeg-sys" }
EOF

    cargo build
)

I see the following problems from the error messages:

• cargo fails to find the package version even though the rust-ffmpeg-sys directory is a Git repository and there's a commit where Cargo.toml says it's version 2.8.8
• I couldn't find a way to get >= to work, as attempt to find the actual version to replace
• The last case does no replacement at all with no warning or error

[AndiDog]

This is on recent nightly.

[alexcrichton]

I think what's happening here is that overrides require the version of the override to match the version of what's being overridden, but that's not happening here?

[antoyo]

I confirm the last case of this issue.
Even if I use a version that does not exist, I won't get any warning/error.

Edit: I confirm that version nightly-2016-11-03 does not have this issue.

[alexcrichton]

@antoyo could you clarify what you mean by "last cast" and perhaps provide a link to your setup as well?

[antoyo]

I mean that the dependency is not replaced and I get no warning or error.
You can see the [replace] here.

[alexcrichton]

@antoyo ah sorry there may be a few things in flight, could you describe the expected, actual, and difference in behaviors?

[antoyo]

The old behavior was that the dependency gets replaced.
The new behavior is that the dependency is not replaced as thought there are no [replace].

[alexcrichton]

@antoyo oh that's actually due to #3248 because the replace should look like:

[replace]
"https://github.com/gtk-rs/sys#glib-sys:0.3.1" = { git = "https://github.com/antoyo/sys" }

(I think that's the right URL).

That is, it was a bug that this worked before, unfortunately.

[antoyo]

Thanks, that actually fixed my issue.
But I still think there should be at least a warning when a dependency is not replaced.

[AndiDog]

I debugged a bit in the cargo source code, and it occurs to me I misunderstood to what extent [replace] can be used.

The docs say

Note that when a crate is overridden the copy it's overridden with must have both the same name and version, but it can come from a different source (e.g. git or a local path).

but my intention is to compile rust-ffmpeg-sys 3.x instead of the transitive dependency 2.8.x. cargo reads my specified git repo as override, but requires is to be of the same version (in my example script: 2.8.8).

@alexcrichton Would that use case ever be possible with [replace], or am I limited to .cargo/config path overrides?

[alexcrichton]

@AndiDog replacing with an entirely new major version tends to not be possible with either [replace] or .cargo/config. Major version bumps tend to be accompanied with major API breakage which would cause a failure to compile anyway. In that that sense the only "fix" would be to literally restructure the crate graph, updating dependencies and running tests along the way to the root (to verify everything works)

[AndiDog]

Close as wontfix then? I'll basically have to use path overrides, and/or simply maintain my own local fork of the projects to use updated dependencies.

[alexcrichton]

Ah yeah, in that case this is basically wontfix. If you have any more trouble though just let me know!