用了镜像之后构建rustc报错 #294

bombless · 2019-09-20T03:48:16Z

重现：

git clone https://github.com/rust-lang/rust
cd rust
./x.py build

报错

……
error: failed to download from `https://crates-io.proxy.ustclug.org/api/v1/crates/signal-hook/0.1.7/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: crates)
thread 'main' panicked at 'command did not execute successfully: "/Users/yuekexiang/rust-homebrew/build/x86_64-apple-darwin/stage0/bin/cargo" "metadata" "--format-version" "1" "--features" "panic-unwind backtrace" "--manifest-path" "/Users/yuekexiang/rust-homebrew/src/libstd/Cargo.toml"
……

The text was updated successfully, but these errors were encountered:

ksqsf · 2019-09-20T10:06:52Z

I can confirm. This sometimes occurs, and I don't know why. You can wait for some time and this problem automatically disappears. Needs investigation.

ksqsf · 2019-10-18T08:09:13Z

Nginx seems to fail to rewrite some urls. Although I haven't fully understood this problem, I have disabled some unnecessary URL rewrites, which possibly works around this problem.

s977120 · 2020-04-29T08:59:43Z

连续十几天这样了频繁出现几乎不可用

taoky · 2020-04-29T09:01:48Z

连续十几天这样了频繁出现几乎不可用

能贴一下复现方式和报错信息吗？

s977120 · 2020-04-29T09:31:07Z

报错：

root $ cargo check
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded actix-http v1.0.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded async-trait v0.1.30 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded chrono v0.4.11 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
error: failed to download from `https://crates-io.proxy.ustclug.org/api/v1/crates/futures-macro/0.3.4/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: crates)

复现：

cargo new project
cd project
rustup override set nightly
echo 'actix-web = "*"' >> Cargo.toml
rm -rf ~/.cargo/registry/
cargo check

taoky · 2020-04-29T10:30:18Z

报错：

root $ cargo check
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded actix-http v1.0.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded async-trait v0.1.30 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded chrono v0.4.11 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
error: failed to download from `https://crates-io.proxy.ustclug.org/api/v1/crates/futures-macro/0.3.4/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: crates)

复现：

cargo new project
cd project
rustup override set nightly
echo 'actix-web = "*"' >> Cargo.toml
rm -rf ~/.cargo/registry/
cargo check

我从一台国外的服务器和一台腾讯云机器上测试了都没有问题。请问访问 https://crates-io.proxy.ustclug.org/ 正常吗？下载包实际是从这个地址下载的。

s977120 · 2020-04-29T10:41:30Z

访问是正常的，我感觉这问题跟并发有关，依赖少就正常，依赖多就出现这种情况

s977120 · 2020-04-29T11:51:48Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

taoky · 2020-04-29T11:55:26Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

那我改一下使用帮助吧……我不是 Rust 用户，也不清楚怎么修复访问的 host 变成了 crates 的问题。

s977120 · 2020-04-29T12:05:01Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

那我改一下使用帮助吧……我不是 Rust 用户，也不清楚怎么修复访问的 host 变成了 crates 的问题。

可能是 http2 没配置好吧

taoky · 2020-04-29T12:06:05Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

那我改一下使用帮助吧……我不是 Rust 用户，也不清楚怎么修复访问的 host 变成了 crates 的问题。

可能是 http2 没配置好吧

但是也不应该访问 https:///crates/ 啊，不知道 cargo 做了什么处理。

taoky · 2020-04-29T12:15:42Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

那我改一下使用帮助吧……我不是 Rust 用户，也不清楚怎么修复访问的 host 变成了 crates 的问题。

使用帮助 https://mirrors.ustc.edu.cn/help/crates.io-index.html 更新了。

ksqsf · 2020-04-29T14:18:04Z

@taoky 已经确定是并发问题了，我用 $ CARGO_HTTP_MULTIPLEXING=false cargo fetch 取消并行下载，不会出现这种情况了

那我改一下使用帮助吧……我不是 Rust 用户，也不清楚怎么修复访问的 host 变成了 crates 的问题。

可能是 http2 没配置好吧

但是也不应该访问 https:///crates/ 啊，不知道 cargo 做了什么处理。

Cargo 看到的是 302 到 https:///crates/ 了 [1]，之前没查出来配置哪里有问题……看来没搞定，先 reopen 了

[1] rust-lang/cargo#7515

knight42 · 2020-04-30T04:32:06Z

我找到这个问题的原因了，是因为我们的 nginx 会根据 client ip 对请求数作限制，默认是 4r/s 即每秒 4 个请求，如果超过的话就会返回 503。而我们的 nginx 配置里有这么一段

error_page 500 502 503 504 =302 $scheme://$proxy_host$uri;

即当上游返回 503 的时候，nginx 会让 client 302 到 $scheme://$proxy_host$uri，而这个时候 nginx 还没开始做 proxy，$proxy_host 应该还是空的，所以 client 会看到 nginx 返回的 302 的响应里的 location 是 https:///crates/futures-io/futures-io-0.3.4.crate.

目前我先对 crates-io.proxy.ustclug.org 稍微放开了一下限制

limit_req_zone $binary_remote_addr zone=rust-crates:10m rate=20r/s;
limit_req zone=rust-crates burst=150;

同时调整了一下配置，减少了一次 302 跳转，@s977120 你现在可以试试看有没有问题。

不过感觉这个只是临时的 workaround，要彻底解决应该要让 cargo 限制一下并发请求数。

taoky · 2022-05-26T14:26:24Z

我今天看了一下 cargo 的代码：https://github.com/rust-lang/cargo/blob/master/src/cargo/core/package.rs#L415，其做 multiplexing 的时候会限制最大连接数为 2，nginx 这里也没有限制其他的东西，照理不应该出现问题。我本地 rustup update 之后跑 cargo install wrangler 也出现了（和 #389 所述相同）问题，看日志的话是超过了 burst=100 的限制。

taoky · 2022-05-26T17:03:01Z

CARGO_HTTP_DEBUG=true CARGO_LOG=cargo::ops::registry=trace cargo install wrangler

可以生成详细的网络请求日志。阅读日志后发现问题在 http2 的最大并发流数量中：

[2022-05-26T16:44:22Z DEBUG cargo::ops::registry] http-debug: * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!

而这个数值在 curl 中默认为 13，服务器端可以调整这个设置。在将 nginx 的 ngx_http_v2_module 的 http2_max_concurrent_streams 从 128 降低到 50 后，问题似乎得到了解决。

ksqsf added the bug label Sep 20, 2019

ksqsf mentioned this issue Oct 16, 2019

Cargo tries to resolve an incorrect host name to download crates rust-lang/cargo#7515

Closed

ksqsf closed this as completed Oct 18, 2019

ksqsf reopened this Apr 29, 2020

taoky mentioned this issue May 15, 2022

cargo install出现问题 #389

Closed

taoky added the service/mirrors label May 15, 2022

taoky closed this as completed Feb 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

用了镜像之后构建rustc报错 #294

用了镜像之后构建rustc报错 #294

bombless commented Sep 20, 2019 •

edited

Loading

ksqsf commented Sep 20, 2019

ksqsf commented Oct 18, 2019

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020 •

edited

Loading

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

taoky commented Apr 29, 2020

ksqsf commented Apr 29, 2020 •

edited

Loading

knight42 commented Apr 30, 2020

taoky commented May 26, 2022

taoky commented May 26, 2022

用了镜像之后构建rustc报错 #294

用了镜像之后构建rustc报错 #294

Comments

bombless commented Sep 20, 2019 • edited Loading

ksqsf commented Sep 20, 2019

ksqsf commented Oct 18, 2019

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020 • edited Loading

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

s977120 commented Apr 29, 2020

taoky commented Apr 29, 2020

taoky commented Apr 29, 2020

ksqsf commented Apr 29, 2020 • edited Loading

knight42 commented Apr 30, 2020

taoky commented May 26, 2022

taoky commented May 26, 2022

bombless commented Sep 20, 2019 •

edited

Loading

s977120 commented Apr 29, 2020 •

edited

Loading

ksqsf commented Apr 29, 2020 •

edited

Loading