Skip to content

Commit

Permalink
rust: Update to 1.78.0
Browse files Browse the repository at this point in the history
- Switch back to .gz tarball
- Replace local bootstrap cache hack with upstreamed option

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
  • Loading branch information
1715173329 committed May 5, 2024
1 parent 3cac19e commit c1b3e04
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 59 deletions.
8 changes: 4 additions & 4 deletions lang/rust/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
include $(TOPDIR)/rules.mk

PKG_NAME:=rust
PKG_VERSION:=1.77.0
PKG_VERSION:=1.78.0
PKG_RELEASE:=1

PKG_SOURCE:=rustc-$(PKG_VERSION)-src.tar.xz
PKG_SOURCE:=rustc-$(PKG_VERSION)-src.tar.gz
PKG_SOURCE_URL:=https://static.rust-lang.org/dist/
PKG_HASH:=66126989782cbf77fa3aff121bbb108429f2d46fe19328c3de231553de711b90
PKG_HASH:=ff544823a5cb27f2738128577f1e7e00ee8f4c83f2a348781ae4fc355e91d5a9
HOST_BUILD_DIR:=$(BUILD_DIR)/host/rustc-$(PKG_VERSION)-src

PKG_MAINTAINER:=Luca Barbato <lu_zero@luminem.org>
Expand Down Expand Up @@ -74,6 +74,7 @@ HOST_CONFIGURE_ARGS = \
--disable-sanitizers \
--release-channel=stable \
--enable-cargo-native-static \
--bootstrap-cache-path=$(DL_DIR)/rustc \
--set=llvm.download-ci-llvm=true \
$(TARGET_CONFIGURE_ARGS)

Expand All @@ -87,7 +88,6 @@ define Host/Compile
$(RUST_SCCACHE_VARS) \
CARGO_HOME=$(CARGO_HOME) \
TARGET_CFLAGS="$(TARGET_CFLAGS)" \
OPENWRT_RUSTC_BOOTSTRAP_CACHE=$(DL_DIR)/rustc \
$(PYTHON) $(HOST_BUILD_DIR)/x.py \
--build-dir $(HOST_BUILD_DIR)/build \
--config $(HOST_BUILD_DIR)/config.toml \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
From bd479113d38aa453cbad9d9f5ca9c5fc8903b0cf Mon Sep 17 00:00:00 2001
From: onur-ozkan <work@onurozkan.dev>
Date: Thu, 11 Apr 2024 14:57:10 +0300
Subject: [PATCH] correct the handling of `bootstrap-cache-path` option

This change makes `build.bootstrap-cache-path` option to be configurable with
`./configure` script, so it can be used like `./configure --bootstrap-cache-path=demo`.

Signed-off-by: onur-ozkan <work@onurozkan.dev>
---
config.example.toml | 2 +-
src/bootstrap/configure.py | 4 +++-
2 files changed, 4 insertions(+), 2 deletions(-)

--- a/config.example.toml
+++ b/config.example.toml
@@ -302,7 +302,7 @@

# Set the bootstrap/download cache path. It is useful when building rust
# repeatedly in a CI invironment.
-# bootstrap-cache-path = /shared/cache
+#bootstrap-cache-path = /path/to/shared/cache

# Enable a build of the extended Rust tool set which is not only the compiler
# but also tools such as Cargo. This will also produce "combined installers"
--- a/src/bootstrap/configure.py
+++ b/src/bootstrap/configure.py
@@ -152,9 +152,9 @@ v("default-linker", "rust.default-linker
# (others are conditionally saved).
o("manage-submodules", "build.submodules", "let the build manage the git submodules")
o("full-bootstrap", "build.full-bootstrap", "build three compilers instead of two (not recommended except for testing reproducible builds)")
-o("bootstrap-cache-path", "build.bootstrap-cache-path", "use provided path for the bootstrap cache")
o("extended", "build.extended", "build an extended rust tool set")

+v("bootstrap-cache-path", None, "use provided path for the bootstrap cache")
v("tools", None, "List of extended tools will be installed")
v("codegen-backends", None, "List of codegen backends to build")
v("build", "build.build", "GNUs ./configure syntax LLVM build triple")
@@ -359,6 +359,8 @@ def apply_args(known_args, option_checki
set('target.{}.llvm-filecheck'.format(build_triple), value, config)
elif option.name == 'tools':
set('build.tools', value.split(','), config)
+ elif option.name == 'bootstrap-cache-path':
+ set('build.bootstrap-cache-path', value, config)
elif option.name == 'codegen-backends':
set('rust.codegen-backends', value.split(','), config)
elif option.name == 'host':
55 changes: 0 additions & 55 deletions lang/rust/patches/0002-rustc-bootstrap-cache.patch

This file was deleted.

6 comments on commit c1b3e04

@feckert
Copy link
Member

@feckert feckert commented on c1b3e04 May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this last change, I can no longer build rust!

    ERROR: package/feeds/packages/rust [host] failed to build.
make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make[1]: *** [/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/include/toplevel.mk:233: world] Fehler 1
make[1]: Verzeichnis „/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt“ wird verlassen
################################################################
Package buildlog '/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/logs/package/feeds/packages/rust/host-compile.txt'
################################################################
Applying /home/bbworker/bbworker/owrt_master_x86_64/System6/build/feeds/packages/lang/rust/patches/0001-Update-xz2-and-use-it-static.patch using plaintext: 
patching file src/bootstrap/Cargo.toml
Applying /home/bbworker/bbworker/owrt_master_x86_64/System6/build/feeds/packages/lang/rust/patches/0002-correct-the-handling-of-bootstrap-cache-path-option.patch using plaintext: 
patching file config.example.toml
patching file src/bootstrap/configure.py
Applying /home/bbworker/bbworker/owrt_master_x86_64/System6/build/feeds/packages/lang/rust/patches/0003-bump-libc-deps-to-0.2.146.patch using plaintext: 
patching file vendor/elasticlunr-rs/Cargo.lock
patching file vendor/libffi/Cargo.lock
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--target ...
configure: build.build          := x86_64-unknown-linux-gnu
configure: build.target         := ['x86_64-unknown-linux-musl', 'x86_64-unknown- ...
configure: build.host           := ['x86_64-unknown-linux-gnu']
configure: install.prefix       := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: install.bindir       := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: install.libdir       := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: install.sysconfdir   := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: install.datadir      := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: install.mandir       := /home/bbworker/bbworker/owrt_master_x86_64/Sys ...
configure: dist.compression-formats := ['gz']
configure: build.sanitizers     := False
configure: rust.channel         := stable
configure: build.cargo-native-static := True
configure: build.bootstrap-cache-path := /home/bbworker/dl/rustc
configure: llvm.download-ci-llvm := True
configure: target.x86_64-unknown-linux-musl.ar := x86_64-openwrt-linux-musl-gcc-a ...
configure: target.x86_64-unknown-linux-musl.cc := x86_64-openwrt-linux-musl-gcc
configure: target.x86_64-unknown-linux-musl.cxx := x86_64-openwrt-linux-musl-g++
configure: target.x86_64-unknown-linux-musl.linker := x86_64-openwrt-linux-musl-g ...
configure: target.x86_64-unknown-linux-musl.ranlib := x86_64-openwrt-linux-musl-g ...
configure: target.x86_64-unknown-linux-musl.crt-static := False
configure: target.x86_64-unknown-linux-musl.musl-root := /home/bbworker/bbworker/ ...
configure: profile              := dist
configure: 
configure: writing `config.toml` in current directory
configure: 
configure: run `python /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/host/rustc-1.78.0-src/x.py --help`
extracting /home/bbworker/dl/rustc/2024-03-21/cargo-1.77.0-x86_64-unknown-linux-gnu.tar.xz
extracting /home/bbworker/dl/rustc/2024-03-21/rust-std-1.77.0-x86_64-unknown-linux-gnu.tar.xz
extracting /home/bbworker/dl/rustc/2024-03-21/rustc-1.77.0-x86_64-unknown-linux-gnu.tar.xz
   Compiling proc-macro2 v1.0.76
   Compiling unicode-ident v1.0.12
   Compiling cfg-if v1.0.0
   Compiling version_check v0.9.4
   Compiling memchr v2.7.1
   Compiling typenum v1.17.0
   Compiling libc v0.2.151
   Compiling regex-syntax v0.8.2
   Compiling cc v1.0.73
   Compiling crossbeam-utils v0.8.18
   Compiling serde v1.0.195
   Compiling rustix v0.38.28
   Compiling linux-raw-sys v0.4.12
   Compiling bitflags v2.4.1
   Compiling pkg-config v0.3.28
   Compiling generic-array v0.14.7
   Compiling clap_lex v0.6.0
   Compiling anstyle v1.0.4
   Compiling aho-corasick v1.1.2
   Compiling quote v1.0.35
   Compiling heck v0.4.1
   Compiling syn v2.0.48
   Compiling clap_builder v4.4.12
   Compiling crossbeam-epoch v0.9.17
   Compiling same-file v1.0.6
   Compiling serde_json v1.0.111
   Compiling semver v1.0.21
   Compiling log v0.4.20
   Compiling lzma-sys v0.1.20
   Compiling crossbeam-deque v0.8.4
   Compiling walkdir v2.4.0
   Compiling filetime v0.2.23
   Compiling bootstrap v0.0.0 (/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/host/rustc-1.78.0-src/src/bootstrap)
   Compiling ryu v1.0.16
   Compiling itoa v1.0.10
   Compiling regex-automata v0.4.3
   Compiling cpufeatures v0.2.12
   Compiling object v0.32.2
   Compiling block-buffer v0.10.4
   Compiling crypto-common v0.1.6
   Compiling digest v0.10.7
   Compiling xattr v1.2.0
   Compiling sha2 v0.10.8
   Compiling tar v0.4.40
   Compiling fd-lock v4.0.2
   Compiling cmake v0.1.48
   Compiling home v0.5.9
   Compiling once_cell v1.19.0
   Compiling termcolor v1.4.0
   Compiling toml v0.5.11
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling bstr v1.9.0
   Compiling globset v0.4.14
   Compiling opener v0.5.2
   Compiling ignore v0.4.21
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0 (/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/host/rustc-1.78.0-src/src/tools/build_helper)
   Compiling xz2 v0.1.7
    Finished dev [unoptimized] target(s) in 30.12s
WARNING: The `change-id` is missing in the `config.toml`. This means that you will not be able to track the major changes made to the bootstrap configurations.
NOTE: to silence this warning, add `change-id = 121754` at the top of `config.toml`
downloading https://ci-artifacts.rust-lang.org/rustc-builds/9b00956e56009bab2aa15d7bff10916599e3d6d6/rust-dev-1.78.0-x86_64-unknown-linux-gnu.tar.xz
#=#=#                                                                         
                                                                           1.3%
##                                                                         4.1%
####                                                                       6.9%
#######                                                                    9.7%
#########                                                                 12.6%
##########                                                                15.1%
############                                                              17.9%
##############                                                            20.7%
################                                                          23.6%
##################                                                        26.3%
#####################                                                     29.2%
#######################                                                   31.9%
#########################                                                 34.8%
###########################                                               37.6%
#############################                                             40.4%
###############################                                           43.3%
#################################                                         46.0%
###################################                                       48.8%
#####################################                                     51.6%
#######################################                                   54.4%
#########################################                                 57.2%
###########################################                               60.0%
#############################################                             62.8%
###############################################                           65.6%
#################################################                         68.3%
###################################################                       71.2%
#####################################################                     73.9%
######################################################                    75.6%
########################################################                  78.4%
##########################################################                81.1%
############################################################              83.9%
##############################################################            86.2%
###############################################################           88.2%
#################################################################         91.0%
###################################################################       93.4%
####################################################################      95.8%
######################################################################    98.2%
######################################################################## 100.0%
thread 'main' panicked at src/core/download.rs:211:9:
std::fs::rename(&tempfile, dest_path) failed with Invalid cross-device link (os error 18) ("failed to rename \"/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/host/rustc-1.78.0-src/build/tmp/rust-dev-1.78.0-x86_64-unknown-linux-gnu.tar.xz\" to \"/home/bbworker/dl/rustc/llvm-9b00956e56009bab2aa15d7bff10916599e3d6d6-false/rust-dev-1.78.0-x86_64-unknown-linux-gnu.tar.xz\"")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Building bootstrap
Build completed unsuccessfully in 0:00:44
make[4]: *** [Makefile:109: /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/host/rustc-1.78.0-src/.built] Error 1
time: package/feeds/packages/rust/host-compile#99.83#14.91#59.05
make: *** [Makefile:37: build] Fehler 2
program finished with exit code 2
elapsedTime=1449.977806

I suspect that the reason is that my dl folder is mounted via NFS?
See: https://doc.rust-lang.org/std/fs/fn.rename.html
@1715173329 Can you make a statement about this? I am not a rustc specialist.

@1715173329
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the inconvenience. By checking history commits, I found I missed the change in f9f1e02.

Ping @lu-zero our rust expert here ;)
Is it possible to upstream this change?

@feckert
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could change the pattern from rename to copy / delete.
https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/core/download.rs#L212-L213
The disadvantage would be that it may take longer, because the file system entry is not simply changed as with rename.

@lu-zero
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to doublecheck if there isn't already a fn rename doing that we can leverage.

@lu-zero
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently didn't so I made rust-lang/rust#124975

@1715173329
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Please sign in to comment.