rustc_llvm fails to build if source directory contains whitespace

[CarePackage17]

Hi all,

I have followed the instructions to build Rust on Windows with the MSVC toolchain (note: this issue is not windows specific) and hit a build error in rustc_llvm; here's the output:

error: failed to run custom build command for `rustc_llvm v0.0.0 (C:\Users\Krzysztof Lesiak\Desktop\devel\rust\src\librustc_llvm)`
process didn't exit successfully: `C:\Users\Krzysztof Lesiak\Desktop\devel\rust\build\x86_64-pc-windows-msvc\stage0-codegen\release\build\rustc_llvm-9e0bf274131fe14d\build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-changed=C:\Users\Krzysztof Lesiak\Desktop\devel\rust\build\x86_64-pc-windows-msvc\llvm\bin\llvm-config.exe
cargo:rerun-if-env-changed=LLVM_CONFIG
cargo:rustc-cfg=llvm_component="aarch64"
cargo:rustc-cfg=llvm_component="arm"
cargo:rustc-cfg=llvm_component="asmparser"
cargo:rustc-cfg=llvm_component="bitreader"
cargo:rustc-cfg=llvm_component="bitwriter"
cargo:rustc-cfg=llvm_component="hexagon"
cargo:rustc-cfg=llvm_component="instrumentation"
cargo:rustc-cfg=llvm_component="interpreter"
cargo:rustc-cfg=llvm_component="ipo"
cargo:rustc-cfg=llvm_component="linker"
cargo:rustc-cfg=llvm_component="lto"
cargo:rustc-cfg=llvm_component="mcjit"
cargo:rustc-cfg=llvm_component="mips"
cargo:rustc-cfg=llvm_component="msp430"
cargo:rustc-cfg=llvm_component="nvptx"
cargo:rustc-cfg=llvm_component="powerpc"
cargo:rustc-cfg=llvm_component="sparc"
cargo:rustc-cfg=llvm_component="systemz"
cargo:rustc-cfg=llvm_component="x86"
cargo:rerun-if-changed-env=LLVM_RUSTLLVM
cargo:rerun-if-changed=../rustllvm\RustWrapper.cpp
cargo:rerun-if-changed=../rustllvm\rustllvm.h
cargo:rerun-if-changed=../rustllvm\README
cargo:rerun-if-changed=../rustllvm\PassWrapper.cpp
cargo:rerun-if-changed=../rustllvm\llvm-rebuild-trigger
cargo:rerun-if-changed=../rustllvm\Linker.cpp
cargo:rerun-if-changed=../rustllvm\ArchiveWrapper.cpp
cargo:rerun-if-changed=../rustllvm\.editorconfig
TARGET = Some("x86_64-pc-windows-msvc")
OPT_LEVEL = Some("2")
HOST = Some("x86_64-pc-windows-msvc")
CXX_x86_64-pc-windows-msvc = None
CXX_x86_64_pc_windows_msvc = None
HOST_CXX = None
CXX = None
CXXFLAGS_x86_64-pc-windows-msvc = None
CXXFLAGS_x86_64_pc_windows_msvc = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
running: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64\\cl.exe" "/nologo" "/MT" "/O2" "-IC:\\Users\\Krzysztof" "Lesiak\\Desktop\\devel\\rust\\build\\x86_64-pc-windows-msvc\\llvm\\include" "/nologo" "/MT" "/Zc:inline" "/Zc:strictStrings" "/Oi" "/Zc:rvalueCast" "/W4" "-wd4141" "-wd4146" "-wd4180" "-wd4244" "-wd4258" "-wd4267" "-wd4291" "-wd4345" "-wd4351" "-wd4355" "-wd4456" "-wd4457" "-wd4458" "-wd4459" "-wd4503" "-wd4624" "-wd4722" "-wd4800" "-wd4100" "-wd4127" "-wd4512" "-wd4505" "-wd4610" "-wd4510" "-wd4702" "-wd4245" "-wd4706" "-wd4310" "-wd4701" "-wd4703" "-wd4389" "-wd4611" "-wd4805" "-wd4204" "-wd4577" "-wd4091" "-wd4592" "-wd4319" "-wd4324" "-w14062" "-we4238" "/nologo" "/MT" "/EHs-c-" "/GR-" "-D_CRT_SECURE_NO_DEPRECATE" "-D_CRT_SECURE_NO_WARNINGS" "-D_CRT_NONSTDC_NO_DEPRECATE" "-D_CRT_NONSTDC_NO_WARNINGS" "-D_SCL_SECURE_NO_DEPRECATE" "-D_SCL_SECURE_NO_WARNINGS" "-DUNICODE" "-D_UNICODE" "-D__STDC_CONSTANT_MACROS" "-D__STDC_FORMAT_MACROS" "-D__STDC_LIMIT_MACROS" "/DLLVM_COMPONENT_AARCH64" "/DLLVM_COMPONENT_ARM" "/DLLVM_COMPONENT_ASMPARSER" "/DLLVM_COMPONENT_BITREADER" "/DLLVM_COMPONENT_BITWRITER" "/DLLVM_COMPONENT_HEXAGON" "/DLLVM_COMPONENT_INSTRUMENTATION" "/DLLVM_COMPONENT_INTERPRETER" "/DLLVM_COMPONENT_IPO" "/DLLVM_COMPONENT_LINKER" "/DLLVM_COMPONENT_LTO" "/DLLVM_COMPONENT_MCJIT" "/DLLVM_COMPONENT_MIPS" "/DLLVM_COMPONENT_MSP430" "/DLLVM_COMPONENT_NVPTX" "/DLLVM_COMPONENT_POWERPC" "/DLLVM_COMPONENT_SPARC" "/DLLVM_COMPONENT_SYSTEMZ" "/DLLVM_COMPONENT_X86" "/DLLVM_RUSTLLVM" "/FoC:\\Users\\Krzysztof Lesiak\\Desktop\\devel\\rust\\build\\x86_64-pc-windows-msvc\\stage0-codegen\\x86_64-pc-windows-msvc\\release\\build\\rustc_llvm-16916bb4d00807d7\\out\\../rustllvm\\PassWrapper.o" "/c" "../rustllvm/PassWrapper.cpp"
cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'Lesiak\Desktop\devel\rust\build\x86_64-pc-windows-msvc\llvm\include', object file assumed
cargo:warning=cl : Command line warning D9027 : source file 'Lesiak\Desktop\devel\rust\build\x86_64-pc-windows-msvc\llvm\include' ignored
PassWrapper.cpp
c:\users\krzysztof lesiak\desktop\devel\rust\src\rustllvm\rustllvm.h(11): fatal error C1083: Cannot open include file: 'llvm-c/BitReader.h': No such file or directory
exit code: 2

When inspecting the first /I argument to cl.exe the build script seems to have problems with the space in my user name and breaks up the path so the compiler can't find BitReader.h as a result.

Edit: The offending code is in librustc_llvm/build.rs line 139:

    for flag in cxxflags.split_whitespace() {
        // Ignore flags like `-m64` when we're doing a cross build
        if is_crossed && flag.starts_with("-m") {
            continue;
        }

        // -Wdate-time is not supported by the netbsd cross compiler
        if is_crossed && target.contains("netbsd") && flag.contains("date-time") {
            continue;
        }

        cfg.flag(flag);
    }

I'm not quite sure how to fix it myself; something like the Args iterator from std::env would do the job, but I've got no idea how to obtain it here.

Edit2: I just noticed something weird, the /Fo argument also contains my username and it seems to have been correctly escaped there.

[euclio]

Looks like this is an upstream bug: https://bugs.llvm.org/show_bug.cgi?id=27743

[euclio]

I managed to get a local build of LLVM working with a space in the filename, but when we try to build rustc with it we run into a problem with the run-make tests, since Makefile syntax doesn't play nice with spaces.

[Mark-Simulacrum]

FWIW I think if the run-make tests are pretty hard (or ugly) to fix, then we shouldn't worry too much -- I would rather invest in #40713.

[Fishrock123]

What is the suggested workaround for this? Build LLVM separately?

Edit: I'm building rust outside my user directory anyways so I just renamed my rust projects folder.

[Fishrock123]

I will note the upstream issue is apparently from 2016 and it does not look like it was taken seriously.

[jyn514]

As a workaround, I set [rust] codegen-backends = ["cranelift"] in config.toml.

[jyn514]

I'm going to remove the Windows label, Windows tends to have more spaces in filenames, but I don't think this is specific to windows in any other way.

[jyn514]

I managed to get a local build of LLVM working with a space in the filename, but when we try to build rustc with it we run into a problem with the run-make tests, since Makefile syntax doesn't play nice with spaces.

@euclio can you share how you got this working?

[euclio]

Sure, though my fork is pretty out-of-date: https://github.com/euclio/llvm-project/tree/build-with-spaces

[ckaran]

I just submitted a PR for this: #101043. It's a temporary fix so that others will know what the issue is, although it doesn't attempt to fix any of the underlying issues.