gcc 编译环境下xmake构建的openmp程序无法多线程运行 #2603

kritha · 2022-07-26T14:00:13Z

Xmake 版本

2.6.9

操作系统版本和架构

Ubuntu 20.04 LTS x86_64, gcc version 9.4.0, linux kernel 5.15.0-42-lowlatency, CPU 12700k

描述问题

使用xmake & gcc编译openmp程序，构建得到的程序只能以单线程运行。而相同的源代码，手动使用gcc命令行编译得到的程序可以正常多线程运行。相同源代码在Windows11系统使用xmake & MSVC构建的openmp程序可以正常多线程运行，但使用xmake & MinGW64编译构建的程序也只能单线程运行。

期待的结果

找到gcc 编译环境下xmake构建的openmp程序只能单线程运行的原因

工程配置

本问题在测试个人编写的粒子跟踪模拟程序时发现的，使用以下最简单的工程配置以及源代码仍然存在该问题，所以大概率不是本人代码的问题。而且如上面问题描述所说的，Windows系统下MinGW64编译环境也有该问题，但是MSVC编译环境是没问题的。
xmake.lua:
add_rules("mode.release")
add_requires("openmp")

target("omptest")
add_packages("openmp")
set_kind("binary")
add_files("src/*.cpp")

main.cpp:
#include <stdio.h>
#include <omp.h>
int main(void)
{
#pragma omp parallel
{
printf("Hello, world.\n");
}
return 0;
}

附加信息和错误日志

xmake编译运行结果：
(base) kritha@kritha-ubuntu:/simulation/data/xmake-test/omptest2$ xmake -v
checking for platform ... linux
checking for architecture ... x86_64
checking for gcc ... /usr/bin/gcc
checking for dmd ... no
checking for ldc2 ... no
checking for gdc ... no
checking for zig ... no
checking for zig ... no
checking for unzip ... /usr/bin/unzip
checking for git ... /usr/bin/git
checking for gzip ... /usr/bin/gzip
checking for tar ... /usr/bin/tar
checking for ping ... /usr/bin/ping
pinging for the host(github.com) ... 65535 ms
pinging for the host(gitlab.com) ... 65535 ms
pinging for the host(gitee.com) ... 65535 ms
/usr/bin/git rev-parse HEAD
checking for gcc ... /usr/bin/gcc
checking for the c compiler (cc) ... gcc
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
checking for xmake-repo::openmp ... openmp
checking for /usr/bin/gcc ... ok
checking for flags (-fPIC) ... ok
checking for flags (-fvisibility-inlines-hidden) ... ok
checking for flags (-O3) ... ok
checking for flags (-fopenmp) ... ok
checking for flags (-DNDEBUG) ... ok
[ 25%]: ccache compiling.release src/main.cpp
/usr/bin/gcc -c -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -fopenmp -DNDEBUG -o build/.objs/omptest/linux/x86_64/release/src/main.cpp.o src/main.cpp
checking for flags (-MMD -MF) ... ok
checking for flags (-fdiagnostics-color=always) ... ok
checking for g++ ... /usr/bin/g++
checking for the linker (ld) ... g++
checking for /usr/bin/g++ ... ok
checking for flags (-fPIC) ... ok
checking for flags (-fopenmp) ... ok
[ 50%]: linking.release omptest
/usr/bin/g++ -o build/linux/x86_64/release/omptest build/.objs/omptest/linux/x86_64/release/src/main.cpp.o -m64 -s -fopenmp
[100%]: build ok!
(base) kritha@kritha-ubuntu:/simulation/data/xmake-test/omptest2$ xmake r
Hello, world.
(base) kritha@kritha-ubuntu:~/simulation/data/xmake-test/omptest2$

使用命令行手动编译及运行结果：
(base) kritha@kritha-ubuntu:/simulation/data/xmake-test/omptest2/src$ g++ -fopenmp -O3 main.cpp -o test
(base) kritha@kritha-ubuntu:/simulation/data/xmake-test/omptest2/src$ ./test
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
Hello, world.
(base) kritha@kritha-ubuntu:~/simulation/data/xmake-test/omptest2/src$

xq114 · 2022-07-26T14:17:36Z

不用xmake run运行，直接输路径运行编译出来这个程序会有问题吗

kritha · 2022-07-26T14:18:55Z

也有这个问题

waruqi · 2022-07-26T15:04:26Z

而相同的源代码，手动使用gcc命令行编译得到的程序可以正常多线程运行

你可以通过 xmake -v 输出的完整编译链接命令，跟你手动的对比下 flags 差异。

xq114 · 2022-07-26T16:25:11Z

我测试了一下，这里第一步的/usr/bin/gcc改成/usr/bin/g++就对了

xq114 · 2022-07-26T16:28:13Z

理论上可以用xmake f --cxx=g++暂时改一下，不过我这出问题了，好像改不了 @waruqi

$ xmake f -vD --cxx=g++
checking for platform ... linux
checking for architecture ... x86_64
checking for gcc ... /usr/bin/gcc
checkinfo: cannot runv(dmd --version), No such file or directory
checking for dmd ... no
checkinfo: cannot runv(ldc2 --version), No such file or directory
checking for ldc2 ... no
checkinfo: cannot runv(gdc --version), No such file or directory
checking for gdc ... no
checkinfo: cannot runv(zig version), No such file or directory
checking for zig ... no
checkinfo: cannot runv(zig version), No such file or directory
checking for zig ... no
checking for unzip ... /usr/bin/unzip
checking for git ... /usr/bin/git
checking for gzip ... /bin/gzip
checking for tar ... /bin/tar
/usr/bin/git rev-parse HEAD
checking for gcc ... /usr/bin/gcc
checking for the c compiler (cc) ... gcc
error: @programdir/core/main.lua:280: @programdir/core/package/package.lua:979: attempt to index a nil value (local 'toolname')
stack traceback:
    [@programdir/core/package/package.lua:979]: in function 'has_tool'
    [...make/repositories/xmake-repo/packages/o/openmp/xmake.lua:13]: in function 'on_load'
    [@programdir/modules/private/action/require/impl/package.lua:837]: in function '_load_package'
    [@programdir/modules/private/action/require/impl/package.lua:864]: in function '_load_packages'
    [@programdir/modules/private/action/require/impl/package.lua:1067]: in function 'load_packages'
    [...modules/private/action/require/impl/install_packages.lua:654]:
    [@programdir/modules/private/action/require/install.lua:85]:
    [@programdir/actions/config/main.lua:420]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:280]:
    [@programdir/core/base/task.lua:501]: in function 'run'
    [@programdir/core/main.lua:278]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:388]:

stack traceback:
        [C]: in function 'error'
        @programdir/core/base/os.lua:872: in function 'os.raiselevel'
        (...tail calls...)
        @programdir/core/main.lua:280: in upvalue 'cotask'
        @programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>

waruqi · 2022-07-27T01:07:55Z

理论上可以用xmake f --cxx=g++暂时改一下，不过我这出问题了，好像改不了 @waruqi

我这里可以，你这应该是有包里用了 package:has_tool，取不到 toolname 导致，我加了个判空处理，不过要让 package 支持tool切换，不建议单独切，走 --toolchain=gcc 来切

waruqi · 2022-07-27T01:08:44Z

我测试了一下，这里第一步的/usr/bin/gcc改成/usr/bin/g++就对了

为啥这跟 g++/gcc 还有区别？

xq114 · 2022-07-27T13:14:38Z

为啥这跟 g++/gcc 还有区别？

不是这个原因，我执行xmake clean -a之后再运行这些命令，发现gcc也可以，不需要改g++，问题可能出在内置的ccache实现。

kritha · 2022-07-27T13:44:52Z

为啥这跟 g++/gcc 还有区别？

不是这个原因，我执行xmake clean -a之后再运行这些命令，发现gcc也可以，不需要改g++，问题可能出在内置的ccache实现。

我试了一下，把xmake -v显示的编译链接两条命令单独执行，程序可以正常运行，且和gcc/g++没关系。但是xmake构建的程序运行就不对。

waruqi · 2022-07-27T13:45:21Z

那吧 ccache 关了试试，xmake f --ccache=n; xmake -r

kritha · 2022-07-27T13:49:18Z

那吧 ccache 关了试试，xmake f --ccache=n; xmake -r

按照你说的关了ccache，xmake构建的程序运行正常。谢谢大佬！

waruqi · 2022-07-27T13:52:56Z

那吧 ccache 关了试试，xmake f --ccache=n; xmake -r

按照你说的关了ccache，xmake构建的程序运行正常。谢谢大佬！

这会你再把 ccache 打开，执行 xmake f --policies=preprocessor.gcc.directives_only:n -c; xmake 试试呢

我确认下是不是预处理加了 -fdirectives-only 导致

kritha · 2022-07-27T14:03:36Z

这样也是可以的，程序运行正常。

waruqi · 2022-07-27T14:07:25Z

那就是 -fdirectives-only 这个的问题，你先暂时禁用好了，回头我详细分析后再改进下，不行就默认禁用 -fdirectives-only。

waruqi · 2022-07-27T14:08:13Z

set_policy("preprocessor.gcc.directives_only", false)

也可以在 xmake.lua 里面禁用掉

kritha · 2022-07-27T14:11:17Z

set_policy("preprocessor.gcc.directives_only", false)
也可以在 xmake.lua 里面禁用掉

谢谢大佬 : )

waruqi · 2022-07-27T15:27:17Z

看了下，加了 -fdirectives-only 后，#pragma omp parallel for 直接被干掉了。

我改了下，默认禁用了，如果要进一步加速 ccache，可以自己选择性开启它。可以更新到 dev 再试试，xmake update dev

set_policy("preprocessor.gcc.directives_only", true)

kritha added the bug label Jul 26, 2022

waruqi added this to the v2.7.1 milestone Jul 27, 2022

waruqi added the improvement label Jul 27, 2022

waruqi mentioned this issue Jul 27, 2022

内置 ccache 的一些表现上的问题 #2425

Closed

waruqi closed this as completed Jul 27, 2022

waruqi mentioned this issue Aug 1, 2022

Not being able to use OpenMP #2634

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gcc 编译环境下xmake构建的openmp程序无法多线程运行 #2603

gcc 编译环境下xmake构建的openmp程序无法多线程运行 #2603

kritha commented Jul 26, 2022

xq114 commented Jul 26, 2022

kritha commented Jul 26, 2022

waruqi commented Jul 26, 2022 •

edited

Loading

xq114 commented Jul 26, 2022

xq114 commented Jul 26, 2022 •

edited

Loading

waruqi commented Jul 27, 2022

waruqi commented Jul 27, 2022

xq114 commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022 •

edited

Loading

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022

waruqi commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022 •

edited

Loading

gcc 编译环境下xmake构建的openmp程序无法多线程运行 #2603

gcc 编译环境下xmake构建的openmp程序无法多线程运行 #2603

Comments

kritha commented Jul 26, 2022

Xmake 版本

操作系统版本和架构

描述问题

期待的结果

工程配置

附加信息和错误日志

xq114 commented Jul 26, 2022

kritha commented Jul 26, 2022

waruqi commented Jul 26, 2022 • edited Loading

xq114 commented Jul 26, 2022

xq114 commented Jul 26, 2022 • edited Loading

waruqi commented Jul 27, 2022

waruqi commented Jul 27, 2022

xq114 commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022 • edited Loading

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022

waruqi commented Jul 27, 2022

kritha commented Jul 27, 2022

waruqi commented Jul 27, 2022 • edited Loading

waruqi commented Jul 26, 2022 •

edited

Loading

xq114 commented Jul 26, 2022 •

edited

Loading

waruqi commented Jul 27, 2022 •

edited

Loading

waruqi commented Jul 27, 2022 •

edited

Loading