Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PCH can be pre-compiled concurrently to the other targets #4167

Open
MaxwellGengYF opened this issue Sep 5, 2023 · 2 comments
Open

PCH can be pre-compiled concurrently to the other targets #4167

MaxwellGengYF opened this issue Sep 5, 2023 · 2 comments

Comments

@MaxwellGengYF
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Now the pre-compilation of PCH is serialized with the other targets' compilation, this is inefficient and unnecessary.

Describe the solution you'd like

PCH and the other targets' compilation can be concurrent.

Describe alternatives you've considered

No response

Additional context

No response

@waruqi
Copy link
Member

waruqi commented Sep 5, 2023

不太好搞,当前 target 间能 cpp并行编译,是因为假定 cpp 之间没有依赖关系的,可以优化的优化。但 pch 和 c++ 源文件之间存在依赖关系,得先编译 pch ,才能继续编译引用到这个 pch 的 cpp 文件,这就会影响现有的跨 target 并行编译,而且其他的 target 也可能会引用到这个 pch。

要处理好,得做跨 target 的所有源文件依赖扫描,分析出完整依赖图,然后根据所有源文件的依赖关系,再去构造 build 任务的依赖,实现尽可能并行编译。。这就复杂了,为了一点并行度的提升,这个实现的复杂度提升更多。。

目前走完整源码扫描来构建的,只有在编译 c++ modules 时候,我这么做了,实现复杂度很高,而且还得借助编译器的依赖分析,才能做到可靠分析,比如 clang-scan-deps ,以及 msvc 的 /scanDependencies ,但目前各家编译器对这个的支持力度,很有限,也就 msvc 可用,clang 一堆问题,gcc 的还没完全可用。

而 fallback 到 xmake 自己去扫描,尽管也可以,但可靠性肯定没编译器原生支持那么好。

而且目前即使 c++modules 的依赖分析,也是需要提前执行处理的,也还有很多问题,会影响内部一些 rule 对 project generator的支持,需要后续改进。

因此,即使现在加上了支持, pch 和 c++modules 混一起,问题更多,同样也会导致一些特性对 project generator 失效。

暂时先不搞了,也没时间,先等后面 c++modules 的源码分析完全可靠了,回头再看看怎么改进这个。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


It's not easy to do. Currently, cpp can be compiled in parallel between targets, because it is assumed that there is no dependency between cpp, and optimization can be optimized. However, there is a dependency relationship between pch and c++ source files. You must compile pch first before you can continue to compile cpp files that refer to this pch. This will affect the existing cross-target parallel compilation, and other targets may also refer to this pch.

To handle it well, it is necessary to scan all source files across targets, analyze the complete dependency graph, and then construct the dependencies of the build task according to the dependencies of all source files to achieve parallel compilation as much as possible. . This is complicated. For a little improvement in parallelism, the complexity of this implementation is increased even more. .

At present, it is built by scanning the complete source code. I only do this when compiling c++ modules. The implementation complexity is very high, and the dependency analysis of the compiler can be used to achieve reliable analysis, such as clang-scan-deps, And /scanDependencies of msvc, but currently the support of various compilers for this is very limited, only msvc is available, clang has a bunch of problems, and gcc is not fully available yet.

And fallback to xmake to scan by itself, although it is also possible, but the reliability is definitely not as good as the native support of the compiler.

And at present, even the dependency analysis of c++modules needs to be processed in advance, and there are still many problems, which will affect the support of some internal rules for the project generator, which needs subsequent improvement.

Therefore, even if support is added now, mixing pch and c++modules will cause more problems, and it will also cause some features to be invalid for the project generator.

Let's not do it for the time being, and we don't have time. Let's wait until the source code analysis of c++modules is completely reliable, and then we will see how to improve this later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants