How to manually override these SSE, SSE2, AVX, etc. checks in configure.

[xiahualiu]

Our cross compiler version gcc 4.x.x is very old.

Even though it passes these SSE AVX sanity tests in configure, it cannot compile libsodium successfully with these arguments turned on, the compiler crashes with internal errors at emmintrin.h header files.

My question is how can I modify configure or configure.ac so I can manually control the result of these tests? I just want to turn them off one by one and check which one is the culprit.

Thanks for help!

[jedisct1]

What errors are you getting during compilation?

More tests can be performed by the configure script to check for these errors. If a feature doesn't work, the configure script should not enable it.

[xiahualiu]

Thanks for the reply.

Our target hardware arch is intel core2, from https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html, core2 doesn't support AVX.

But the cross compiler gcc we uses support AVX as well, I tried to override with CFLAGS = -march=core2 before configure, but the result is the same, -mavx still appears in the generated Makefile.

CC       crypto_aead/aes256gcm/aesni/libaesni_la-aead_aes256gcm_aesni.lo
/tmp/ccuwmR2U.s: Assembler messages:
/tmp/ccuwmR2U.s:790: Error: no such instruction: `vpclmulqdq $0,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:794: Error: no such instruction: `vpclmulqdq $17,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:829: Error: no such instruction: `vpclmulqdq $17,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:833: Error: no such instruction: `vpclmulqdq $0,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:837: Error: no such instruction: `vpclmulqdq $16,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:840: Error: no such instruction: `vpclmulqdq $1,%xmm2,%xmm1,%xmm1'
/tmp/ccuwmR2U.s:904: Error: no such instruction: `vpclmulqdq $0,%xmm1,%xmm0,%xmm0'
/tmp/ccuwmR2U.s:917: Error: no such instruction: `vpclmulqdq $0,%xmm1,%xmm0,%xmm0'
make[3]: *** [crypto_aead/aes256gcm/aesni/libaesni_la-aead_aes256gcm_aesni.lo] Error 1

[jedisct1]

The compiler looks fine.

The problem is an inconsistent toolchain : the compiler and the GNU binutils are incompatible. The compiler emits code that the as tool is not able to assemble.

So, the binutils package needs to be updated, or you need to adjust the paths so that the ones from the compiler toolchains are used instead of the system ones.

[xiahualiu]

This toolchain is customized built for just a single CPU which is core2 type, the compiler itself has the capability to generate AVX code, but it shouldn't since I set -march=core2.

My problem is if I set CFLAGS="-march=core2", I want to disable the auto-detection step in configure, because it will add things like -mavx or -mavx2 that are not supported by the target architecture but supported by the compiler.

[jedisct1]

In configure.ac, the check for AVX usability is the compile _mm256_zeroall();, which may be too simple to trigger your issue with a compiler more recent than the binutils. You can try adding more test code, such as _mm256_storeu_si256() into a volatile pointer.

[xiahualiu]

Ok let me try that, thanks.