You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
$ kubectl exec -it openssl-qat-engine bash
$ openssl engine -c -t -v qatengine
(qatengine) Reference implementation of QAT crypto engine(qat_hw & qat_sw) v1.4.0
[RSA, AES-128-CBC-HMAC-SHA256, AES-256-CBC-HMAC-SHA256, ChaCha20-Poly1305, id-aes128-GCM, id-aes192-GCM, id-aes256-GCM, SHA3-256, SHA3-384, SHA3-512, TLS1-PRF, X25519, X448, SM2]
[ available ]
ENABLE_EXTERNAL_POLLING, POLL, SET_INSTANCE_FOR_THREAD,
GET_NUM_OP_RETRIES, SET_MAX_RETRY_COUNT, SET_INTERNAL_POLL_INTERVAL,
GET_EXTERNAL_POLLING_FD, ENABLE_EVENT_DRIVEN_POLLING_MODE,
GET_NUM_CRYPTO_INSTANCES, DISABLE_EVENT_DRIVEN_POLLING_MODE,
SET_EPOLL_TIMEOUT, SET_CRYPTO_SMALL_PACKET_OFFLOAD_THRESHOLD,
ENABLE_INLINE_POLLING, ENABLE_HEURISTIC_POLLING,
GET_NUM_REQUESTS_IN_FLIGHT, INIT_ENGINE, SET_CONFIGURATION_SECTION_NAME,
ENABLE_SW_FALLBACK, HEARTBEAT_POLL, DISABLE_QAT_OFFLOAD, HW_ALGO_BITMAP,
SW_ALGO_BITMAP
80FBC9006E7F0000:error:1280006A:DSO support routines:dlfcn_bind_func:could not bind to the requested symbol name:../crypto/dso/dso_dlfcn.c:188:symname(EVP_PKEY_base_id): /usr/lib/x86_64-linux-gnu/engines-3/qatengine.so: undefined symbol: EVP_PKEY_base_id
80FBC9006E7F0000:error:1280006A:DSO support routines:DSO_bind_func:could not bind to the requested symbol name:../crypto/dso/dso_lib.c:176:
# This works fine
$ openssl speed -engine qatengine -elapsed -async_jobs 8 rsa2048
Engine "qatengine" set.
You have chosen to measure elapsed time instead of user CPU time.
Doing 2048 bits private rsa's for 10s: 199083 2048 bits private RSA's in 10.00s
Doing 2048 bits public rsa's for 10s: 959389 2048 bits public RSA's in 10.00s
version: 3.0.2
built on: Fri Feb 16 08:51:30 2024 UTC
options: bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/openssl-olCZw9/openssl-3.0.2=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_ia32cap=0x7ffef3ffffebffff:0xfb417ffef3bfb7ef
sign verify sign/s verify/s
rsa 2048 bits 0.000050s 0.000010s 19908.3 95938.9
# When setting multiple processes, the error pop up
$ openssl speed -engine qatengine -elapsed -async_jobs 8 -multi 8 rsa2048
Forked child 0
Forked child 1
Forked child 2
Forked child 3
Forked child 4
Forked child 5
Forked child 6
Forked child 7
No devices found
No devices found
No devices found
No device found
No device found
No device found
Engine "qatengine" set.
Engine "qatengine" set.
Engine "qatengine" set.
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
No devices found
No device found
...
Got: +F2:2:2048:6680.100000:83645.800000 from 0
Got: +F2:2:2048:6630.500000:83514.600000 from 1
Don't understand line 'ADF_UIO_PROXY err: icp_adf_userProcessToStart: Failed to start SHIM' from child 2Got: +F2:2:2048:11644.800000:144419.200000 from 2Got: +F2:2:2048:6952.300000:84143.700000 from 3Don't understand line 'ADF_UIO_PROXY err: icp_adf_userProcessToStart: Failed to start SHIM' from child 4
Got: +F2:2:2048:11627.200000:142830.069930 from 4
Don't understand line 'ADF_UIO_PROXY err: icp_adf_userProcessToStart: Failed to start SHIM' from child 5Got: +F2:2:2048:11668.800000:143320.000000 from 5Don't understand line 'ADF_UIO_PROXY err: icp_adf_userProcessToStart: Failed to start SHIM' from child 6
Got: +F2:2:2048:11376.800000:127321.378621 from 6
Don't understand line 'ADF_UIO_PROXY err: icp_adf_userProcessToStart: Failed to start SHIM' from child 7Got: +F2:2:2048:11647.200000:142272.027972 from 7# The result seems also get boosted somehow, but not sure via qat_sw or qat_hwcompiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/openssl-olCZw9/openssl-3.0.2=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2CPUINFO: OPENSSL_ia32cap=0x7ffef3ffffebffff:0xfb417ffef3bfb7ef sign verify sign/s verify/srsa 2048 bits 0.000013s 0.000001s 78227.7 951466.8
Expected behavior
All 8 processes should be able to use qatengine as 16 instances are passthrough
Screenshots
System (please complete the following information):
OS version: CentOS Stream release 8
Kernel version: 6.8.1-1.el8.elrepo.x86_64
Device plugins version: v0.29.0
Hardware info: CPU: 6454S + intree driver
Additional context
As you can see from the screenshot, not all processes fail to fetch the qat handler, which makes me curious.
What makes the problem more tricky is that if I add privileged: true to pod manifest, everything works fine (i.e., I can create 16 processes when running openssl speed without error info) but I think that may not be used in production env.
The text was updated successfully, but these errors were encountered:
Can you try with 4 jobs? It could be that the qatlib allocation limitation triggers the problem you're seeing. Try settin QAT_POLICY=1 environment variable. If that helps, we'll need to update our docs a bit.
Describe the bug
Some processes report "No devices found" during running
openssl speed
command whenprivilege
is not set to containerTo Reproduce
demo/openssl-qat-engine/Dockerfile
openssl-qat-engine
with below manifestopenssl
Expected behavior
All 8 processes should be able to use qatengine as 16 instances are passthrough
Screenshots
![image](https://private-user-images.githubusercontent.com/102018874/316882920-362ad80b-1f86-46a7-a6a6-582884d11dce.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ2MDA2MTMsIm5iZiI6MTcyNDYwMDMxMywicGF0aCI6Ii8xMDIwMTg4NzQvMzE2ODgyOTIwLTM2MmFkODBiLTFmODYtNDZhNy1hNmE2LTU4Mjg4NGQxMWRjZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODI1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyNVQxNTM4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kNDIxYTc3NDUzM2I1NzUwMjg1NTU0MTQ5OGY0NThjZjljYTAwNzVlMWFkZTBkMWZiYmRlZTJjMmExM2NhZGU2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.QUU0SYBvsH4Wu7nudeF3DOgVOsCexrkkg_obGjZJONY)
![image](https://private-user-images.githubusercontent.com/102018874/316882978-3478a463-2a79-4ced-bda9-76ddd83e1fb2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ2MDA2MTMsIm5iZiI6MTcyNDYwMDMxMywicGF0aCI6Ii8xMDIwMTg4NzQvMzE2ODgyOTc4LTM0NzhhNDYzLTJhNzktNGNlZC1iZGE5LTc2ZGRkODNlMWZiMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODI1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyNVQxNTM4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yNjI4MzNjMzYzNDYzN2RlZDdjNDY3ZDhhZDIyZmMxZTFjYmRhYmUzM2NlYWQ5MmE5OGRhNDc5YTBiM2RmMDljJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.Je7i0slpv5KbmC4neITTpfXMYl-Ava0UrR52n9J3nE4)
System (please complete the following information):
Additional context
As you can see from the screenshot, not all processes fail to fetch the qat handler, which makes me curious.
What makes the problem more tricky is that if I add
![image](https://private-user-images.githubusercontent.com/102018874/316885242-af2ceac6-572c-4548-91d4-73d8d3f28c5f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ2MDA2MTMsIm5iZiI6MTcyNDYwMDMxMywicGF0aCI6Ii8xMDIwMTg4NzQvMzE2ODg1MjQyLWFmMmNlYWM2LTU3MmMtNDU0OC05MWQ0LTczZDhkM2YyOGM1Zi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODI1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyNVQxNTM4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04ZDhlNDlmNjAxZjJiMDE5MjBlOWU1NTcyODExNmUxYzQwMjk5MmE3MWZiY2M5ZTYxNWY0M2UzOTI5ZmNjNjUxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.k939TYLzsdwPLtAB6435x1qJG8-SFXrJ-UhYCRjeebk)
![image](https://private-user-images.githubusercontent.com/102018874/316885301-6f8b49a4-d4e6-40b3-b6f3-7950cf13d301.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ2MDA2MTMsIm5iZiI6MTcyNDYwMDMxMywicGF0aCI6Ii8xMDIwMTg4NzQvMzE2ODg1MzAxLTZmOGI0OWE0LWQ0ZTYtNDBiMy1iNmYzLTc5NTBjZjEzZDMwMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODI1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyNVQxNTM4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lMTcxZTcyZjdiZDI1NWMwODk5OTgxMWJmZGNlMDIwNzFlYTUwYjQwN2Q4OWIyMDM1M2I1NTY0MDc3MDc2YTkzJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.rsyD7KXiD_cCNRSMSH-WIgcyjG-j1kMKka7-3N-WItk)
privileged: true
to pod manifest, everything works fine (i.e., I can create 16 processes when runningopenssl speed
without error info) but I think that may not be used in production env.The text was updated successfully, but these errors were encountered: