ERROR RdmaNode: libdisni not found! It must be installed within the java.library.path on each Executor and Driver instance

[RummySugar]

can you give a more detailed configuration method?

[petro-rudenko]

Installation:

1. Download spark-rdma release
2. In archive you'll find libdisni.so
3. Put it on every executor in the same folder. Let it be '/usr/lib/disni`
4. Add to spark next configurations:

spark.driver.extraJavaOptions         -Djava.library.path=/usr/lib/disni/
spark.executor.extraJavaOptions    -Djava.library.path=/usr/lib/disni/

Let me know if you need help.

[RummySugar]

Following your steps, my problem is still unsolved.
in my spark-defaults-conf :

spark.driver.extraClassPath /usr/local/spark-rdma/spark-rdma-3.1-for-spark-2.3.0-jar-with-dependencies.jar
spark.executor.extraClassPath /use/local/spark-rdma/spark-rdma-3.1-for-park-2.3.0-jar-with-dependencies.jar
spark.driver.extraJavaOptions -Djava.library.path=/usr/lib/disni/
spark.executor.extraJavaOptions -Djava.library.path=/usr/lib/disni/
spark.shuffle.manager org.apache.spark.shuffle.rdma.RdmaShuffleManager

in my /etc/profile

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/disni/
export JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/usr/lib/disni/
export SPARK_YARN_USER_ENV="JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH,LD_LIBRARY_PATH=$LD_LIBRARY_PATH"

[finalTestin]

the same