You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some large supercomputing systems have IBM Spectrum Scale (a.k.a GPFS) for the parallel file-system. The code in openmpi-4.1.2 for GPFS support in romio321 has one minor issue. It will not compile unless one changes "stat64" to "stat" in source file ad_gpfs_open.c. To enable GPFS support in the current openmpi's romio321 path, one must pass the following option to the top-level configure script : configure ... --with-io-romio-flags='--with-file-system=gpfs', then "make" will attempt to build support for GPFS. Doing this helps significantly with MPI-IO performance on systems that have GPFS, when adding -mca io romio321 to the mpirun command. My experience has been that romio321 when properly configured provides better performance on GPFS than the current "ompio" layer. I am familiar with this issue because I was involved in getting romio321 support into IBM' Spectrum MPI for the Summit system at Oak Ridge.
I mention that current versions of MPICH have somewhat newer ROMIO code for GPFS, which does not have this issue. I don't think it is necessary to update the GPFS code in openmpi, other than to fix the issue with stat64 => stat in ad_gpfs_open.c .
The text was updated successfully, but these errors were encountered:
Yeah, I checked and it looks like both v4.0.x and v4.1.x are using romio321 with a full copy in OMPI. I know at some point there's a goal to have newer OMPI just pulling romio straight from mpich master, but I don't think v4.0.x / v4.1.x will be doing that. In any case they're not yet doing that. So I believe these trees are still taking one-off fixes into our romio321 copy.
So I made a couple PRs for changing stat64 to stat: #9854 #9855
Some large supercomputing systems have IBM Spectrum Scale (a.k.a GPFS) for the parallel file-system. The code in openmpi-4.1.2 for GPFS support in romio321 has one minor issue. It will not compile unless one changes "stat64" to "stat" in source file ad_gpfs_open.c. To enable GPFS support in the current openmpi's romio321 path, one must pass the following option to the top-level configure script : configure ... --with-io-romio-flags='--with-file-system=gpfs', then "make" will attempt to build support for GPFS. Doing this helps significantly with MPI-IO performance on systems that have GPFS, when adding -mca io romio321 to the mpirun command. My experience has been that romio321 when properly configured provides better performance on GPFS than the current "ompio" layer. I am familiar with this issue because I was involved in getting romio321 support into IBM' Spectrum MPI for the Summit system at Oak Ridge.
I mention that current versions of MPICH have somewhat newer ROMIO code for GPFS, which does not have this issue. I don't think it is necessary to update the GPFS code in openmpi, other than to fix the issue with stat64 => stat in ad_gpfs_open.c .
The text was updated successfully, but these errors were encountered: