Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

old code for GPFS support in romio321 needs stat64=>stat in ad_gpfs_open.c #9842

Open
walkup opened this issue Jan 7, 2022 · 2 comments
Open
Assignees

Comments

@walkup
Copy link

walkup commented Jan 7, 2022

Some large supercomputing systems have IBM Spectrum Scale (a.k.a GPFS) for the parallel file-system. The code in openmpi-4.1.2 for GPFS support in romio321 has one minor issue. It will not compile unless one changes "stat64" to "stat" in source file ad_gpfs_open.c. To enable GPFS support in the current openmpi's romio321 path, one must pass the following option to the top-level configure script : configure ... --with-io-romio-flags='--with-file-system=gpfs', then "make" will attempt to build support for GPFS. Doing this helps significantly with MPI-IO performance on systems that have GPFS, when adding -mca io romio321 to the mpirun command. My experience has been that romio321 when properly configured provides better performance on GPFS than the current "ompio" layer. I am familiar with this issue because I was involved in getting romio321 support into IBM' Spectrum MPI for the Summit system at Oak Ridge.

I mention that current versions of MPICH have somewhat newer ROMIO code for GPFS, which does not have this issue. I don't think it is necessary to update the GPFS code in openmpi, other than to fix the issue with stat64 => stat in ad_gpfs_open.c .

@gpaulsen
Copy link
Member

gpaulsen commented Jan 7, 2022

@walkup we can take a look at this.

@gpaulsen gpaulsen assigned markalle and unassigned jjhursey Jan 7, 2022
@markalle
Copy link
Contributor

Yeah, I checked and it looks like both v4.0.x and v4.1.x are using romio321 with a full copy in OMPI. I know at some point there's a goal to have newer OMPI just pulling romio straight from mpich master, but I don't think v4.0.x / v4.1.x will be doing that. In any case they're not yet doing that. So I believe these trees are still taking one-off fixes into our romio321 copy.

So I made a couple PRs for changing stat64 to stat:
#9854
#9855

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants