-
Notifications
You must be signed in to change notification settings - Fork 168
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #231 from hpc/master
Merging master back into RC
- Loading branch information
Showing
65 changed files
with
5,620 additions
and
2,212 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| Package: ior | ||
| Version: 3.2.0 | ||
| Version: 3.3.0+dev | ||
| Release: 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,31 +1,31 @@ | ||
| # HPC IO Benchmark Repository [](https://travis-ci.org/hpc/ior) | ||
|
|
||
| This repo now contains both IOR and mdtest. | ||
| See also NOTES.txt | ||
| This repository contains the IOR and mdtest parallel I/O benchmarks. The | ||
| [official IOR/mdtest documention][] can be found in the `docs/` subdirectory or | ||
| on Read the Docs. | ||
|
|
||
| # Building | ||
| ## Building | ||
|
|
||
| 0. If "configure" is missing from the top level directory, you | ||
| probably retrieved this code directly from the repository. | ||
| Run "./bootstrap". | ||
| 1. If `configure` is missing from the top level directory, you probably | ||
| retrieved this code directly from the repository. Run `./bootstrap` | ||
| to generate the configure script. Alternatively, download an | ||
| [official IOR release][] which includes the configure script. | ||
|
|
||
| If your versions of the autotools are not new enough to run | ||
| this script, download and official tarball in which the | ||
| configure script is already provided. | ||
| 1. Run `./configure`. For a full list of configuration options, use | ||
| `./configure --help`. | ||
|
|
||
| 1. Run "./configure" | ||
| 2. Run `make` | ||
|
|
||
| See "./configure --help" for configuration options. | ||
| 3. Optionally, run `make install`. The installation prefix | ||
| can be changed via `./configure --prefix=...`. | ||
|
|
||
| 2. Run "make" | ||
| ## Testing | ||
|
|
||
| 3. Optionally, run "make install". The installation prefix | ||
| can be changed as an option to the "configure" script. | ||
| * Run `make check` to invoke the unit tests. | ||
| * More comprehensive functionality tests are included in `testing/`. These | ||
| scripts will launch IOR and mdtest via MPI. | ||
| * Docker scripts are also provided in `testing/docker/` to test various | ||
| distributions at once. | ||
|
|
||
| # Testing | ||
|
|
||
| Run "make check" to invoke the unit test framework of Automake. | ||
|
|
||
| * To run basic functionality tests that we use for continuous integration, see ./testing/ | ||
| * There are docker scripts provided to test various distributions at once. | ||
| * See ./testing/docker/ | ||
| [official IOR release]: https://github.com/hpc/ior/releases | ||
| [official IOR/mdtest documention]: http://ior.readthedocs.org/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| Building | ||
| ---------------------- | ||
|
|
||
| The DAOS library must be installed on the system. | ||
|
|
||
| ./bootstrap | ||
| ./configure --prefix=iorInstallDir --with-daos=DIR --with-cart=DIR | ||
|
|
||
| One must specify "--with-daos=/path/to/daos/install and --with-cart". When that | ||
| is specified the DAOS and DFS driver will be built. | ||
|
|
||
| The DAOS driver uses the DAOS API to open a container (or create it if it | ||
| doesn't exist first) then create an array object in that container (file) and | ||
| read/write to the array object using the daos Array API. The DAOS driver works | ||
| with IOR only (no mdtest support yet). The file name used by IOR (passed by -o | ||
| option) is hashed to an object ID that is used as the array oid. | ||
|
|
||
| The DFS (DAOS File System) driver creates an encapsulated namespace and emulates | ||
| the POSIX driver using the DFS API directly on top of DAOS. The DFS driver works | ||
| with both IOR and mdtest. | ||
|
|
||
| Running with DAOS API | ||
| --------------------- | ||
|
|
||
| ior -a DAOS [ior_options] [daos_options] | ||
|
|
||
| In the IOR options, the file name should be specified as a container uuid using | ||
| "-o <container_uuid>". If the "-E" option is given, then this UUID shall denote | ||
| an existing container created by a "matching" IOR run. Otherwise, IOR will | ||
| create a new container with this UUID. In the latter case, one may use | ||
| uuidgen(1) to generate the UUID of the new container. | ||
|
|
||
| The DAOS options include: | ||
|
|
||
| Required Options: | ||
| --daos.pool <pool_uuid>: pool uuid to connect to (has to be created beforehand) | ||
| --daos.svcl <pool_svcl>: pool svcl list (: separated) | ||
| --daos.cont <cont_uuid>: container for the IOR files/objects (can use `uuidgen`) | ||
|
|
||
| Optional Options: | ||
| --daos.group <group_name>: group name of servers with the pool | ||
| --daos.chunk_size <chunk_size>: Chunk size of the array object controlling striping over DKEYs | ||
| --daos.destroy flag to destory the container on finalize | ||
| --daos.oclass <object_class>: specific object class for array object | ||
|
|
||
| Examples that should work include: | ||
|
|
||
| - "ior -a DAOS -w -W -o file_name --daos.pool <pool_uuid> --daos.svcl <svc_ranks>\ | ||
| --daos.cont <cont_uuid>" | ||
|
|
||
| - "ior -a DAOS -w -W -r -R -o file_name -b 1g -t 4m \ | ||
| --daos.pool <pool_uuid> --daos.svcl <svc_ranks> --daos.cont <cont_uuid>\ | ||
| --daos.chunk_size 1024 --daos.oclass R2" | ||
|
|
||
| Running with DFS API | ||
| --------------------- | ||
|
|
||
| ior -a DFS [ior_options] [dfs_options] | ||
| mdtest -a DFS [mdtest_options] [dfs_options] | ||
|
|
||
| Required Options: | ||
| --dfs.pool <pool_uuid>: pool uuid to connect to (has to be created beforehand) | ||
| --dfs.svcl <pool_svcl>: pool svcl list (: separated) | ||
| --dfs.cont <co_uuid>: container uuid that will hold the encapsulated namespace | ||
|
|
||
| Optional Options: | ||
| --dfs.group <group_name>: group name of servers with the pool | ||
| --dfs.chunk_size <chunk_size>: Chunk size of the files | ||
| --dfs.destroy flag to destory the container on finalize | ||
| --dfs.oclass <object_class>: specific object class for files | ||
|
|
||
| In the IOR options, the file name should be specified on the root dir directly | ||
| since ior does not create directories and the DFS container representing the | ||
| encapsulated namespace is not the same as the system namespace the user is | ||
| executing from. | ||
|
|
||
| Examples that should work include: | ||
| - "ior -a DFS -w -W -o /test1 --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" | ||
| - "ior -a DFS -w -W -r -R -o /test2 -b 1g -t 4m -C --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" | ||
| - "ior -a DFS -w -r -o /test3 -b 8g -t 1m -C --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" | ||
|
|
||
| Running mdtest, the user needs to specify a directory with -d where the test | ||
| tree will be created. Some examples: | ||
| - "mdtest -a DFS -n 100 -F -D -d /bla --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" | ||
| - "mdtest -a DFS -n 1000 -F -C -d /bla --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" | ||
| - "mdtest -a DFS -I 10 -z 5 -b 2 -L -d /bla --dfs.pool <pool_uuid> --dfs.svcl <svc_ranks> --dfs.cont <co_uuid>" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.