-
Notifications
You must be signed in to change notification settings - Fork 167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
there is no clean way to specify backend-specific args #98
Comments
Generally, I do believe the way "scripts" are handled should be changed alltogether. I do agree, though. I'm working to get this fixed changing the parser to support the "plugin.option" syntax which fixes the issue hopefully for the next release. |
I have updated the parser, see #104. |
Are we good with this, then the pull request can be merged... |
Sorry, I haven't had a lot of time to look at this. I only have two concerns (which you may have addressed):
If both are yes, then I'm in favor of merging it in. |
1) yes.
2) not yet. But bits are already in for some backends. Hence this improves
the situation...
Glenn K. Lockwood <notifications@github.com> schrieb am Fr., 19. Okt. 2018,
19:06:
… Sorry, I haven't had a lot of time to look at this. I only have two
concerns (which you may have addressed):
1.
Does this maintain backwards compatibility with any existing options?
For example, will IO-500 have to be updated?
2.
Does this still support taking input through the script file?
If both are yes, then I'm in favor of merging it in.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#98 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AE1uyj42EhKZzpupZJBkp8XybJaJrYDTks5umhOEgaJpZM4XLNLC>
.
|
As long as it doesn't break input scripts, let's merge it in then. But we should keep this issue open until input scripts and CLI options are both in harmony. If you couldn't tell, I prefer using input scripts over CLI-based job setup, and ensuring that I can continue doing so is a personal priority 🙂 |
This is an issue following the thread on issue #94.
Right now different AIORIs have different methods of exposing backend-specific options. There have been a couple of generalized ways in which users specify these:
ior -a HDF5 -- --collectiveMetadata
ior -a HDF5 --hdf5.collective_md
ior -a HDF5 --custom "hdf5_collective_md=1"
ior -a HDF5 --hdf5-collective-md=1
There is the added problem that the current option parser treats anything input via
-f
as a second-class citizen, which can cause bugs to manifest if the API is specified in the script rather than on the command line.In such cases, the default API (MPI-IO) would be what gets detected by the time
backend->get_options
gets called. If the API is then changed to, say, RADOS in the config script,backend->get_options
will never get called.Fixing this issue and bringing command-line options and configfile options at parity might require some major restructuring because the semantics of the command line are different from the config file.
-f
)RUN
statements that are to use those configurationsThe correct way to handle specification of the backend-specific options would be to immediately do the following (from
ParseCommandLine
) every time the API option is specified:However doing this means that the backend-specific options would have to first be pulled from the remaining CLI and/or configfile options and passed to the backend-specific
option_parse
before resuming the parsing the "universal" command-line or configfile options as before.Unless we want to go down this very complex route (essentially build an entire ordered execution plan in memory before doing anything), we'll have to embed backend-specific options in the API option itself. For example,
This bundles the backend-specific options with the line that should trigger the parsing of these backend-specific options.
Does this seem ridiculous, or should we plan on writing a much more complicated option parser?
The text was updated successfully, but these errors were encountered: