add .toml support

[rvhonorato]

We can try to think of a wrapper function somehow. For example, reading the toml plain string, replace the values that differ from the HADDOCK3 specifications, and then reading with the config reader. It is a bit of double work, but we facilitate the life of other projects.

Second, if the other projects write the configs to toml, we can read the toml files without problem, considering the types of values used in HADDOCK3. So, we don't need actually to use the toml lib, maybe only upgrading the config_reader to cope with the extra suffixes.

Originally posted by @joaomcteixeira in #516 (comment)

[joaomcteixeira]

I would like to discuss this before proceeding with the implementation. Please read my comment here: #516 (comment)

Cheers,

[sverhoeven]

In the past I made #473 which made it possible for some other app (aka workflow builder) to write a config file using a toml library.

[rvhonorato]

Maybe we should focus on that one! I don't care if we add the support with a dependency or changing whatever code is there already. Seems like adding the code that its already there should be more straightforward

[joaomcteixeira]

Finally,

We can use toml to read/write the configs if we create an intermediate step where the user config files are edited (internally) to meet the toml specifications. For example:

1. reading: haddock3 reads the user config, edits headers and other stuff to meet toml and the toml reads that string.
2. writing: toml writes the config string from the dictionary, and haddock does the final edits to meet the user config specifications.

I think this way is the more general one that could be applied in every language. I am pasting below again the specifications the user config is now differing from toml:

1. repeated header names. The config_reader don't need them. I think this is the most important one.
2. the config_reader can read None or none values, and nulls. toml can't
3. We have already implemented many internal value conversion formats in the config reader. For example, we have a list of special keys that are automatically converted to Path, and we also have the EmptyPath object.

[rvhonorato]

Is this discussion closed with #473?

[joaomcteixeira]

Partially yes. But not enterily. Things missing:

• why the quotes in the headers with indexes? (discussing this currently with @sverhoeven
• the config_reader reads variables nicely, but I have not made tests to ensure the variable reading is 100% compatible with toml
• toml does not address None and nulls, we use that a lot in HADDOCK3
• It only reads, it does not write. We can extend the config_writer for that purpose.
• before passing the parameters dictionary to the haddock3 machinery, we convert path variables to pathlib.Path. That is something that needs to be addressed if you want to implement pure toml reading. Whatever reads the toml must make the proper format conversions before sending the dictionary to the haddock3 machinery.

I am giving a lot of thought to this issue, and I still think my comment above is the strongest idea. IMHO.

Cheers

[rvhonorato]

Sorry - be more objective.

Does #473 adds the proper toml support or not? If not we must create a new issue and add it. The support for toml is needed.

[sverhoeven]

#473 makes haddock3 a bit more toml compliant.

The toml spec says you can not have repeated [somemodule].
The PR make haddock3 accept a config file with repeated modules in a toml spec compliant format.

Regarding #518 (comment) in the workflow builder I am doing something similar

1. When reading a config file, first the text is preprocessed to replace repeated modules with ['somemodule.<digit>'] then the text is parsed with a TOML library
2. When writing a config file. The used TOML library can write nulls, but will depend on library support.

[rvhonorato]

What is the advantage of using null instead of simply ""?

[joaomcteixeira]

Parameters with None are not written to CNS. I thought empty strings were written to CNS scripts, but it seems that is not the case

haddock3/src/haddock/libs/libcns.py

Line 77 in c4883fb

(lambda x: isinstance(x, str) and not bool(x), false),

nan values are not written to CNS.

[joaomcteixeira]

#539 should address all topics discussed here.
Let me know of any doubts, cheers.