Start using readthedocs.io

[jsquyres]

Work in progress to convert README + FAQ + HACKING + ...other docs into Restructured Text so that we can publish on readthedocs.io.

This is envisioned for v5.0 and beyond.

See the latest version built on readthedocs.io here: https://ompi--8329.org.readthedocs.build/en/8329/ .

[rhc54]

This looks simple enough, but I'd really like to know more about the new dependencies. Pandoc turned out to be a real PITA as it isn't as widely available as we thought. I'd prefer we find a new solution that alleviates that pain.

[jsquyres]

Here's what I know so far (and this process is not yet complete):

• ReStructured Text ("RST") is much, much better for writing large documents than Markdown.
  • That being said, RST isn't perfect, either.
  • RST is a little more strict than Markdown, too -- that takes a little getting used to (it's worth it, but there is a bit of a learning curve). For example: like Python, RST uses indenting for blocking/association.
• The end state of this experiment may well be to drop Pandoc as a requirement.
  • That being said, we may well just end up replacing Pandoc with Sphinx. Sphinx is a lot easier to install (it doesn't require a Haskell compiler to build from source, for example), but it still may end up being a new dependency, regardless.

[leofang]

This is awesome!

[jjhursey]

bot:ibm:pgi:retest

[ibm-ompi]

The IBM CI (PGI) build failed! Please review the log, linked below.

Gist: https://gist.github.com/add3ad50faf31d4baa3fe7b033da3170

[ibm-ompi]

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/10d4e30d69ad5c6ccfa7e5bcc443c250

[jjhursey]

IBM Ci had an issue over the weekend. Retrying. bot:ibm:retest

[gpaulsen]

@jsquyres You mentioned in slack we were doing away with the FAQ section, Does that include
https://ompi--8329.org.readthedocs.build/en/8329/faq/index.html ?

[jsquyres]

@gpaulsen Yes. I'm converting over the FAQ section by section. When I'm done, I'll be folding all that content up in to the main part of the document, and the "FAQ" chapter will go away.

Man pages will likely eventually also be added here, too -- @hkuno is working on that part.

[jsquyres]

Squashed everything so far, and rebased to HEAD of master.

[bwbarrett]

bot:aws:retest (updated Jenkins AMIs to include Sphinx, so distcheck should work)

[jsquyres]

Hey @bwbarrett -- have a look:

1. I ended up giving up on trying to use wildcards in the dist_manX_MANS macros: there were subtleties there about Automake rules that I could work around, but I'm not sure my workarounds were portable, reliable, and/or didn't depend on un-documented implementation quirks. So I just ended up listing all of the man pages explicitly.
   • I didn't need to explicitly list the HTML files because Automake doesn't install them; dist-related rules seem to be fine with the wildcards.
2. I did have to list a .NOTPARALLEL target so that make -j N ... doesn't simultaneously invoke the Sphinx rule multiple times (since I list all the generated man page files on the left hand side of the recipe). AFAICT, .NOTPARALLEL is portable between both GNU Make and BSD make, but I'm open to a better solution here.
3. I also used variable substitution like this: OMPI_MAN1_RST = $(OMPI_MAN1:%.1=man-openmpi/man1/%.1.rst). I checked the docs for both GNU and BSD Make, and this appears to be valid in both. Are there other make(1)s that we care about / need to be portable to?

[jsquyres]

@bwbarrett It looks like the Sphinx installed in the Ubuntu 18.04 AMI is too old -- we need Sphinx >= v4.2.0 (@hkuno got a patch accepted in upstream Sphinx to correct the man page rendering, and it first appeared in Sphinx v4.2.0). It looks like the Ubuntu 18.04 AMI has Sphinx v1.8.6 (see the bottom of https://jenkins.open-mpi.org/jenkins/job/open-mpi.build.platforms/10145/Platform=ubuntu_18.04/console).

That being said, it looks like the the failure was in the platform AMI testing Ubuntu 18.04 -- and the AMI for the make distcheck worked just fine. Can we just uninstall Sphinx from the Ubuntu 18.04 AMI...?

[jsquyres]

@bwbarrett I think I have the configury/Makefilery correct now. Could you review again?

(obviously, this will all be squashed down before merging)

[gpaulsen]

@jsquyres How do you feel about rebasing this, and merging soon and addressing @bwbarrett and other content matters on master / v5.0.x ?

[jsquyres]

I plan to return to this PR during my next availability window: this upcoming weekend.

I think that merging this PR before all the content is 100% complete is fine, but I really need to address the configury stuff before it can merge to master (because configury stuff affects anyone who tries to build).

[jsquyres]

bot:aws:retest

[jsquyres]

CI found an issue where I needed to add extra quoting in a configure regexp that Brian+me added earlier this afternoon. Fixed / re-pushed.

[jsquyres]

This PR is now "ready", where "ready" is defined as:

• It's been reviewed by Brian and me
• It seems to pass all the configury cases that we care about

I'm sure that as more people start using Sphinx, we're going to run into some more bugs with the configury -- but we're at the limit of what Brian and I can test by ourselves.

Additionally, we know that the content is not yet complete. But we need to prevent things like #10079 from happening (people continuing to edit the old docs) -- it's time to merge the initial implementation to master to have everyone starting using the new docs, and then continue working on the content with regular pull requests.

We'll probably let this bake on master for a while before PR'ing it to the v5.0.x branch.