[INFRA] Auto adjust table fences before PDF conversion

[sebastientourbier]

This PR is dedicated to correct automatically the dashes and align table fences before generation of the PDF.
It takes over PR #553 (where the intention was a manual fix) and is dedicated to close #549

[sebastientourbier]

@sappelhoff I opened this PR to address automatically the table alignment issue in PDFs.
I implemented two functions in the pandoc_script.py that correct and rewrite the markdown files before the generation of the PDF. At this stage, this should fix most all tables in the generated PDF. Just the build_docs_pdf job fails as I had to use numpy which should be install on circleci. What would be the best way?

[sappelhoff]

this should fix most all tables in the generated PDF

awesome!

What would be the best way?

you can add numpy to https://github.com/bids-standard/bids-specification/blob/master/requirements.txt

let me correct that. Look at these lines in .circleci/config.yml:

bids-specification/.circleci/config.yml

Lines 8 to 10 in d61b1aa

	- run:
	command: |
	pip install --upgrade pip

you add the same syntax in the build_docs_pdf job, and use pip to install numpy

That is, insert the following lines:

      - run:
          command: |
            pip install --upgrade pip
            pip install numpy

directly after these lines:

bids-specification/.circleci/config.yml

Lines 46 to 48 in d61b1aa

	steps:
	- checkout:
	path: ~/bids-specification

[sappelhoff]

I see a lot of good progress here! Some remarks below.

[sappelhoff]

where do these numbers come from?

[sebastientourbier]

This is indeed really not trivial to readjust the tables and these offsets are very useful in adjusting columns where a few or a lot of characters are present. This would be ideal to get rid of them but I have not ended up with a golden rule for the moment. Any idea?

[sappelhoff]

some nitpicks with the docstrings, but I think that style is important and that's how it's done in many big python packages. :-)

[sebastientourbier]

@sappelhoff Seems at this point, tables are properly rendered in the PDF and all suggestions that you made have been incorporated 👍

[sappelhoff]

The entity table still seems a bit buggy, do you know why? --> https://3448-150465237-gh.circle-artifacts.com/0/bids-spec.pdf

I see now why the entity table is still buggy :-) --> I think you can fix that one as well, because with the automatic script we don't run into the problems I anticipated earlier when you were still doing manual fixes.

also, the "submultiples" in the units section (appendix VI)

[sappelhoff]

One more review - note that I have made several code suggestions. You can directly merge them from the GitHub GUI using these buttons (screenshot):

just make sure that after you have done that, do a git pull before continuing to work locally on the code, otherwise you will create a merge conflict.

[sebastientourbier]

@sappelhoff I created the functions as suggested and re-factorized the code accordingly. I also removed 04-entity-table.md from the exclude_files list. This however do not fix the entity table. I think the main problem here is a number of 16 columns is far too big to correctly render this table in portrait layout. A solution could be to adopt a paysage layout but I guess for that we would need to edit the latex file generated by pandoc before it builds the PDF. Not sure if it is possible and/or how to do. @Arshitha any idea?

[sebastientourbier]

@sappelhoff Seems my suggestion #562 (comment) (PR #562) works to fix #551 regarding emojis.
See the generated PDF: https://3545-150465237-gh.circle-artifacts.com/0/bids-spec.pdf
I will revert my changes here thought to not create a mess.

[sebastientourbier]

@sappelhoff This should resolve all remaining opened concerns 👍

[sappelhoff]

@sebastientourbier do you know why from dd6545e onwards the diff is completely unreadable?

[sappelhoff]

I checked the output and only one table was screwed up (and the entity table, but that's fine).

for that single table, I submitted a fix in adbde72

Overall only 3 minor style comments, then I think we can merge this.

[sappelhoff]

great :-) Thanks a lot @sebastientourbier