use macro to render examples in a "tree" like fashion (#837)

* Initial work. * update code to render simple examples (list and simple dictionaries) * create a separate module for example creation * start applying to spec * remove dead code * apply to MRI page * fix tabulation on scans.tsv template * apply example macro to longitudinal page * apply to common principles * revert changes on scans.tsv templates addressed in #805 * remove extra line after listing directory content * only use python dictionaries for examples rendering * update examples common principles * update example longitudinal page * update examples MRI page * update examples MEG * update examples iEEG * update examples task * update physio example * update examples common data types derivatives * update examples imaging datatypes * change "participant-label" to "label" in derivatives templates * update examples meg file formats * update qmri examples * reduce width prefixes * add a flag to be able to not use "pipes" prefix * switch use_pipe flag to off when building pdf * create an examplecode module * add demo jupyter notebook for tree example * add doc in code * add doc to CONTRIBUTING * remove non macro examples * fix typos (comment padding and missing folders) * temporary commit * remove comment * fix import * fix markdown linting errors * Apply suggestions from code review Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org> * Apply suggestions from code review Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org> * remove jupyter notebook output * revert changes to import in schema code init * revert change of import of main in macro module this style of import is required for the building of the pdf Co-authored-by: Taylor Salo <tsalo006@fiu.edu> Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>
bids-standard · Aug 31, 2021 · f21e9b2 · f21e9b2
1 parent 8345d50
commit f21e9b2
Show file tree

Hide file tree

Showing 22 changed files with 1,248 additions and 607 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -228,12 +228,11 @@ to render parts of the BIDS specification from the BIDS schema.
 Macros make it easy to achieve a consistent style throughout the specification,
 and changing a given macro will automatically change all appropriate paragraphs in the specification.
 
-
 For example, all tables on BIDS metadata are generated via macros that make use of data in the
 [yaml files](src/schema/metadata) in the [schema](src/schema/README.md).
 
-The macros are written in Python
-(see the folders [tools/schemacode](tools/schemacode) and [tools/mkdocs_macros_bidsschema](tools/mkdocs_macros_bidsschema)),
+These macros are written in Python
+(see the folders [tools/schemacode](tools/schemacode) and [tools/mkdocs_macros_bids](tools/mkdocs_macros_bids)),
 and are called directly in the Markdown document where you want the output of the macro to be inserted.
 
 For example:
@@ -267,6 +266,39 @@ by specifying it in the macro call:
 ) }}
 ```
 
+### Writing folder content examples
+
+We also use macros to have a consistent style to render the examples of folder contents.
+
+These code for these macros are in the folder [tools/schemacode](tools/schemacode).
+
+To insert examples in the code you have make calls to the macro like this:
+
+```
+{{ MACROS___make_filetree_example(
+
+   {
+   "sub-01": {
+      "func": {
+         "sub-control01_task-nback_bold.json": "",
+         },
+      }
+   }
+
+) }}
+```
+
+And this will be turned into this.
+
+```Text
+└─ sub-01/
+   └─ func/
+      └─ sub-control01_task-nback_bold.json 
+```
+
+When you have complex files and folder structure, we suggest you use this 
+[Jupyter notebook](tools/filetree_example.ipynb) for sandboxing your example 
+before you insert the macro call into the markdown document.
 
 ## Building the specification using mkdocs
 

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -19,7 +19,7 @@ plugins:
            +extra_css:
              - css/watermark.css
     - macros:
-        module_name: tools/mkdocs_macros_bidsschema/main
+        module_name: tools/mkdocs_macros_bids/main
 docs_dir: 'src'
 use_directory_urls: false
 nav:

diff --git a/pdf_build_src/process_markdowns.py b/pdf_build_src/process_markdowns.py
@@ -16,7 +16,7 @@
 import numpy as np
 
 sys.path.append("../tools/")
-from schemacode import macros  # noqa   (used in "eval" call later on)
+from mkdocs_macros_bids import macros  # noqa   (used in "eval" call later on)
 
 
 def run_shell_cmd(command):
@@ -453,6 +453,12 @@ def process_macros(duplicated_src_dir_path):
                     "MACROS___",
                     "macros."
                 )
+                # switch "use_pipe" flag OFF to render examples
+                if "make_filetree_example" in function_string:
+                    function_string = function_string.replace(
+                    ")",
+                    ", False)"
+                    )
                 # Run the function to get the output
                 new = eval(function_string)
                 # Replace the code snippet with the function output

diff --git a/src/02-common-principles.md b/src/02-common-principles.md
@@ -254,21 +254,24 @@ However, in the case that these data are to be included:
 
 Alternatively one can organize their data in the following way
 
-```Text
-my_dataset/
-  sourcedata/
-    ...
-  rawdata/
-    dataset_description.json
-    participants.tsv
-    sub-01/
-    sub-02/
-    ...
-  derivatives/
-    pipeline_1/
-    pipeline_2/
-    ...
-```
+{{ MACROS___make_filetree_example(
+    {
+    "my_dataset-1": {
+            "sourcedata": "",
+            "...": "",
+            "sourcedata": {
+                "sub-01": {},
+                "sub-02": {},
+                "...": "",
+            },
+            "derivatives": {
+                "pipeline_1": {},
+                "pipeline_2": {},
+                "...": "",
+            },
+        }
+    }
+) }}
 
 In this example, where `sourcedata` and `derivatives` are not nested inside
 `rawdata`, **only the `rawdata` subfolder** needs to be a BIDS-compliant
@@ -344,23 +347,25 @@ Derivatives can be stored/distributed in two ways:
 
     Example of a derivative dataset including the raw dataset as source:
 
-    ```Plain
-    my_processed_data/
-      code/
-        processing_pipeline-1.0.0.img
-        hpc_submitter.sh
-        ...
-      sourcedata/
-        dataset_description.json
-        participants.tsv
-        sub-01/
-        sub-02/
-        ...
-      dataset_description.json
-      sub-01/
-      sub-02/
-      ...
-    ```
+    {{ MACROS___make_filetree_example(
+        {
+        "my_processed_data": {
+            "code": {
+                "processing_pipeline-1.0.0.img": "",
+                "hpc_submitter.sh": "",
+                "...": "",
+            },
+            "sourcedata": {
+                "sub-01": {},
+                "sub-02": {},
+                "...": "",
+            },
+            "sub-01": {},
+            "sub-02": {},
+            "...": "",
+            }
+        }
+    ) }}
 
 Throughout this specification, if a section applies particularly to derivatives,
 then Case 1 will be assumed for clarity in templates and examples, but removing
@@ -403,17 +408,23 @@ specific to a participant is to be declared only at top level of dataset for exa
 
 Example 1: Two JSON files that are erroneously at the same level
 
-```Text
-sub-01/
-    ses-test/
-        sub-01_ses-test_task-overtverbgeneration_bold.json
-        sub-01_ses-test_task-overtverbgeneration_run-2_bold.json
-        anat/
-            sub-01_ses-test_T1w.nii.gz
-        func/
-            sub-01_ses-test_task-overtverbgeneration_run-1_bold.nii.gz
-            sub-01_ses-test_task-overtverbgeneration_run-2_bold.nii.gz
-```
+{{ MACROS___make_filetree_example(
+    {
+    "sub-01": {
+        "ses-test":{
+            "sub-01_ses-test_task-overtverbgeneration_bold.json": "",
+            "sub-01_ses-test_task-overtverbgeneration_run-2_bold.json": "",
+            "anat": {
+                "sub-01_ses-test_T1w.nii.gz": "",
+                },
+            "func": {
+                "sub-01_ses-test_task-overtverbgeneration_run-1_bold.nii.gz": "",
+                "sub-01_ses-test_task-overtverbgeneration_run-2_bold.nii.gz": "",
+                }
+            }
+        }
+    }
+) }}
 
 In the above example, two JSON files are listed under `sub-01/ses-test/`, which
 are each applicable to
@@ -424,16 +435,20 @@ should have been under `sub-01/ses-test/func/`.
 
 Example 2: Multiple `run`s and `rec`s with same acquisition (`acq`) parameters
 
-```Text
-sub-01/
-    anat/
-    func/
-        sub-01_task-xyz_acq-test1_run-1_bold.nii.gz
-        sub-01_task-xyz_acq-test1_run-2_bold.nii.gz
-        sub-01_task-xyz_acq-test1_rec-recon1_bold.nii.gz
-        sub-01_task-xyz_acq-test1_rec-recon2_bold.nii.gz
-        sub-01_task-xyz_acq-test1_bold.json
-```
+{{ MACROS___make_filetree_example(
+    {
+    "sub-01": {
+        "anat": {},
+        "func": {
+            "sub-01_task-xyz_acq-test1_run-1_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_run-2_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_rec-recon1_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_rec-recon2_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_bold.json": "",
+            }
+        }
+    }
+) }}
 
 For the above example, all NIfTI files are acquired with same scanning
 parameters (`acq-test1`). Hence a JSON file describing the acq parameters will
@@ -443,16 +458,20 @@ will be applicable to all task runs with `test1` acquisition parameter.
 
 Example 3: Multiple JSON files at different levels for same task and acquisition parameters
 
-```Text
-task-xyz_acq-test1_bold.json
-sub-01/
-    anat/
-    func/
-        sub-01_task-xyz_acq-test1_run-1_bold.nii.gz
-        sub-01_task-xyz_acq-test1_rec-recon1_bold.nii.gz
-        sub-01_task-xyz_acq-test1_rec-recon2_bold.nii.gz
-        sub-01_task-xyz_acq-test1_bold.json
-```
+{{ MACROS___make_filetree_example(
+    {
+    "task-xyz_acq-test1_bold.json": "",
+    "sub-01": {
+        "anat": {},
+        "func": {
+            "sub-01_task-xyz_acq-test1_run-1_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_rec-recon1_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_rec-recon2_bold.nii.gz": "",
+            "sub-01_task-xyz_acq-test1_bold.json": "",
+            }
+        }
+    }
+) }}
 
 In the above example, the fields from the `task-xyz_acq-test1_bold.json` file
 at the top directory will apply to all bold runs. However, if there is a key
@@ -732,37 +751,44 @@ This is an example of the folder and file structure. Because there is only one
 session, the session level is not required by the format. For details on
 individual files see descriptions in the next section:
 
-```Text
-sub-control01/
-    sub-control01_scans.tsv
-    anat/
-        sub-control01_T1w.nii.gz
-        sub-control01_T1w.json
-        sub-control01_T2w.nii.gz
-        sub-control01_T2w.json
-    func/
-        sub-control01_task-nback_bold.nii.gz
-        sub-control01_task-nback_bold.json
-        sub-control01_task-nback_events.tsv
-        sub-control01_task-nback_physio.tsv.gz
-        sub-control01_task-nback_physio.json
-        sub-control01_task-nback_sbref.nii.gz
-    dwi/
-        sub-control01_dwi.nii.gz
-        sub-control01_dwi.bval
-        sub-control01_dwi.bvec
-    fmap/
-        sub-control01_phasediff.nii.gz
-        sub-control01_phasediff.json
-        sub-control01_magnitude1.nii.gz
-code/
-    deface.py
-derivatives/
-README
-participants.tsv
-dataset_description.json
-CHANGES
-```
+{{ MACROS___make_filetree_example(
+    {
+    "sub-control01": {
+        "anat":{
+            "sub-control01_T1w.nii.gz": "",
+            "sub-control01_T1w.json": "",
+            "sub-control01_T2w.nii.gz": "",
+            "sub-control01_T2w.json": "",
+            },
+        "func":{
+            "sub-control01_task-nback_bold.nii.gz": "",
+            "sub-control01_task-nback_bold.json": "",
+            "sub-control01_task-nback_events.tsv": "",
+            "sub-control01_task-nback_physio.tsv.gz": "",
+            "sub-control01_task-nback_physio.json": "",
+            "sub-control01_task-nback_sbref.nii.gz": "",
+            },
+        "dwi":{
+            "sub-control01_dwi.nii.gz": "",
+            "sub-control01_dwi.bval": "",
+            "sub-control01_dwi.bvec": "",
+            },
+        "fmap":{
+            "sub-control01_phasediff.nii.gz": "",
+            "sub-control01_phasediff.json": "",
+            "sub-control01_magnitude1.nii.gz": "",
+            }
+        },
+    "code": {
+        "deface.py": ""
+        },
+    "derivatives": {},
+    "README": "",
+    "participants.tsv": "",
+    "dataset_description.json": "",
+    "CHANGES": "",
+    }
+) }}
 
 ## Unspecified data