Upgrade pymdown-extensions and markdown (#3053)

* Upgrade pymdown-extensions and markdown

Also merges where python dependencies are specified to
requirements.txt.

Except for entrypoint.sh which needs to be kepy in sync,
added note in requirement.txt for that.

Fixes: #3041

* Update CHANGELOG.md

---------

Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com>

.config/python/dev/requirements.txt

@@ -1,3 +1,4 @@
+# Please check changes with entrypoint.sh and keep in sync
 aiofiles
 azure-devops==6.0.0b4
 beautifulsoup4
@@ -9,14 +10,15 @@ importlib-metadata>=3.10
 json-schema-for-humans
 jsonpickle
 jsonschema
+markdown
 mdx_truly_sane_lists
 mkdocs
 mkdocs-glightbox==0.3.2
 mkdocs-material
 multiprocessing_logging
 pychalk
 pygithub
-pymdown-extensions==9.11
+pymdown-extensions
 pytablewriter
 pytest-cov
 pytest-timeout

.devcontainer/Dockerfile

@@ -11,7 +11,7 @@ RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/
 # [Optional] If your pip requirements rarely change, uncomment this section to add them to the image.
 COPY .config/python/dev/requirements.txt /tmp/pip-tmp/
 RUN pip3 --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.dev.txt \
-   && rm -rf /tmp/pip-tmp && pip3 install --no-cache-dir mkdocs-material pymdown-extensions==9.11 mkdocs-glightbox==0.3.2 pymdown-extensions==9.11
+   && rm -rf /tmp/pip-tmp
 
 RUN npm install markdown-table-formatter -g
 

.github/CONTRIBUTING.md

@@ -114,17 +114,21 @@ Draft pull requests are also welcome to get feedback early on, or if there is so
 
 Apart from the descriptors, it will usually involve modifying files such as [.automation/build.py](https://github.com/oxsecurity/megalinter/blob/main/.automation/build.py)
 
-In order to be able to run locally a server that serves all the documentation and make the testing as real as possible you will have to run at least 2 commands.
+In order to be able to run locally a server that serves all the documentation and make the testing as real as possible you should setup a virtual environment.
 
-Command to execute (only one time):
+Commands to execute (only one time):
 
 ```bash
-pip install --upgrade "markdown==3.3.7" mike mkdocs-material "pymdown-extensions==9.11" "mkdocs-glightbox==0.3.2" mdx_truly_sane_lists jsonschema json-schema-for-humans giturlparse webpreview github-dependents-info
+mkdir venv
+python -m venv venv/
+source venv/bin/activate
+pip install --upgrade -r .config/python/dev/requirements.txt
 ```
 
-Command to run every time you want to bring up the server:
+Commands to run every time you want to enter the environment and run the server:
 
 ```bash
+source venv/bin/activate
 mkdocs serve
 ```
 

.github/workflows/build-deploy-docs.yml

@@ -29,7 +29,7 @@ jobs:
       - uses: actions/setup-python@v4.5.0
         with:
           python-version: 3.10.4
-      - run: pip install --upgrade markdown==3.3.7 mike mkdocs-material pymdown-extensions==9.11 mkdocs-glightbox==0.3.2 mdx_truly_sane_lists json-schema-for-humans giturlparse webpreview
+      - run: pip install --upgrade -r .config/python/dev/requirements.txt
       - run: cd .automation && bash build_schemas_doc.sh && cd ..
       # - run: mkdocs gh-deploy --force
       - run: |

.github/workflows/deploy-RELEASE.yml

@@ -155,7 +155,7 @@ jobs:
       - uses: actions/setup-python@v4.5.0
         with:
           python-version: 3.x
-      - run: pip install --upgrade markdown==3.3.7 mike mkdocs-material pymdown-extensions==9.11 mkdocs-glightbox==0.3.2 mdx_truly_sane_lists json-schema-for-humans giturlparse webpreview
+      - run: pip install --upgrade -r .config/python/dev/requirements.txt
       - run: cd .automation && bash build_schemas_doc.sh && cd ..
       # - run: mkdocs gh-deploy --force
       - run: |

CHANGELOG.md

@@ -10,6 +10,9 @@ Note: Can be used with `oxsecurity/megalinter@beta` in your GitHub Action mega-l
 
 - Allow to use value `any` to always activate a linter who as a **_DIRECTORY** variable. Example: `KUBERNETES_DIRECTORY: any`
 
+- CI
+  - Upgrade pymdown-extensions and markdown, by @BryanQuigley in [#3053](https://github.com/oxsecurity/megalinter/pull/3053)
+
 - Linter versions upgrades
   - [protolint](https://github.com/yoheimuta/protolint) from 0.46.2 to **0.46.3** on 2023-10-29
   - [checkov](https://www.checkov.io/) from 3.0.12 to **3.0.13** on 2023-10-30

entrypoint.sh

@@ -35,7 +35,7 @@ if [ "${UPGRADE_LINTERS_VERSION}" == "true" ]; then
   # Run only get_linter_help test methods
   pytest -v --durations=0 -k _get_linter_help megalinter/
   # Reinstall mkdocs-material because of broken dependency
-  pip3 install --upgrade "markdown==3.3.7" mike mkdocs-material "pymdown-extensions==9.11" "mkdocs-glightbox==0.3.2" mdx_truly_sane_lists jsonschema json-schema-for-humans giturlparse webpreview "github-dependents-info==0.10.0"
+  pip3 install --upgrade -r markdown mike mkdocs-material pymdown-extensions "mkdocs-glightbox==0.3.2" mdx_truly_sane_lists jsonschema json-schema-for-humans giturlparse webpreview github-dependents-info
   cd /tmp/lint || exit 1
   chmod +x build.sh
   GITHUB_TOKEN="${GITHUB_TOKEN}" bash build.sh --doc --dependents --stats

mkdocs.yml

@@ -30,6 +30,7 @@ markdown_extensions:
   - pymdownx.snippets:
       base_path: docs
       check_paths: true
+      restrict_base_path: False
   - mdx_truly_sane_lists
   - attr_list
 extra_javascript: