Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bin/lint-markdown: Only run against git-tracked markdown files #780

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

doshitan
Copy link
Contributor

Changes

Rather than maintaining effectively another ignore list, use the one we already have via git. This also adapts better to usage in child projects, as they will likely have more language/project specific locations to ignore that we can't realistically maintain in this parent template.

Testing

Running normally:

❯ make lint-markdown                               
./bin/lint-markdown

  ERROR: 1 dead links found in docs/decisions/infra/0011-network-layer-design.md !
  [✖] #application-specific-network-configuration → Status: 404

  ERROR: 1 dead links found in docs/infra/environment-variables-and-secrets.md !
  [✖] #secrets → Status: 404

  ERROR: 10 dead links found in docs/infra/style-guide.md !
  [✖] #style-guide → Status: 404
  [✖] #table-of-contents → Status: 404
  [✖] #terraform-code-style → Status: 404
  [✖] #exceptions-and-additions-to-hashicorps-terraform-style-guide → Status: 404
  [✖] #modules → Status: 404
  [✖] #variables → Status: 404
  [✖] #gitignore → Status: 404
  [✖] #integration-and-unit-testing → Status: 404
  [✖] #policy → Status: 404
  [✖] #shell-script-style → Status: 404
./bin/lint-markdown: line 22: unexpected EOF while looking for matching `"'
make: *** [Makefile:230: lint-markdown] Error 123

Running with set -x on the script and -t to xargs for what it's doing:

❯ make lint-markdown                                                                                                                                                                                               
./bin/lint-markdown                                                                                                                                                                                                
++ readlink -f ./bin/lint-markdown                                                                                                                                                                                 
++ xargs dirname                                                                                                                                                                                                   
+ script_dir=/home/doshitan/projects/template-infra/bin                                                                                                                                                            
+ cd /home/doshitan/projects/template-infra/bin/..                                                                                                                                                                 
+ link_check_config=.github/workflows/markdownlint-config.json                                                                                                                                                     
+ command -v markdown-link-check                                                                                                                                                                                   
+ link_check_cmd=markdown-link-check                                                                                                                                                                               
+ git ls-files -z '*.md'                                                                                                                                                                                           
+ xargs -t -0 '-I{}' sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json {}'                                                                                                    
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json .github/pull_request_template.md'                                                                                           
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json .github/workflows/README.md'                                                                                                
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json CONTRIBUTING.md'                                                                                                            
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json LICENSE.md'                                                                                                                 
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json README.md'                                                                                                                  
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/code-reviews.md'                                                                                                       
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/compliance.md'                                                                                                         
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/index.md'                                                                                                    
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0000-use-markdown-architectural-decision-records.md'                                                   
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0001-ci-cd-interface.md'                                                                               
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0002-use-custom-implementation-of-github-oidc.md'                                                      
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0003-manage-ecr-in-prod-account-module.md'                                                             
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0004-separate-terraform-backend-configs-into-separate-config-files.md'                                 
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0005-separate-database-infrastructure-into-separate-layer.md'                                          
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0006-provision-database-users-with-serverless-function.md'                                             
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0007-database-migration-architecture.md'                                                               
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0008-consolidate-infra-config-from-tfvars-files-into-config-module.md'                                 
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0009-separate-app-infrastructure-into-layers.md'                                                       
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0010-feature-flags-system-design.md'                                                                   
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/infra/0011-network-layer-design.md'                                                                          
  
  ERROR: 1 dead links found in docs/decisions/infra/0011-network-layer-design.md !
  [✖] #application-specific-network-configuration → Status: 404
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/decisions/template.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/e2e/e2e-checks.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/feature-flags.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/background-jobs.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/cloud-access-control.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/database-access-control.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/destroy-infrastructure.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/develop-and-test-infrastructure-in-isolation-using-workspaces.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/environment-variables-and-secrets.md'

  ERROR: 1 dead links found in docs/infra/environment-variables-and-secrets.md !
  [✖] #secrets → Status: 404
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/https-support.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/infrastructure-configuration.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/making-infra-changes.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/module-architecture.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/module-dependencies.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/pull-request-environments.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/service-command-execution.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-app-build-repository.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-app-env.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-aws-account.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-custom-domains.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-database.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-infrastructure-tools.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-monitoring-alerts.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-network.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/set-up-public-internet-access.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/style-guide.md'

  ERROR: 10 dead links found in docs/infra/style-guide.md !
  [✖] #style-guide → Status: 404
  [✖] #table-of-contents → Status: 404
  [✖] #terraform-code-style → Status: 404
  [✖] #exceptions-and-additions-to-hashicorps-terraform-style-guide → Status: 404
  [✖] #modules → Status: 404
  [✖] #variables → Status: 404
  [✖] #gitignore → Status: 404
  [✖] #integration-and-unit-testing → Status: 404
  [✖] #policy → Status: 404
  [✖] #shell-script-style → Status: 404
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/upgrade-database.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/infra/vulnerability-management.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/releases.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json docs/system-architecture.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json infra/README.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json infra/modules/auth-github-actions/README.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json infra/modules/terraform-backend-s3/README.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json infra/project-config/README.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/application-requirements.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/set-up-cd.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/set-up-ci.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/set-up-pr-environments.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/set-up-team-workflow.md'
sh -c 'markdown-link-check --quiet --config .github/workflows/markdownlint-config.json template-only-docs/template-development-workflow.md'

Rather than maintaining effectively another ignore list, use the one we
already have via git. This also adapts better to usage in child
projects, as they will likely have more language/project specific
locations to ignore that we can't realistically maintain in this parent
template.
# Recursively find all markdown files (*.md) in the current directory, excluding node_modules and .venv subfolders.
# Pass them in as args to the lint command using the handy `xargs` command.
find . -name \*.md -not -path "*/node_modules/*" -not -path "*/.venv/*" -not -path "*/.terraform/*" -print0 | xargs -0 -I{} sh -c "${link_check_cmd} --quiet --config ${link_check_config} {}"
git ls-files -z '*.md' | xargs -0 -I{} sh -c "${link_check_cmd} --quiet --config ${link_check_config} {}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional nit: I think it's still useful to include a comment about what this does

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants