Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automated JSON schema validation #8039

Merged
merged 18 commits into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/validate-theme.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Validate Themes

on:
pull_request:
paths:
- '**/theme.json'
- '**/styles/*.json'
- '**/assets/fonts/*.json'

env:
PR_PATHS: >-
'**/theme.json'
'**/styles/*.json'
'**/assets/fonts/*.json'
HUSKY: 0

jobs:
validate-theme:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
cache: 'npm'

- name: Install dependencies
run: npm ci

- name: Validate Themes
run: node theme-utils.mjs validate-theme $(git diff --name-only HEAD^ -- ${{ env.PR_PATHS }} | awk -F/ '{print $1}' | uniq | paste -s -d, -)
124 changes: 124 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,127 @@ If you use a sandbox to test or develop your themes you can use a couple of util
- From the top-level directory, run `node ./theme-utils.mjs clean-sandbox` to bring the public themes GIT repository to a clean state. (This will only matter if your sandbox uses GIT such as how _WordPress.com_ is currently managed.) Alternately you can trigger that as an npm script: `npm run sandbox:clean`. `npm run sandbox:clean-all` will clean the entire sandbox.

- From the top-level directory, run `node ./theme-utils.mjs push-to-sandbox` to push your entire working copy to the public themes folder of your sandbox. Alternately you can trigger this as an npm script: `npm run deploy:push:all` This command will rsync your local copy with the exception of anything in the `.sandbox-ignore` file. You should clean your sandbox before pushing any changes to it. `node ./theme-utils.mjs push-changes-to-sandbox` or `npm run deploy:push:changes` alternatively will push only files that have changed since the last deployment.

## Validation Tools

When you update a theme and open a pull request, the validation tools will automatically run as a check on your PR.

However you can run them locally as well.

```sh-session
$ node theme-utils.mjs help validate-theme

validate-theme [--format=FORMAT] [--color=WHEN] [--table-width=COLUMNS] <array of theme slugs>

Validates a theme against the WordPress theme requirements.

--format=FORMAT

Output format. Possible values: *table*, json, dir.

--color=WHEN

Colorize the output for table or dir formats. The automatic mode only enables colors if
an interactive terminal is detected. Possible values: *auto*, always, never.

--table-width=COLUMNS

Explicitly set the width of the table format instead of determining it automatically.
Will default to 120 if omitted and width cannot be determined automatically.
```

Here's an example of the output.

<img width="828" alt="image" src="https://github.com/user-attachments/assets/987d6c2f-b993-4c16-b678-1757752ea820">

This is how you run validation on a single theme:

```sh-session
$ node theme-utils.mjs validate-theme grammer

Progress: [ 'β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– β– ', 100 ] 1/1

╔═════════════════════════════════════════════════════════════╀══════════════════════════╗
β•‘ ERROR β”‚ Message : missing 'Requi β•‘
β•‘ Theme : Grammer β”‚ res at least' header met β•‘
β•‘ File : style.css β”‚ adata β•‘
β•Ÿβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β•’
β•‘ WARNING β”‚ Actual : undefined β•‘
β•‘ Theme : Grammer β”‚ Expected : 5.9 or greate β•‘
β•‘ File : style.css β”‚ r β•‘
β•‘ β”‚ Message : the 'Requires β•‘
β•‘ β”‚ at least' version does β•‘
β•‘ β”‚ not support theme.json β•‘
β•Ÿβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β•’
β•‘ WARNING β”‚ Instance path : /setting β•‘
β•‘ Theme : Grammer β”‚ s/spacing/spacingScale β•‘
β•‘ File : theme.json β”‚ Schema path : #/defini β•‘
β•‘ Schema : https://schemas.wp.org/trunk/theme.json β”‚ tions/settingsSpacingPro β•‘
β•‘ β”‚ perties/properties/spaci β•‘
β•‘ β”‚ ng/properties/spacingSca β•‘
β•‘ β”‚ le/additionalProperties β•‘
β•‘ β”‚ Keyword : addition β•‘
β•‘ β”‚ alProperties β•‘
β•‘ β”‚ Params : { additi β•‘
β•‘ β”‚ onalProperty: 'theme' } β•‘
β•‘ β”‚ Message : must NOT β•‘
β•‘ β”‚ have additional properti β•‘
β•‘ β”‚ es β•‘
β•Ÿβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β•’
β•‘ WARNING β”‚ Instance path : β•‘
β•‘ Theme : Grammer β”‚ Schema path : #/requir β•‘
β•‘ File : assets/fonts/font-collection.json β”‚ ed β•‘
β•‘ Schema : https://schemas.wp.org/wp/6.5/font-collection.json β”‚ Keyword : required β•‘
β•‘ β”‚ Params : { missin β•‘
β•‘ β”‚ gProperty: 'slug' } β•‘
β•‘ β”‚ Message : must hav β•‘
β•‘ β”‚ e required property 'slu β•‘
β•‘ β”‚ g' β•‘
β•‘ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β•’
β•‘ β”‚ Instance path : β•‘
β•‘ β”‚ Schema path : #/requir β•‘
β•‘ β”‚ ed β•‘
β•‘ β”‚ Keyword : required β•‘
β•‘ β”‚ Params : { missin β•‘
β•‘ β”‚ gProperty: 'name' } β•‘
β•‘ β”‚ Message : must hav β•‘
β•‘ β”‚ e required property 'nam β•‘
β•‘ β”‚ e' β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•

Validation passed with warnings.
```

<img width="1728" alt="image" src="https://github.com/user-attachments/assets/68e38c35-224b-407c-8333-b4ac0b881fd8">

It works with pagers. This example shows how to match the table width to your terminal width inside a pager.

```sh-session
$ # You may need to wait a while for longer lists of themes.
$ node theme-utils.mjs validate-theme --color=always --table-width=$(( $(tput cols) )) atheme,adventurer,grammer,skatepark | less -R
```

The added `--format=json` option is super helpful when combined with [`jq`](https://jqlang.github.io/jq/download/) to drill down into the data.

This, for example, is the breakdown of all the current themes and a count of the types of problems that they have:

```sh-session
$ # Scroll to see the long command β†’
$ node theme-utils.mjs validate-theme --format=json $(find . -name 'theme.json' | awk -F/ '{print $2}' | uniq | sort | paste -s -d, -) | jq '.[].data[].message' | sort | uniq -c | sort -bgr
5815 "must NOT have additional properties"
696 "must be object"
551 "must be string"
328 "the $schema version does not match style.css 'Requires at least' version"
153 "must match exactly one schema in oneOf"
73 "must be equal to constant"
71 "the 'Requires at least' version does not support theme.json"
71 "must be equal to one of the allowed values"
47 "Missing $schema URI: undefined"
6 "property name must be valid"
5 "must be number"
2 "must be >= 1"
2 "missing 'Requires at least' header metadata"
1 "must have required property 'version'"
1 "must have required property 'slug'"
1 "must have required property 'name'"
```
Loading
Loading