Can we add a json schema to complete pyproject.toml's [tool.black]?

[Freed-Wu]

Describe the solution you'd like

json schema can let editor which support LSP to complete some configuration file. Such as:

There are many tools have supported json schema: https://json.schemastore.org/pyproject.json

Can tool.black be supported?

Describe alternatives you've considered

Refer cmhughes/latexindent.pl#206:

The steps are following:

1. Create a json schema
2. Add its URL to schemastore.

[JelleZijlstra]

I'd accept a PR for this.

[henryiii]

I've (slowly) started writing one.

[henryiii]

How about something like this:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://json.schemastore.org/partial-black.json",
  "$comment": "black table in pyproject.toml",
  "type": "object",
  "definitions": {
    "click-bool": {
      "enum": [
        true,
        false,
        0,
        1,
        "True",
        "False",
        "true",
        "false",
        "1",
        "0"
      ]
    }
  },
  "additionalProperties": false,
  "properties": {
    "code": {
      "type": "string",
      "description": "Format the code passed in as a string."
    },
    "line-length": {
      "default": 88,
      "type": "integer",
      "description": "How many characters per line to allow."
    },
    "target-version": {
      "type": "array",
      "items": {
        "enum": [
          "py33",
          "py34",
          "py35",
          "py36",
          "py37",
          "py38",
          "py39",
          "py310",
          "py311",
          "py312"
        ]
      },
      "description": "Python versions that should be supported by Black's output. You should include all versions that your code supports. By default, Black will infer target versions from the project metadata in pyproject.toml. If this does not yield conclusive results, Black will use per-file auto-detection."
    },
    "pyi": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Format all input files like typing stubs regardless of file extension. This is useful when piping source on standard input."
    },
    "ipynb": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Format all input files like Jupyter Notebooks regardless of file extension.This is useful when piping source on standard input."
    },
    "python-cell-magics": {
      "default": [],
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "When processing Jupyter Notebooks, add the given magic to the list of known python-magics (capture, prun, pypy, python, python3, time, timeit). Useful for formatting cells with custom python magics."
    },
    "skip-source-first-line": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Skip the first line of the source code."
    },
    "skip-string-normalization": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Don't normalize string quotes or prefixes."
    },
    "skip-magic-trailing-comma": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Don't use trailing commas as a reason to split lines."
    },
    "preview": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Enable potentially disruptive style changes that may be added to Black's main functionality in the next major release."
    },
    "unstable": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Enable potentially disruptive style changes that have known bugs or are not currently expected to make it into the stable style Black's next major release. Implies --preview."
    },
    "enable-unstable-feature": {
      "type": "array",
      "items": {
        "enum": [
          "hex_codes_in_unicode_sequences",
          "string_processing",
          "hug_parens_with_braces_and_square_brackets",
          "unify_docstring_detection",
          "no_normalize_fmt_skip_whitespace",
          "wrap_long_dict_values_in_parens",
          "multiline_string_handling"
        ]
      },
      "description": "Enable specific features included in the `--unstable` style. Requires `--preview`. No compatibility guarantees are provided on the behavior or existence of any unstable features."
    },
    "check": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Don't write the files back, just return the status. Return code 0 means nothing would change. Return code 1 means some files would be reformatted. Return code 123 means there was an internal error."
    },
    "diff": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Don't write the files back, just output a diff to indicate what changes Black would've made. They are printed to stdout so capturing them is simple."
    },
    "color": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Show (or do not show) colored diff. Only applies when --diff is given."
    },
    "fast": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "By default, Black performs an AST safety check after formatting your code. The --fast flag turns off this check and the --safe flag explicitly enables it. [default: --safe]"
    },
    "required-version": {
      "type": "string",
      "description": "Require a specific version of Black to be running. This is useful for ensuring that all contributors to your project are using the same version, because different versions of Black may format code a little differently. This option can be set in a configuration file for consistent results across environments."
    },
    "exclude": {
      "type": "string",
      "description": "A regular expression that matches files and directories that should be excluded on recursive searches. An empty value means no paths are excluded. Use forward slashes for directories on all platforms (Windows, too). By default, Black also ignores all paths listed in .gitignore. Changing this value will override all default exclusions. [default: /(\\.direnv|\\.eggs|\\.git|\\.hg|\\.ipynb_checkpoints|\\.mypy_cache|\\.nox|\\.pytest_cache|\\.ruff_cache|\\.tox|\\.svn|\\.venv|\\.vscode|__pypackages__|_build|buck-out|build|dist|venv)/]"
    },
    "extend-exclude": {
      "type": "string",
      "description": "Like --exclude, but adds additional files and directories on top of the default values instead of overriding them."
    },
    "force-exclude": {
      "type": "string",
      "description": "Like --exclude, but files and directories matching this regex will be excluded even when they are passed explicitly as arguments. This is useful when invoking Black programmatically on changed files, such as in a pre-commit hook or editor plugin."
    },
    "include": {
      "default": "(\\.pyi?|\\.ipynb)$",
      "type": "string",
      "description": "A regular expression that matches files and directories that should be included on recursive searches. An empty value means all files are included regardless of the name. Use forward slashes for directories on all platforms (Windows, too). Overrides all exclusions, including from .gitignore and command line options."
    },
    "workers": {
      "type": "integer",
      "description": "When Black formats multiple files, it may use a process pool to speed up formatting. This option controls the number of parallel workers. This can also be specified via the BLACK_NUM_WORKERS environment variable. Defaults to the number of CPUs in the system."
    },
    "quiet": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Stop emitting all non-critical output. Error messages will still be emitted (which can silenced by 2>/dev/null)."
    },
    "verbose": {
      "default": false,
      "$ref": "#/definitions/click-bool",
      "description": "Emit messages about files that were not changed or were ignored due to exclusion patterns. If Black is using a configuration file, a message detailing which one it is using will be emitted."
    }
  }
}

?

[henryiii]

For example, do all of those items make sense in pyproject.toml? I implemented the current bool-not-required for bool, rather than just making it a bool, but could change that. I didn't do the same thing for integers, so I'm not consistent. :)

Generation script black scripts/schema.py:

import json
from typing import Any

import black
import click


def generate_schema_from_click(
    cmd: click.Command,
) -> dict[str, Any]:
    result = {}
    for param in cmd.params:
        if isinstance(param, click.Argument) or param.is_eager:
            continue
        name = param.name.replace("_", "-")
        default = {"default": param.default} if param.default is not None else {}
        click_type = param.type
        if isinstance(click_type, click.types.IntParamType):
            json_type = {"type": "integer"}
        elif isinstance(click_type, (click.types.StringParamType, click.types.Path)):
            json_type = {"type": "string"}
        elif isinstance(click_type, click.types.Choice):
            json_type = {"enum": click_type.choices}
        elif isinstance(click_type, click.types.BoolParamType):
            # type: boolean would be nicer!
            json_type = {"$ref": "#/definitions/click-bool"}
        else:
            msg = f"{click_type!r} not a known type for {param}"
            raise TypeError(msg)

        if param.multiple:
            json_type = {"type": "array", "items": json_type}

        result[name] = {**default, **json_type, "description": param.help}
    return result


if __name__ == "__main__":
    properties = generate_schema_from_click(black.main)
    del properties["line-ranges"]
    schema = {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "$id": "https://json.schemastore.org/partial-black.json",
        "$comment": "black table in pyproject.toml",
        "type": "object",
        "definitions": {
            "click-bool": {
                "enum": [True, False, 0, 1, "True", "False", "true", "false", "1", "0"]
            }
        },
        "additionalProperties": False,
        "properties": properties,
    }

    print(json.dumps(schema, indent=2))

[henryiii]

Also, what's the min Python version for scripts? I would rather use a match statement there (I actually used a smidge of 3.10 above, but was writing it assuming scripts might be limited to the same versions as the package)

[henryiii]

extension.This Hmm, one or more of those has spacing issues.

[JelleZijlstra]

Thanks!

I think check and diff don't make a ton of sense in pyproject.toml, but if someone wants to use it, it should work, and it's simpler if we just include everything.

I'd be fine accepting just boolean values for the boolean options. Do you think the variants you listed are common?

What would be the process for updating the schema in SchemaStore? In particular the list of allowed options in --enable-unstable-feature is likely to change frequently.

For a script that's only expected to run by maintainers, I'd accept match-case.

[henryiii]

For bools:

tool.black.skip-string-normalization contents:
True: 2512
1: 59
'true': 24
'True': 14
'1': 12

tool.black.preview contents:
True: 650
'True': 4
1: 1
'true':

We can avoid specifying lists of options that might change frequently. The pyXX ones would change once per year, we could put a regex there allowing pyXX instead. Also, I can add a "--schema-store" flag, and only strip the frequently changing stuff for schema store export, and have a more explicit one in black since it's always in sync with the version.

SchemaStore is a PR to a repo with the schemas, it's pretty easy, but not automated.

[henryiii]

Made a PR. Realized the irony of processing click options using an argparse based script, so fixed that. :)