[PR #8066/cba34699 backport][3.9] 💅📝 Restructure the changelog for cl…

…arity (#8068)

**This is a backport of PR #8066 as merged into master
(cba3469).**

PR #8066

(cherry picked from commit cba3469)

<!-- Thank you for your contribution! -->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

## Are there changes in behavior for the user?

<!-- Outline any notable behaviour for the end users. -->

## Related issue number

<!-- Are there any issues opened that will be resolved by merging this
change? -->
<!-- Remember to prefix with 'Fixes' if it should close the issue (e.g.
'Fixes #123'). -->

## Checklist

- [ ] I think the code is well written
- [ ] Unit tests for the changes exist
- [ ] Documentation reflects the changes
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`
  * The format is &lt;Name&gt; &lt;Surname&gt;.
  * Please keep alphabetical order, the file is sorted by names.
- [ ] Add a new news fragment into the `CHANGES/` folder
  * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`)
  * if you don't have an issue number, change it to the pull request
    number after creating the PR
    * `.bugfix`: A bug fix for something the maintainers deemed an
      improper undesired behavior that got corrected to match
      pre-agreed expectations.
    * `.feature`: A new behavior, public APIs. That sort of stuff.
    * `.deprecation`: A declaration of future API removals and breaking
      changes in behavior.
    * `.breaking`: When something public is removed in a breaking way.
      Could be deprecated in an earlier release.
    * `.doc`: Notable updates to the documentation structure or build
      process.
    * `.packaging`: Notes for downstreams about unobvious side effects
      and tooling. Changes in the test invocation considerations and
      runtime assumptions.
    * `.contrib`: Stuff that affects the contributor experience. e.g.
      Running tests, building the docs, setting up the development
      environment.
    * `.misc`: Changes that are hard to assign to any of the above
      categories.
  * Make sure to use full sentences with correct case and punctuation,
    for example:
    ```rst
    Fixed issue with non-ascii contents in doctest text files
    -- by :user:`contributor-gh-handle`.
    ```

    Use the past tense or the present tense a non-imperative mood,
    referring to what's changed compared to the last released version
    of this project.

.github/PULL_REQUEST_TEMPLATE.md

@@ -21,13 +21,35 @@
 - [ ] If you provide code modification, please add yourself to `CONTRIBUTORS.txt`
   * The format is <Name> <Surname>.
   * Please keep alphabetical order, the file is sorted by names.
-- [ ] Add a new news fragment into the `CHANGES` folder
-  * name it `<issue_id>.<type>` for example (588.bugfix)
-  * if you don't have an `issue_id` change it to the pr id after creating the pr
-  * ensure type is one of the following:
-    * `.feature`: Signifying a new feature.
-    * `.bugfix`: Signifying a bug fix.
-    * `.doc`: Signifying a documentation improvement.
-    * `.removal`: Signifying a deprecation or removal of public API.
-    * `.misc`: A ticket has been closed, but it is not of interest to users.
-  * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."
+- [ ] Add a new news fragment into the `CHANGES/` folder
+  * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`)
+  * if you don't have an issue number, change it to the pull request
+    number after creating the PR
+    * `.bugfix`: A bug fix for something the maintainers deemed an
+      improper undesired behavior that got corrected to match
+      pre-agreed expectations.
+    * `.feature`: A new behavior, public APIs. That sort of stuff.
+    * `.deprecation`: A declaration of future API removals and breaking
+      changes in behavior.
+    * `.breaking`: When something public is removed in a breaking way.
+      Could be deprecated in an earlier release.
+    * `.doc`: Notable updates to the documentation structure or build
+      process.
+    * `.packaging`: Notes for downstreams about unobvious side effects
+      and tooling. Changes in the test invocation considerations and
+      runtime assumptions.
+    * `.contrib`: Stuff that affects the contributor experience. e.g.
+      Running tests, building the docs, setting up the development
+      environment.
+    * `.misc`: Changes that are hard to assign to any of the above
+      categories.
+  * Make sure to use full sentences with correct case and punctuation,
+    for example:
+    ```rst
+    Fixed issue with non-ascii contents in doctest text files
+    -- by :user:`contributor-gh-handle`.
+    ```
+
+    Use the past tense or the present tense a non-imperative mood,
+    referring to what's changed compared to the last released version
+    of this project.

.github/workflows/ci-cd.yml

@@ -414,8 +414,10 @@ jobs:
         version_file: aiohttp/__init__.py
         github_token: ${{ secrets.GITHUB_TOKEN }}
         dist_dir: dist
-        fix_issue_regex: "`#(\\d+) <https://github.com/aio-libs/aiohttp/issues/\\1>`_"
-        fix_issue_repl: "(#\\1)"
+        fix_issue_regex: >-
+          :issue:`(\d+)`
+        fix_issue_repl: >-
+          #\1
 
     - name: >-
         Publish 🐍📦 to PyPI

.pre-commit-config.yaml

@@ -6,9 +6,35 @@ repos:
     language: fail
     entry: >-
       Changelog files must be named
-      ####.(bugfix|feature|removal|doc|misc)(.#)?(.rst)?
+      ####.(
+      bugfix
+      | feature
+      | deprecation
+      | breaking
+      | doc
+      | packaging
+      | contrib
+      | misc
+      )(.#)?(.rst)?
     exclude: >-
-      ^CHANGES/(\.TEMPLATE\.rst|\.gitignore|\d+\.(bugfix|feature|removal|doc|misc)(\.\d+)?(\.rst)?|README\.rst)$
+      (?x)
+      ^
+        CHANGES/(
+          \.gitignore
+          |(\d+|[0-9a-f]{8}|[0-9a-f]{7}|[0-9a-f]{40})\.(
+            bugfix
+            |feature
+            |deprecation
+            |breaking
+            |doc
+            |packaging
+            |contrib
+            |misc
+          )(\.\d+)?(\.rst)?
+          |README\.rst
+          |\.TEMPLATE\.rst
+        )
+      $
     files: ^CHANGES/
   - id: changelogs-user-role
     name: Changelog files should use a non-broken :user:`name` role

CHANGES/.TEMPLATE.rst

@@ -11,11 +11,56 @@
 {{ underline * definitions[category]['name']|length }}
 
 {% if definitions[category]['showcontent'] %}
-{% for text, values in sections[section][category].items() %}
+{% for text, change_note_refs in sections[section][category].items() %}
 - {{ text + '\n' }}
-  {{ values|join(',\n  ') + '\n' }}
-{% endfor %}
 
+  {#
+      NOTE: Replacing 'e' with 'f' is a hack that prevents Jinja's `int`
+      NOTE: filter internal implementation from treating the input as an
+      NOTE: infinite float when it looks like a scientific notation (with a
+      NOTE: single 'e' char in between digits), raising an `OverflowError`,
+      NOTE: subsequently. 'f' is still a hex letter so it won't affect the
+      NOTE: check for whether it's a (short or long) commit hash or not.
+      Ref: https://github.com/pallets/jinja/issues/1921
+  -#}
+  {%-
+    set pr_issue_numbers = change_note_refs
+    | map('lower')
+    | map('replace', 'e', 'f')
+    | map('int', default=None)
+    | select('integer')
+    | map('string')
+    | list
+  -%}
+  {%- set arbitrary_refs = [] -%}
+  {%- set commit_refs = [] -%}
+  {%- with -%}
+    {%- set commit_ref_candidates = change_note_refs | reject('in', pr_issue_numbers) -%}
+    {%- for cf in commit_ref_candidates -%}
+      {%- if cf | length in (7, 8, 40) and cf | int(default=None, base=16) is not none -%}
+        {%- set _ = commit_refs.append(cf) -%}
+      {%- else -%}
+        {%- set _ = arbitrary_refs.append(cf) -%}
+      {%- endif -%}
+    {%- endfor -%}
+  {%- endwith -%}
+
+  {% if pr_issue_numbers -%}
+  *Related issues and pull requests on GitHub:*
+  :issue:`{{ pr_issue_numbers | join('`, :issue:`') }}`.
+  {% endif %}
+
+  {% if commit_refs -%}
+  *Related commits on GitHub:*
+  :commit:`{{ commit_refs | join('`, :commit:`') }}`.
+  {% endif %}
+
+  {% if arbitrary_refs -%}
+  *Unlinked references:*
+  {{ arbitrary_refs | join(', ') }}`.
+  {% endif %}
+
+{% endfor %}
 {% else %}
 - {{ sections[section][category]['']|join(', ') }}
 
@@ -34,3 +79,4 @@ No significant changes.
 {% endif %}
 {% endfor %}
 ----
+{{ '\n' * 2 }}

CHANGES/.gitignore

@@ -1 +1,28 @@
+*
+!.TEMPLATE.rst
 !.gitignore
+!README.rst
+!*.bugfix
+!*.bugfix.rst
+!*.bugfix.*.rst
+!*.breaking
+!*.breaking.rst
+!*.breaking.*.rst
+!*.contrib
+!*.contrib.rst
+!*.contrib.*.rst
+!*.deprecation
+!*.deprecation.rst
+!*.deprecation.*.rst
+!*.doc
+!*.doc.rst
+!*.doc.*.rst
+!*.feature
+!*.feature.rst
+!*.feature.*.rst
+!*.misc
+!*.misc.rst
+!*.misc.*.rst
+!*.packaging
+!*.packaging.rst
+!*.packaging.*.rst

CHANGES/8066.contrib.rst

@@ -0,0 +1,21 @@
+The changelog categorization was made clearer. The
+contributors can now mark their fragment files more
+accurately -- by :user:`webknjaz`.
+
+The new category tags are:
+
+    * ``bugfix``
+
+    * ``feature``
+
+    * ``deprecation``
+
+    * ``breaking`` (previously, ``removal``)
+
+    * ``doc``
+
+    * ``packaging``
+
+    * ``contrib``
+
+    * ``misc``

CHANGES/8066.packaging.rst

@@ -0,0 +1 @@
+8066.contrib.rst

CHANGES/README.rst

@@ -43,19 +43,32 @@ with your own!).
 Finally, name your file following the convention that Towncrier
 understands: it should start with the number of an issue or a
 PR followed by a dot, then add a patch type, like ``feature``,
-``doc``, ``misc`` etc., and add ``.rst`` as a suffix. If you
+``doc``, ``contrib`` etc., and add ``.rst`` as a suffix. If you
 need to add more than one fragment, you may add an optional
 sequence number (delimited with another period) between the type
 and the suffix.
 
 In general the name will follow ``<pr_number>.<category>.rst`` pattern,
 where the categories are:
 
-- ``feature``: Any new feature
-- ``bugfix``: A bug fix
-- ``doc``: A change to the documentation
-- ``misc``: Changes internal to the repo like CI, test and build changes
-- ``removal``: For deprecations and removals of an existing feature or behavior
+- ``bugfix``: A bug fix for something we deemed an improper undesired
+  behavior that got corrected in the release to match pre-agreed
+  expectations.
+- ``feature``: A new behavior, public APIs. That sort of stuff.
+- ``deprecation``: A declaration of future API removals and breaking
+  changes in behavior.
+- ``breaking``: When something public gets removed in a breaking way.
+  Could be deprecated in an earlier release.
+- ``doc``: Notable updates to the documentation structure or build
+  process.
+- ``packaging``: Notes for downstreams about unobvious side effects
+  and tooling. Changes in the test invocation considerations and
+  runtime assumptions.
+- ``contrib``: Stuff that affects the contributor experience. e.g.
+  Running tests, building the docs, setting up the development
+  environment.
+- ``misc``: Changes that are hard to assign to any of the above
+  categories.
 
 A pull request may have more than one of these components, for example
 a code change may introduce a new feature that deprecates an old

docs/spelling_wordlist.txt

@@ -17,6 +17,7 @@ Arsenic
 async
 asyncio
 asyncpg
+attrs
 auth
 autocalculated
 autodetection
@@ -34,6 +35,7 @@ backports
 BaseEventLoop
 basename
 BasicAuth
+behaviour
 BodyPartReader
 boolean
 botocore
@@ -89,6 +91,7 @@ Cythonize
 cythonized
 de
 deduplicate
+defs
 Dependabot
 deprecations
 DER
@@ -104,6 +107,7 @@ DNSResolver
 docstring
 docstrings
 DoS
+downstreams
 Dup
 elasticsearch
 encodings
@@ -313,6 +317,8 @@ Testsuite
 Tf
 timestamps
 TLS
+tmp
+tmpdir
 toolbar
 toplevel
 towncrier
@@ -329,6 +335,7 @@ Unittest
 unix
 unsets
 unstripped
+untyped
 uppercased
 upstr
 url

pyproject.toml

@@ -5,12 +5,73 @@ requires = [
 build-backend = "setuptools.build_meta"
 
 [tool.towncrier]
-package = "aiohttp"
-filename = "CHANGES.rst"
-directory = "CHANGES/"
-title_format = "{version} ({project_date})"
-template = "CHANGES/.TEMPLATE.rst"
-issue_format = "`#{issue} <https://github.com/aio-libs/aiohttp/issues/{issue}>`_"
+  package = "aiohttp"
+  filename = "CHANGES.rst"
+  directory = "CHANGES/"
+  title_format = "{version} ({project_date})"
+  template = "CHANGES/.TEMPLATE.rst"
+  issue_format = "{issue}"
+
+  # NOTE: The types are declared because:
+  # NOTE: - there is no mechanism to override just the value of
+  # NOTE:   `tool.towncrier.type.misc.showcontent`;
+  # NOTE: - and, we want to declare extra non-default types for
+  # NOTE:   clarity and flexibility.
+
+  [[tool.towncrier.section]]
+    path = ""
+
+  [[tool.towncrier.type]]
+    # Something we deemed an improper undesired behavior that got corrected
+    # in the release to match pre-agreed expectations.
+    directory = "bugfix"
+    name = "Bug fixes"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # New behaviors, public APIs. That sort of stuff.
+    directory = "feature"
+    name = "Features"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # Declarations of future API removals and breaking changes in behavior.
+    directory = "deprecation"
+    name = "Deprecations (removal in next major release)"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # When something public gets removed in a breaking way. Could be
+    # deprecated in an earlier release.
+    directory = "breaking"
+    name = "Removals and backward incompatible breaking changes"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # Notable updates to the documentation structure or build process.
+    directory = "doc"
+    name = "Improved documentation"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # Notes for downstreams about unobvious side effects and tooling. Changes
+    # in the test invocation considerations and runtime assumptions.
+    directory = "packaging"
+    name = "Packaging updates and notes for downstreams"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # Stuff that affects the contributor experience. e.g. Running tests,
+    # building the docs, setting up the development environment.
+    directory = "contrib"
+    name = "Contributor-facing changes"
+    showcontent = true
+
+  [[tool.towncrier.type]]
+    # Changes that are hard to assign to any of the above categories.
+    directory = "misc"
+    name = "Miscellaneous internal changes"
+    showcontent = true
 
 
 [tool.cibuildwheel]