gh-112301: Compiler warning management tooling #121730

nohlson · 2024-07-13T21:53:41Z

#112301: This is the first version of compiler warning check tooling integrated into the Ubuntu build ant test GitHub action job.

The tool parses compiler output and checks against an ignore file, which is a list of files for which warnings should be ignored. The script includes options to fail on regression, or fail on improvement (if a file is in the ignore list but the compiler emitted no warnings).

A more detailed writeup about the idea behind the tool can be found here

The current configuration in reusable-ubuntu.yml does not include flags to --fail-on-regression or --fail-on-improvement and simply writes to the log warnings that were emitted but not ignored, or file names that were unexpectedly missing warnings but will not fail the step in the job. Per discussion in the issue we can eventually enable those options.

Future improvements would include removal of duplicate warnings, warning count per file tracking, and adding GitHub Actions annotations.

Issue: Consider applying flags for warnings about potential security issues #112301

hugovk · 2024-07-14T09:28:57Z

(I updated the branch from main to fix the unrelated docs error: Error: new NEWS nits: /home/runner/work/cpython/cpython/build/NEWS:72: WARNING: py:func reference target not found: warnings.filterswarnings.)

hugovk

Looking good! Here's a few comments and questions.

Tools/build/check_warnings.py

hugovk · 2024-07-14T09:26:31Z

Tools/build/check_warnings.py

+        try:
+            json_data = json.loads(array)
+            json_objects_in_array = [entry for entry in json_data]
+            compiler_warnings.extend([entry for entry in json_objects_in_array if entry.get('kind') == 'warning'])


Please could you limit lines to 79 chars?

https://peps.python.org/pep-0008/#maximum-line-length

This file is in Tools/, I think it's okay to just run Black on it:

black --line-length 79 Tools/build/check_warnings.py

Broke this line into two

Misc/NEWS.d/next/Tests/2024-07-13-21-55-58.gh-issue-112301.YJS1dl.rst

hugovk · 2024-07-14T09:33:14Z

Tools/build/check_warnings.py

+import sys
+from pathlib import Path
+import argparse
+import json
+import re


Let's sort 'em:

Suggested change

import sys

from pathlib import Path

import argparse

import json

import re

import argparse

import json

import re

import sys

from pathlib import Path

Sorted imports

.github/workflows/reusable-ubuntu.yml

hugovk · 2024-07-14T10:04:38Z

Tools/build/check_warnings.py

+            for key in ['caret', 'start', 'end']:
+                if key in location:
+                    file = location[key]['file']
+                    file = file.lstrip('./') # Remove leading curdir if present


Suggested change

file = file.lstrip('./') # Remove leading curdir if present

file = file.lstrip('./') # Remove leading current directory if present

Changed comment

hugovk · 2024-07-14T10:04:46Z

Tools/build/check_warnings.py

+            for key in ['caret', 'start', 'end']:
+                if key in location:
+                    file = location[key]['file']
+                    file = file.lstrip('./') # Remove leading curdir if present


Suggested change

file = file.lstrip('./') # Remove leading curdir if present

file = file.lstrip('./') # Remove leading current directory if present

Changed comment

sethmlarson

Thanks @hugovk for reviewing too, I have some more comments:

Tools/build/check_warnings.py

nohlson · 2024-07-17T03:41:06Z

I have made the requested changes; please review again

bedevere-app · 2024-07-17T03:41:11Z

Thanks for making the requested changes!

: please review the changes made to this pull request.

sethmlarson

Few more comments, otherwise looks good for the first PR :)

Tools/build/check_warnings.py

Misc/NEWS.d/next/Tests/2024-07-13-21-55-58.gh-issue-112301.YJS1dl.rst

hugovk · 2024-07-26T06:35:03Z

Tools/build/check_warnings.py

+import sys
+from pathlib import Path
+
+def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:


This looks almost good to merge, just some style nits :)

Technically we're outside the stdlib so strictly speaking PEP 8 doesn't apply, but let's follow it anyway. For example, please could you add extra newlines before these defs and wrap things to 79 chars?

The easiest way is to run Black (which we don't run on the stdlib but again we're outside the stdlib :)

black --line-length 79 Tools/build/check_warnings.py

Then some comments and long strings will need manual wrapping.

Suggested change

def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:

def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:

Also, please don't force push in this repo, it makes it easier to review new commits, and we squash merge everything at the end anyway. I think force-pushing is sometimes to blame for pinging lots of unrelated CODEOWNERS too.

In order to keep the commit history intact, please avoid squashing or amending history and then force-pushing to the PR. Reviewers often want to look at individual commits.

https://devguide.python.org/getting-started/pull-request-lifecycle/#quick-guide

Thanks!

Updated to adhere to PEP8.

After a rebase there were several commits that came from branches from different forks. Not really sure how that happened but I had to revert to my last commit and force push to get rid of them.

Thank you! I just pushed a commit to also wrap comments and strings to 79 chars as well.

hugovk

Thanks!

Tools/build/check_warnings.py

picnixz · 2024-07-27T09:40:58Z

Tools/build/check_warnings.py

@@ -0,0 +1,195 @@
+#!/usr/bin/env python3


Do we need this one?

I meant to comment this too: no, I don't think we do because the file does not have the executable bit and we're running it via python Tools/build/check_warnings.py

hugovk · 2024-07-30T06:44:30Z

@picnixz Your review comments came after I'd enabled auto-merge, sorry about that!

@nohlson Hi! @corona10 noticed that with this change, the build output is no longer being output to the logs, because we're redirecting it to a file. It can sometimes be useful to see this output.

Before: https://github.com/python/cpython/actions/runs/10122181963/job/27993976874
After: https://github.com/python/cpython/actions/runs/10122668622/job/27995097214

@zware suggested a command like this:

set -o pipefail; make -j4 2>&1 | tee compiler_output.txt

Please could you open a PR to try this out? Thank you!

nohlson requested review from ezio-melotti and hugovk as code owners July 13, 2024 21:53

bedevere-app bot mentioned this pull request Jul 13, 2024

Consider applying flags for warnings about potential security issues #112301

Open

bedevere-app bot added the awaiting review label Jul 13, 2024

hugovk reviewed Jul 14, 2024

View reviewed changes

sethmlarson reviewed Jul 15, 2024

View reviewed changes

Tools/build/check_warnings.py Show resolved Hide resolved

Tools/build/check_warnings.py Show resolved Hide resolved

Tools/build/check_warnings.py Outdated Show resolved Hide resolved

bedevere-app bot added awaiting change review and removed awaiting review labels Jul 17, 2024

nohlson and others added 12 commits July 23, 2024 03:14

Create simple warning check tool and add to ubuntu build and test job

5148727

Add flags to check warnings script to fail on regression or improvement

3bd5a10

Remove redundant comment

6813003

Rename warnigore file to warningignore

ab4d754

Use regex to extract json arrays

615d228

Trim whitespace

fc0a60b

Test on github unexpected improvement

7793d80

Add config for improve fail check

99715d2

Revert to prod check warning state

bc44ec2

📜🤖 Added by blurb_it.

e2ca75f

Refactor creating set of files with warnings to a dedicated function

bd1634e

Move cflags configure option to top level build configuration

b07b1d6

nohlson force-pushed the warning-management-tooling branch from 767308a to b07b1d6 Compare July 23, 2024 08:18

nohlson added 2 commits July 23, 2024 03:20

Add json diagnostics to ubuntu configuration as first argument

e1954a5

Add newline to news

5935143

sethmlarson reviewed Jul 23, 2024

View reviewed changes

Tools/build/check_warnings.py Outdated Show resolved Hide resolved

Tools/build/check_warnings.py Show resolved Hide resolved

nohlson mentioned this pull request Jul 24, 2024

gh-112301: Add macOS warning tracking tooling #122211

Merged

sethmlarson approved these changes Jul 24, 2024

View reviewed changes

bedevere-app bot removed the awaiting change review label Jul 24, 2024

sethmlarson requested review from ezio-melotti and removed request for JelleZijlstra, ericvsmith, rhettinger, markshannon, pablogsal, pganssle, abalkin, ericsnowcurrently, ambv, iritkatriel, methane, 1st1, asvetlov and willingc July 25, 2024 04:13

hugovk reviewed Jul 26, 2024

View reviewed changes

nohlson and others added 3 commits July 26, 2024 17:40

Enforce PEP8 on check_warnigns

0c0fc31

Add period to news

e568d6c

Wrap comments and strings to 79 chars

9f6281e

hugovk approved these changes Jul 27, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting change review labels Jul 27, 2024

hugovk enabled auto-merge (squash) July 27, 2024 09:35

picnixz reviewed Jul 27, 2024

View reviewed changes

hugovk merged commit 8ac5565 into python:main Jul 27, 2024
38 of 39 checks passed

bedevere-app bot removed the awaiting merge label Jul 27, 2024

This was referenced Jul 30, 2024

gh-112301: Add argument aliases and tee compiler output for check warnings #122465

Merged

gh-112301: Add -Wformat=2 compiler option to NODIST #122474

Open

nohlson mentioned this pull request Aug 7, 2024

Add compiler warning tracking tooling failure remediation guidance python/devguide#1364

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-112301: Compiler warning management tooling #121730

gh-112301: Compiler warning management tooling #121730

nohlson commented Jul 13, 2024 •

edited by bedevere-app bot

Loading

hugovk commented Jul 14, 2024

hugovk left a comment

hugovk Jul 14, 2024

nohlson Jul 17, 2024

hugovk Jul 14, 2024

nohlson Jul 17, 2024

hugovk Jul 14, 2024

nohlson Jul 17, 2024

hugovk Jul 14, 2024

nohlson Jul 17, 2024

sethmlarson left a comment

nohlson commented Jul 17, 2024

bedevere-app bot commented Jul 17, 2024

sethmlarson left a comment

hugovk Jul 26, 2024 •

edited

Loading

nohlson Jul 27, 2024

hugovk Jul 27, 2024

hugovk left a comment

picnixz Jul 27, 2024

hugovk Jul 30, 2024

hugovk commented Jul 30, 2024 •

edited

Loading

	file = file.lstrip('./') # Remove leading curdir if present
	file = file.lstrip('./') # Remove leading current directory if present

	def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:

	def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:

gh-112301: Compiler warning management tooling #121730

gh-112301: Compiler warning management tooling #121730

Conversation

nohlson commented Jul 13, 2024 • edited by bedevere-app bot Loading

hugovk commented Jul 14, 2024

hugovk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sethmlarson left a comment

Choose a reason for hiding this comment

nohlson commented Jul 17, 2024

bedevere-app bot commented Jul 17, 2024

sethmlarson left a comment

Choose a reason for hiding this comment

hugovk Jul 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hugovk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hugovk commented Jul 30, 2024 • edited Loading

nohlson commented Jul 13, 2024 •

edited by bedevere-app bot

Loading

hugovk Jul 26, 2024 •

edited

Loading

hugovk commented Jul 30, 2024 •

edited

Loading