Add support for thread sanitizer (TSAN) via --with-thread-sanitizer

[colesbury]

Feature or enhancement

GCC and Clang provide ThreadSanitizer, a tool that detects data races. We should add support for thread sanitizer to CPython. Note that Python already supports the memory and address sanitizers.

• Add --with-thread-sanitizer as a configure option
• Define _Py_THREAD_SANITIZER if thread-sanitizer is enabled (see, e.g., _Py_ADDRESS_SANITIZER in pyport.h)
• Move the definition of _Py_NO_SANITIZE_THREAD from obmalloc.c to a place that's more widely available (like pyport.h). We're going to need this in a few places

Eventually, it would be helpful to have a continuous build for the combination of --disable-gil --with-thread-sanitizer. Note that we probably won't want to run all the tests. ThreadSanitizer is slow and also not very useful for single-threaded tests. We should collect a subset of our tests that use threading for the ThreadSanitizer continuous build.

Linked PRs

• gh-112536: Add support for thread sanitizer (TSAN) #112648
• gh-112536: enable tsan in build #113232
• gh-112536: Set up TSAN CI for free-threading #116555
• gh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-thread-sanitizer #116558
• gh-112536: Add --tsan test for reasonable TSAN execution times. #116601
• gh-112536: Add TSAN builds on Github Actions #116872
• gh-112536: Add more TSAN tests #116896
• gh-112536: Add test_threading to TSAN tests #116898
• GH-112536: Add more TSan tests #116911
• [3.12] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116924
• [3.11] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116926
• [3.12] gh-112536: Add --tsan test for reasonable TSAN execution times. (gh-116601) #116929
• [3.12] gh-112536: Add TSAN build on Github Actions (GH-116872) #116953
• gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled #117702
• [3.12] gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled (GH-117702) #117713
• [3.10]gh-112536: Add support for thread sanitizer (TSAN)(gh-112648) #123833

[samety]

Hey,
I would like to pick this up.
As a first step tried copying the struct for ASAN to TSAN
Here is a draft PR: #112648

Happy to get some initial feedback on it and then add remaining things like _Py_THREAD_SANITIZER etc.

[samety]

Added --with-thread-sanitizer as a configure option in pr 112648.

Still looking for a good way to enable TSAN for the continuous build, I will make another PR when I find one.

[pitrou]

Do we want to backport these at some point?

[corona10]

@pitrou We can go for the default build. I will submit the PR soon

[pitrou]

Ok, closing as done now.

[comicfans]

hello, possible to backport this to 3.10 too ?

[picnixz]

Unfortunately, 3.8 to 3.11 are security-only branches. As such, they would not accept such backports. The 3.11 backport was not merged since it was close to the last bugfix deadline.