diff --git a/images/python-cyclic-gc-1-new-page.png b/_static/python-cyclic-gc-1-new-page.png similarity index 100% rename from images/python-cyclic-gc-1-new-page.png rename to _static/python-cyclic-gc-1-new-page.png diff --git a/images/python-cyclic-gc-2-new-page.png b/_static/python-cyclic-gc-2-new-page.png similarity index 100% rename from images/python-cyclic-gc-2-new-page.png rename to _static/python-cyclic-gc-2-new-page.png diff --git a/images/python-cyclic-gc-3-new-page.png b/_static/python-cyclic-gc-3-new-page.png similarity index 100% rename from images/python-cyclic-gc-3-new-page.png rename to _static/python-cyclic-gc-3-new-page.png diff --git a/images/python-cyclic-gc-4-new-page.png b/_static/python-cyclic-gc-4-new-page.png similarity index 100% rename from images/python-cyclic-gc-4-new-page.png rename to _static/python-cyclic-gc-4-new-page.png diff --git a/images/python-cyclic-gc-5-new-page.png b/_static/python-cyclic-gc-5-new-page.png similarity index 100% rename from images/python-cyclic-gc-5-new-page.png rename to _static/python-cyclic-gc-5-new-page.png diff --git a/clang.rst b/advanced-tools/clang.rst similarity index 99% rename from clang.rst rename to advanced-tools/clang.rst index 76afd4786e..d626c33ea7 100644 --- a/clang.rst +++ b/advanced-tools/clang.rst @@ -1,6 +1,8 @@ -*************************** +.. _clang: + +=========================== Dynamic Analysis with Clang -*************************** +=========================== .. highlight:: bash diff --git a/coverity.rst b/advanced-tools/coverity.rst similarity index 100% rename from coverity.rst rename to advanced-tools/coverity.rst index 5546805176..7c165a3126 100644 --- a/coverity.rst +++ b/advanced-tools/coverity.rst @@ -1,9 +1,9 @@ +.. _coverity: + ============= Coverity Scan ============= -.. _coverity: - Coverity Scan is a free service for static code analysis of Open Source projects. It is based on Coverity's commercial product and is able to analyze C, C++ and Java code. diff --git a/gdb.rst b/advanced-tools/gdb.rst similarity index 99% rename from gdb.rst rename to advanced-tools/gdb.rst index 8a1fc4504e..df4d71e738 100644 --- a/gdb.rst +++ b/advanced-tools/gdb.rst @@ -1,6 +1,7 @@ .. _gdb: -gdb Support +=========== +GDB Support =========== .. highlight:: none @@ -17,7 +18,7 @@ limitation. gdb 7 and later ---------------- +=============== In gdb 7, support for `extending gdb with Python `_ was @@ -300,7 +301,7 @@ thread is doing at the Python level:: gdb 6 and earlier ------------------ +================= The file at ``Misc/gdbinit`` contains a gdb configuration file which provides extra commands when working with a CPython process. To register these commands @@ -311,7 +312,7 @@ from your gdb session. Updating auto-load-safe-path to allow test_gdb to run ------------------------------------------------------ +===================================================== ``test_gdb`` attempts to automatically load additional Python specific hooks into gdb in order to test them. Unfortunately, the command line diff --git a/advanced-tools/index.rst b/advanced-tools/index.rst new file mode 100644 index 0000000000..afa96741f3 --- /dev/null +++ b/advanced-tools/index.rst @@ -0,0 +1,10 @@ +============== +Advanced Tools +============== + +.. toctree:: + :maxdepth: 5 + + gdb + clang + coverity diff --git a/appendix.rst b/appendix.rst deleted file mode 100644 index dc2a2efe14..0000000000 --- a/appendix.rst +++ /dev/null @@ -1,70 +0,0 @@ -Appendix: Topics -================ - -Basics for contributors ------------------------ - -.. note:: **Recommended reading** - - - :doc:`setup` - - :doc:`pullrequest` - -* :doc:`help` -* :doc:`communication` - -Core developers ---------------- - -.. note:: **Recommended reading** - - * :doc:`setup` - * :doc:`pullrequest` - * :doc:`committing` - -* :doc:`coredev` -* :doc:`developers` -* :doc:`motivations` -* :doc:`experts` - -Development workflow for contributors -------------------------------------- - -* :doc:`devcycle` -* :doc:`pullrequest` -* :doc:`fixingissues` - -Documenting Python and style guide ----------------------------------- - -* :doc:`docquality` -* :doc:`documenting` - -Issue tracking and triaging ---------------------------- - -* :doc:`tracker` -* :doc:`triaging` -* :doc:`gh-labels` -* :doc:`gh-faq` - -Language development in depth ------------------------------ - -* :doc:`exploring` -* :doc:`grammar` -* :doc:`compiler` -* :doc:`garbage_collector` -* :doc:`stdlibchanges` -* :doc:`langchanges` -* :doc:`porting` - -Testing and continuous integration ----------------------------------- - -* :doc:`runtests` -* :doc:`coverage` -* :doc:`silencewarnings` -* :doc:`buildbots` -* :doc:`buildworker` -* :doc:`coverity` - diff --git a/conf.py b/conf.py index 422de53bb6..bd5d850978 100644 --- a/conf.py +++ b/conf.py @@ -29,7 +29,12 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx_copybutton'] +extensions = [ + 'sphinx.ext.intersphinx', + 'sphinx.ext.todo', + 'sphinx_copybutton', + 'sphinxext.rediraffe', +] intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} todo_include_todos = True @@ -226,6 +231,44 @@ # match any anchor that starts with a '/' since this is an invalid HTML anchor '\/.*', ] +rediraffe_redirects = { + "clang.rst": "advanced-tools/clang.rst", + "coverity.rst": "advanced-tools/coverity.rst", + "gdb.rst": "advanced-tools/gdb.rst", + "coredev.rst": "core-developers/become-core-developer.rst", + "committing.rst": "core-developers/committing.rst", + "developers.rst": "core-developers/developer-log.rst", + "experts.rst": "core-developers/experts.rst", + "motivations.rst": "core-developers/motivations.rst", + "c-api.rst": "developer-workflow/c-api.rst", + "communication.rst": "developer-workflow/communication-channels.rst", + "devcycle.rst": "developer-workflow/development-cycle.rst", + "extensions.rst": "developer-workflow/extension-modules.rst", + "grammar.rst": "developer-workflow/grammar.rst", + "langchanges.rst": "developer-workflow/lang-changes.rst", + "porting.rst": "developer-workflow/porting.rst", + "stdlibchanges.rst": "developer-workflow/stdlib.rst", + "docquality.rst": "documentation/help-documenting.rst", + "documenting.rst": "documentation/start-documenting.rst", + "fixingissues.rst": "getting-started/fixing-issues.rst", + "help.rst": "getting-started/getting-help.rst", + "gitbootcamp.rst": "getting-started/git-boot-camp.rst", + "pullrequest.rst": "getting-started/pull-request-lifecycle.rst", + "setup.rst": "getting-started/setup-building.rst", + "compiler.rst": "internals/compiler.rst", + "exploring.rst": "internals/exploring.rst", + "garbage_collector.rst": "internals/garbage-collector.rst", + "parser.rst": "internals/parser.rst", + "buildbots.rst": "testing/buildbots.rst", + "coverage.rst": "testing/coverage.rst", + "buildworker.rst": "testing/new-buildbot-worker.rst", + "runtests.rst": "testing/run-write-tests.rst", + "silencewarnings.rst": "testing/silence-warnings.rst", + "gh-faq.rst": "triage/github-bpo-faq.rst", + "tracker.rst": "triage/issue-tracker.rst", + "gh-labels.rst": "triage/labels.rst", + "triaging.rst": "triage/triaging.rst", +} linkcheck_ignore = [ # The voters repo is private and appears as a 404 @@ -238,6 +281,8 @@ 'https://discuss.python.org/groups/admins', ] + + # Use our custom CSS stylesheet to differentiate us from the official python # docs. def setup(app): diff --git a/core-developers/become-core-developer.rst b/core-developers/become-core-developer.rst new file mode 100644 index 0000000000..3c352c037e --- /dev/null +++ b/core-developers/become-core-developer.rst @@ -0,0 +1,59 @@ +.. _become-core-developer: +.. _coredev: + +============================== +How to Become a Core Developer +============================== + +What it Takes +============= + +When you have consistently contributed patches which meet quality standards +without requiring extensive rewrites prior to being committed, +you may qualify for commit privileges and become a core developer of Python. +You must also work well with other core developers (and people in general) +as you become an ambassador for the Python project. + +Typically a core developer will offer you the chance to gain commit privilege. +The person making the offer will become your mentor and watch your commits for +a while to make sure you understand the development process. If other core +developers agree that you should gain commit privileges you are then extended +an official offer. How core developers come to that agreement are outlined in +:pep:`13`. + + +Gaining Commit Privileges +========================= + +The steps to gaining commit privileges are: + +1. A core developer starts a poll at https://discuss.python.org/c/committers/5 + + - Open for 7 days + - Results shown upon close + +2. The poll is announced on python-committers +3. Wait for the poll to close and see if the results confirm your membership + as per the voting results required by PEP 13 +4. The person who nominated you emails the steering council with your email + address and a request that the council either accept or reject the proposed + membership +5. Assuming the steering council does not object, a member of the council will + email you asking for: + + - Account details as required by + 🔒 https://github.com/python/voters/ + - Your preferred email address to + subscribe to python-committers with + - A reminder about the `Code of Conduct`_ and to report issues to the PSF + Conduct WG + +6. Once you have provided the pertinent details, your various new privileges + will be turned on +7. Your details will be added to 🔒 https://github.com/python/voters/ +8. They will update the devguide to publicly list your team membership at + :ref:`developers` +9. An announcement email by the steering council member handling your new + membership will be sent to python-committers + +.. _Code of Conduct: https://www.python.org/psf/conduct/ diff --git a/committing.rst b/core-developers/committing.rst similarity index 97% rename from committing.rst rename to core-developers/committing.rst index 164b2193d8..d4832ebbd2 100644 --- a/committing.rst +++ b/core-developers/committing.rst @@ -52,8 +52,8 @@ to enter the public source tree. Ask yourself the following questions: developer can apply the label ``needs backport to X.Y`` to the pull request. Once the backport pull request has been created, remove the ``needs backport to X.Y`` label from the original pull request. (Only - core developers and members of the `Python Triage Team`_ can apply - labels to GitHub pull requests). + core developers and members of the :ref:`Python Triage Team ` + can apply labels to GitHub pull requests). * **Does the pull request pass a check indicating that the submitter has signed the CLA?** Make sure that the contributor has signed a `Contributor @@ -224,11 +224,10 @@ Note that cherry_picker.py_ adds the branch prefix automatically. Once the backport pull request has been created, remove the ``needs backport to X.Y`` label from the original pull request. (Only -core developers and members of the `Python Triage Team`_ can apply -labels to GitHub pull requests). +core developers and members of the :ref:`Python Triage Team ` +can apply labels to GitHub pull requests). .. _cherry_picker.py: https://github.com/python/cherry-picker -.. _`Python Triage Team`: https://devguide.python.org/triaging/#python-triage-team Reverting a merged pull request diff --git a/developers.rst b/core-developers/developer-log.rst similarity index 97% rename from developers.rst rename to core-developers/developer-log.rst index 160b7060ec..5a7ff01dde 100644 --- a/developers.rst +++ b/core-developers/developer-log.rst @@ -1,3 +1,4 @@ +.. _developer-log: .. _developers: Developer Log diff --git a/developers.csv b/core-developers/developers.csv similarity index 100% rename from developers.csv rename to core-developers/developers.csv diff --git a/experts.rst b/core-developers/experts.rst similarity index 99% rename from experts.rst rename to core-developers/experts.rst index 2ed035ef4c..5683e8bc2d 100644 --- a/experts.rst +++ b/core-developers/experts.rst @@ -1,5 +1,6 @@ .. _experts: +============= Experts Index ============= @@ -44,7 +45,8 @@ not feel qualified to make a decision in a particular context. Stdlib ------- +====== + ==================== ============================================= Module Maintainers ==================== ============================================= @@ -276,7 +278,8 @@ zlib Yhg1s, gpshead* Tools ------ +===== + ================== =========== Tool Maintainers ================== =========== @@ -286,7 +289,8 @@ PEG Generator gvanrossum, pablogsal, lysnikolaou Platforms ---------- +========= + =================== =========== Platform Maintainers =================== =========== @@ -305,7 +309,8 @@ JVM/Java frank.wierzbicki^ Miscellaneous -------------- +============= + ================== ========================================================== Interest Area Maintainers ================== ========================================================== @@ -359,6 +364,6 @@ version control merwok, ezio-melotti Documentation Translations --------------------------- +========================== For a list of translators, see :ref:`this table about translations `. diff --git a/core-developers/index.rst b/core-developers/index.rst new file mode 100644 index 0000000000..cf743c405c --- /dev/null +++ b/core-developers/index.rst @@ -0,0 +1,13 @@ +=============== +Core Developers +=============== + +.. toctree:: + :maxdepth: 5 + + responsibilities + committing + experts + developer-log + motivations + become-core-developer diff --git a/motivations.rst b/core-developers/motivations.rst similarity index 99% rename from motivations.rst rename to core-developers/motivations.rst index 214853b187..41e3156ecb 100644 --- a/motivations.rst +++ b/core-developers/motivations.rst @@ -1,7 +1,8 @@ .. _motivations: -Core Developer Motivations and Affiliations -=========================================== +============================ +Motivations and Affiliations +============================ CPython core developers participate in the core development process for a variety of reasons. Being accepted as a core developer indicates that @@ -29,7 +30,7 @@ For more information on the origins and purpose of this page, see .. _published-motivations: Published entries ------------------ +================= The following core developers have chosen to provide additional details regarding their professional affiliations and (optionally) other reasons for @@ -270,7 +271,7 @@ participating in the CPython core development process: .. _goals-of-the-motivations-page: Goals of this page ------------------- +================== The `issue metrics`_ automatically collected by the CPython issue tracker strongly suggest that the current core development process is bottlenecked on @@ -329,7 +330,7 @@ time contributing to CPython development. Limitations on scope --------------------- +==================== * Specific technical areas of interest for core developers should be captured in the :ref:`Experts Index `. diff --git a/coredev.rst b/core-developers/responsibilities.rst similarity index 71% rename from coredev.rst rename to core-developers/responsibilities.rst index d6b8a850e6..da51b24636 100644 --- a/coredev.rst +++ b/core-developers/responsibilities.rst @@ -1,26 +1,8 @@ -.. _coredev: +.. _responsibilities: -How to Become a Core Developer -============================== - -What it Takes -------------- - -When you have consistently contributed patches which meet quality standards -without requiring extensive rewrites prior to being committed, -you may qualify for commit privileges and become a core developer of Python. -You must also work well with other core developers (and people in general) -as you become an ambassador for the Python project. - -Typically a core developer will offer you the chance to gain commit privilege. -The person making the offer will become your mentor and watch your commits for -a while to make sure you understand the development process. If other core -developers agree that you should gain commit privileges you are then extended -an official offer. How core developers come to that agreement are outlined in -:pep:`13`. - -What it Means -------------- +================ +Responsibilities +================ As contributors to the CPython project, our shared responsibility is to collaborate constructively with other contributors, including core developers. @@ -58,45 +40,9 @@ is a large project, and different core developers accept responsibility for making design and development decisions in different areas (as documented in the :ref:`experts` and :ref:`developers`). -Gaining Commit Privileges -------------------------- - -The steps to gaining commit privileges are: - -1. A core developer starts a poll at https://discuss.python.org/c/committers/5 - - - Open for 7 days - - Results shown upon close - -2. The poll is announced on python-committers -3. Wait for the poll to close and see if the results confirm your membership - as per the voting results required by PEP 13 -4. The person who nominated you emails the steering council with your email - address and a request that the council either accept or reject the proposed - membership -5. Assuming the steering council does not object, a member of the council will - email you asking for: - - - Account details as required by - 🔒 https://github.com/python/voters/ - - Your preferred email address to - subscribe to python-committers with - - A reminder about the `Code of Conduct`_ and to report issues to the PSF - Conduct WG - -6. Once you have provided the pertinent details, your various new privileges - will be turned on -7. Your details will be added to 🔒 https://github.com/python/voters/ -8. They will update the devguide to publicly list your team membership at - :ref:`developers` -9. An announcement email by the steering council member handling your new - membership will be sent to python-committers - -.. _Code of Conduct: https://www.python.org/psf/conduct/ - Mailing Lists -''''''''''''' +============= You are expected to subscribe to python-committers, python-dev, python-checkins, and one of new-bugs-announce or python-bugs-list. See @@ -106,7 +52,7 @@ python-checkins, and one of new-bugs-announce or python-bugs-list. See .. _contributor_agreement: Sign a Contributor Agreement -'''''''''''''''''''''''''''' +============================ Submitting a `contributor form for Python`_ licenses any code you contribute to the Python Software Foundation. While you retain the copyright, giving the PSF @@ -123,9 +69,8 @@ username into your details on the issue tracker. .. _contributor form for Python: https://www.python.org/psf/contrib/ - Pull Request merging -'''''''''''''''''''' +==================== Once you have your commit privileges on GitHub you will be able to accept pull requests on GitHub. You should plan to continue to submit your own @@ -134,8 +79,8 @@ from various things such as automatic integration testing, but you can accept your own pull requests if you feel comfortable doing so. -Responsibilities ----------------- +Expectations +============ As a core developer, there are certain things that are expected of you. @@ -174,5 +119,4 @@ And finally, enjoy yourself! Contributing to open source software should be fun (overall). If you find yourself no longer enjoying the work then either take a break or figure out what you need to do to make it enjoyable again. - .. _PSF Code of Conduct: https://www.python.org/psf/conduct/ diff --git a/c-api.rst b/developer-workflow/c-api.rst similarity index 99% rename from c-api.rst rename to developer-workflow/c-api.rst index b49b579d04..1d042f70c2 100644 --- a/c-api.rst +++ b/developer-workflow/c-api.rst @@ -1,3 +1,5 @@ +.. _c-api: + ======================= Changing Python's C API ======================= diff --git a/communication.rst b/developer-workflow/communication-channels.rst similarity index 95% rename from communication.rst rename to developer-workflow/communication-channels.rst index eacf8c1e6f..aaff8eeae7 100644 --- a/communication.rst +++ b/developer-workflow/communication-channels.rst @@ -1,5 +1,7 @@ +.. _communication-channels: .. _communication: +============================== Following Python's Development ============================== @@ -8,7 +10,8 @@ mailing lists, but also other forms. Standards of behaviour in these communication channels ------------------------------------------------------- +====================================================== + We try to foster environments of mutual respect, tolerance and encouragement, as described in the PSF's `Diversity Statement`_. Abiding by the guidelines in this document and asking questions or posting suggestions in the @@ -22,7 +25,7 @@ in return. .. _mailinglists: Mailing Lists -------------- +============= python-dev_ is the primary mailing list for discussions about Python's development. The list is open to the public and is subscribed to by all core @@ -92,8 +95,10 @@ RSS feed readers. .. _web gateway: https://mail.python.org/archives/ +.. _discourse_discuss: + Discourse (discuss.python.org web forum) ----------------------------------------- +======================================== We have our own `Discourse`_ forum for both developers and users. This forum complements the `python-dev`_, `python-ideas`_, `python-help`_, and @@ -111,7 +116,8 @@ It is also a common venue for the core developer promotion votes (this category is equivalent to the python-committers mailing list). Tutorials for new users -''''''''''''''''''''''' +------------------------- + To start a topic or participate in any discussions in the forum, sign up and create an account using an email address or GitHub account. You can do so by clicking the "Sign Up" button on the top right hand corner of the `Discourse`_ @@ -142,7 +148,7 @@ dots under the message and then on the ⚐ icon. You can also mention the Reading topics -''''''''''''''' +----------------- Click a topic title and read down the list of replies in chronological order, following links or previewing replies and quotes as you go. Use your mouse to scroll the screen, or use the timeline scroll bar on the right which also shows @@ -151,10 +157,11 @@ bottom progress bar to expand it. Notifications -''''''''''''' +------------- Following categories (Category notifications) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +''''''''''''''''''''''''''''''''''''''''''''' + Notifications can be set for individual categories and topics. To change any of these defaults, you can either go to your user preferences, or visit the category page, and use the notification button 🔔 above the topic list, @@ -166,7 +173,8 @@ All categories are set by default in Normal mode where you will only be notified if someone mentions your @name or replies to you. Following individual threads (Topic notifications) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +'''''''''''''''''''''''''''''''''''''''''''''''''' + To follow any individual topics or threads, you can adjust your notifications through the notification button 🔔 found on the right of the topic at the end of the timeline. You can also do so at the bottom of each topic. @@ -174,13 +182,15 @@ Select "Watching" and you will be notified when there is any new updated reply from that particular thread. Customising notifications on user preference -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +'''''''''''''''''''''''''''''''''''''''''''' + To get a bird's eye view of all your customised notifications, you can go to `Preferences of your account `_. This allows you to make adjustments according to categories, users, and tags. Enabling mailing list mode -'''''''''''''''''''''''''' +-------------------------- + In mailing list mode, you will receive one email per post, as happens with traditional mailing lists. This is desirable if you prefer to interact via email, without visiting the forum website. @@ -195,7 +205,7 @@ mailing list mode" and save changes. Discord (private chat server) ------------------------------ +============================= For more real-time discussions, the core development team have a private Discord server available. Core developers, Steering Council members, triagers, and @@ -233,7 +243,7 @@ set a specific `Server Nickname`_ IRC ---- +=== Some core developers still participate in the ``#python-dev`` IRC channel on ``irc.libera.chat``. This is not a place to ask for help with Python, but to @@ -242,7 +252,7 @@ discuss issues related to Python's own development. See also the Blogs ------ +===== Several core developers are active bloggers and discuss Python's development that way. You can find their blogs (and various other developers who use Python) @@ -250,7 +260,7 @@ at https://planetpython.org/. Setting Expectations for Open Source Participation --------------------------------------------------- +================================================== Burn-out is common in open source due to a misunderstanding of what users, contributors, and maintainers should expect from each other. Brett Cannon gave a `talk `_ @@ -258,7 +268,7 @@ about this topic that sets out to help everyone set reasonable expectations of e order to make open source pleasant for everyone involved. Additional Repositories ------------------------ +======================= `Python Core Workflow`_ hosts the codebase for tools such as `cherry_picker`_ and `blurb`_. diff --git a/devcycle.rst b/developer-workflow/development-cycle.rst similarity index 87% rename from devcycle.rst rename to developer-workflow/development-cycle.rst index c20a19713a..4ab70670ae 100644 --- a/devcycle.rst +++ b/developer-workflow/development-cycle.rst @@ -1,3 +1,4 @@ +.. _development-cycle: .. _devcycle: Development Cycle @@ -126,29 +127,7 @@ is frozen and no longer has a branch in the repo. The final state of the end-of-lifed branch is recorded as a tag with the same name as the former branch, e.g. ``3.3`` or ``2.6``. -For reference, here are the Python versions that most recently reached their end-of-life: - -+------------------+--------------+----------------+----------------+----------------------------------+ -| Branch | Schedule | First release | End-of-life | Release manager | -+==================+==============+================+================+==================================+ -| 3.6 | :pep:`494` | 2016-12-23 | 2021-12-23 | Ned Deily | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.5 | :pep:`478` | 2015-09-13 | 2020-09-30 | Larry Hastings | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.4 | :pep:`429` | 2014-03-16 | 2019-03-18 | Larry Hastings | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.3 | :pep:`398` | 2012-09-29 | 2017-09-29 | Georg Brandl, Ned Deily (3.3.7+) | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.2 | :pep:`392` | 2011-02-20 | 2016-02-20 | Georg Brandl | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.1 | :pep:`375` | 2009-06-27 | 2012-04-09 | Benjamin Peterson | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 3.0 | :pep:`361` | 2008-12-03 | 2009-06-27 | Barry Warsaw | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 2.7 | :pep:`373` | 2010-07-03 | 2020-01-01 | Benjamin Peterson | -+------------------+--------------+----------------+----------------+----------------------------------+ -| 2.6 | :pep:`361` | 2008-10-01 | 2013-10-29 | Barry Warsaw | -+------------------+--------------+----------------+----------------+----------------------------------+ +The :ref:`versions` page contains list of active and end-of-life branches. The latest release for each Python version can be found on the `download page `_. @@ -273,6 +252,8 @@ who no longer necessitate this level of access will be removed with notice. Multi-Factor Authentication must be enabled by the user in order to remain an Owner of the Python Organization. +.. _current owners: + Current Owners -------------- diff --git a/extensions.rst b/developer-workflow/extension-modules.rst similarity index 76% rename from extensions.rst rename to developer-workflow/extension-modules.rst index 0d6eee609c..3a5a759a53 100644 --- a/extensions.rst +++ b/developer-workflow/extension-modules.rst @@ -1,7 +1,9 @@ +.. _extension-modules: .. _extensions: -Updating standard library extension modules -=========================================== +================================== +Standard Library Extension Modules +================================== In this section, we could explain how to write a CPython extension with the C language, but the topic can take a complete book. diff --git a/grammar.rst b/developer-workflow/grammar.rst similarity index 96% rename from grammar.rst rename to developer-workflow/grammar.rst index 471d4e827e..7ff1ed12ad 100644 --- a/grammar.rst +++ b/developer-workflow/grammar.rst @@ -1,10 +1,11 @@ .. _grammar: +========================== Changing CPython's Grammar ========================== Abstract --------- +======== There's more to changing Python's grammar than editing :file:`Grammar/python.gram`. Here's a checklist. @@ -20,7 +21,7 @@ For more information on how to use the new parser, check the :ref:`section on how to use CPython's parser `. Checklist ---------- +========= Note: sometimes things mysteriously don't work. Before giving up, try ``make clean``. @@ -48,7 +49,7 @@ Note: sometimes things mysteriously don't work. Before giving up, try ``make cl * :file:`Python/ast_unparse.c` will need changes to unparse AST objects involved with the grammar change ("unparsing" is used to turn annotations into strings per :pep:`563`). -* The :doc:`compiler` has its own page. +* The :ref:`compiler` has its own page. * ``_Unparser`` in the :file:`Lib/ast.py` file may need changes to accommodate any modifications in the AST nodes. diff --git a/developer-workflow/index.rst b/developer-workflow/index.rst new file mode 100644 index 0000000000..bc90a832af --- /dev/null +++ b/developer-workflow/index.rst @@ -0,0 +1,15 @@ +==================== +Development Workflow +==================== + +.. toctree:: + :maxdepth: 5 + + communication-channels + development-cycle + stdlib + extension-modules + c-api + lang-changes + grammar + porting diff --git a/langchanges.rst b/developer-workflow/lang-changes.rst similarity index 99% rename from langchanges.rst rename to developer-workflow/lang-changes.rst index 1a29939f13..108b7b334a 100644 --- a/langchanges.rst +++ b/developer-workflow/lang-changes.rst @@ -1,3 +1,4 @@ +.. _lang-changes: .. _langchanges: Changing the Python Language diff --git a/porting.rst b/developer-workflow/porting.rst similarity index 96% rename from porting.rst rename to developer-workflow/porting.rst index fdf01eef91..3efbb33799 100644 --- a/porting.rst +++ b/developer-workflow/porting.rst @@ -1,7 +1,8 @@ .. _porting: -Porting Python to a new platform --------------------------------- +========================= +Porting to a New Platform +========================= The first step is to familiarize yourself with the development toolchain on the platform in question, notably the C compiler. Make sure you can compile and diff --git a/stdlibchanges.rst b/developer-workflow/stdlib.rst similarity index 99% rename from stdlibchanges.rst rename to developer-workflow/stdlib.rst index 99e1eda289..208629bd41 100644 --- a/stdlibchanges.rst +++ b/developer-workflow/stdlib.rst @@ -1,3 +1,4 @@ +.. _stdlib: .. _stdlibchanges: Adding to the Stdlib diff --git a/documentation/devguide.rst b/documentation/devguide.rst new file mode 100644 index 0000000000..fb69988561 --- /dev/null +++ b/documentation/devguide.rst @@ -0,0 +1,54 @@ +.. _devguide: + +================================== +Helping with the Developer's Guide +================================== + +.. highlight:: console + +The Developer's Guide (what you're reading now) uses the same process as the +main Python documentation, except for some small differences. The source +lives in a `separate repository`_ and bug reports should be submitted to the +`devguide GitHub tracker`_. + +Our devguide workflow uses continuous integration and deployment so changes to +the devguide are normally published when the pull request is merged. Changes +to CPython documentation follow the workflow of a CPython release and are +published in the release. + + +Developer's Guide workflow +========================== + +To submit a :ref:`pull request `, you can fork the +`devguide repo`_ to your GitHub account and clone it using:: + + $ git clone https://github.com//devguide + +For your PR to be accepted, you will also need to sign the +:ref:`contributor agreement `. + +To build the devguide, some additional dependencies are required (most +importantly, `Sphinx`_), and the standard way to install dependencies in +Python projects is to create a virtualenv, and then install dependencies from +a ``requirements.txt`` file. For your convenience, this is all *automated for +you*. To build the devguide on a Unix-like system use:: + + $ make html + +in the checkout directory. On Windows use: + +.. code-block:: doscon + + > .\make html + +You will find the generated files in ``_build/html`` or, if you use +``make htmlview``, the docs will be opened in a browser once the build +completes. Note that ``make check`` runs automatically when you submit +a :ref:`pull request `. You may wish to run ``make check`` +and ``make linkcheck`` to make sure that it runs without errors. + +.. _separate repository: +.. _devguide repo: https://github.com/python/devguide +.. _devguide GitHub tracker: https://github.com/python/devguide/issues +.. _Sphinx: https://www.sphinx-doc.org/ diff --git a/docquality.rst b/documentation/help-documenting.rst similarity index 61% rename from docquality.rst rename to documentation/help-documenting.rst index 353446d24f..c8e320ccd8 100644 --- a/docquality.rst +++ b/documentation/help-documenting.rst @@ -1,5 +1,7 @@ +.. _help-documenting: .. _docquality: +========================== Helping with Documentation ========================== @@ -13,14 +15,13 @@ This high-level **Helping with Documentation** section provides: * an overview of Python's documentation * how to help with documentation issues * information on proofreading -* guidance on contributing to this Developer's Guide -The next chapter, :ref:`Documenting Python `, gives extensive, -detailed information on how to write documentation and submit changes. +You will find extensive and detailed information on how to write documentation +and submit changes on the :ref:`Documenting Python ` page. Python Documentation --------------------- +==================== The :ref:`Documenting Python ` section covers the details of how Python's documentation works. It includes information about the markup @@ -43,7 +44,7 @@ mailing list discusses the documentation toolchain, projects, and standards. Helping with documentation issues ---------------------------------- +================================= If you look at `documentation issues`_ on the `issue tracker`_, you will find various documentation problems that may need work. Issues vary from @@ -57,7 +58,7 @@ If you see a documentation issue that you would like to tackle, you can: * leave a comment on the issue saying you are going to try and create a pull request and roughly how long you think you will take to do so (this allows others to take on the issue if you happen to forget or lose interest). -* submit a :doc:`pull request ` for the issue. +* submit a :ref:`pull request ` for the issue. By following the steps in the :ref:`Quick Guide to Pull Requests `, you will learn the workflow for documentation pull requests. @@ -67,7 +68,7 @@ you will learn the workflow for documentation pull requests. Proofreading ------------- +============ While an issue filed on the `issue tracker`_ means there is a known issue somewhere, that does not mean there are not other issues lurking about in the @@ -82,58 +83,4 @@ a pull request directly. It's best to avoid filing a single issue for an entire section containing multiple problems; instead, file several issues so that it is easier to break the work up for multiple people and more efficient review. - -.. _helping-with-the-developers-guide: - -Helping with the Developer's Guide ----------------------------------- - -.. highlight:: console - -The Developer's Guide (what you're reading now) uses the same process as the -main Python documentation, except for some small differences. The source -lives in a `separate repository`_ and bug reports should be submitted to the -`devguide GitHub tracker`_. - -Our devguide workflow uses continuous integration and deployment so changes to -the devguide are normally published when the pull request is merged. Changes -to CPython documentation follow the workflow of a CPython release and are -published in the release. - - -Developer's Guide workflow --------------------------- - -To submit a :doc:`pull request `, you can fork the -`devguide repo`_ to your GitHub account and clone it using:: - - $ git clone https://github.com//devguide - -For your PR to be accepted, you will also need to sign the -:ref:`contributor agreement `. - -To build the devguide, some additional dependencies are required (most -importantly, `Sphinx`_), and the standard way to install dependencies in -Python projects is to create a virtualenv, and then install dependencies from -a ``requirements.txt`` file. For your convenience, this is all *automated for -you*. To build the devguide on a Unix-like system use:: - - $ make html - -in the checkout directory. On Windows use: - -.. code-block:: doscon - - > .\make html - -You will find the generated files in ``_build/html`` or, if you use -``make htmlview``, the docs will be opened in a browser once the build -completes. Note that ``make check`` runs automatically when you submit -a :doc:`pull request `. You may wish to run ``make check`` -and ``make linkcheck`` to make sure that it runs without errors. - -.. _separate repository: -.. _devguide repo: https://github.com/python/devguide -.. _devguide GitHub tracker: https://github.com/python/devguide/issues -.. _Sphinx: https://www.sphinx-doc.org/ .. _issue tracker: https://github.com/python/cpython/issues diff --git a/documentation/index.rst b/documentation/index.rst new file mode 100644 index 0000000000..3f2512bcfb --- /dev/null +++ b/documentation/index.rst @@ -0,0 +1,13 @@ +============= +Documentation +============= + +.. toctree:: + :maxdepth: 5 + + start-documenting + help-documenting + style-guide + markup + translating + devguide diff --git a/documenting.rst b/documentation/markup.rst similarity index 59% rename from documenting.rst rename to documentation/markup.rst index c52a6dc649..c657770217 100644 --- a/documenting.rst +++ b/documentation/markup.rst @@ -1,281 +1,13 @@ -.. _documenting: +.. _markup: -================== -Documenting Python -================== +======================= +reStructuredText Markup +======================= .. highlight:: rest -The Python language has a substantial body of documentation, much of it -contributed by various authors. The markup used for the Python documentation is -`reStructuredText`_, developed by the `docutils`_ project, amended by custom -directives and using a toolset named `Sphinx`_ to post-process the HTML output. - -This document describes the style guide for our documentation as well as the -custom reStructuredText markup introduced by Sphinx to support Python -documentation and how it should be used. - -The documentation in HTML, PDF or EPUB format is generated from text files -written using the :ref:`reStructuredText format ` and contained in the -:ref:`CPython Git repository `. - -.. _reStructuredText: https://docutils.sourceforge.io/rst.html - -.. note:: - - If you're interested in contributing to Python's documentation, there's no - need to write reStructuredText if you're not so inclined; plain text - contributions are more than welcome as well. Send an e-mail to - docs@python.org or open an issue on the :ref:`tracker `. - - -Introduction -============ - -Python's documentation has long been considered to be good for a free -programming language. There are a number of reasons for this, the most -important being the early commitment of Python's creator, Guido van Rossum, to -providing documentation on the language and its libraries, and the continuing -involvement of the user community in providing assistance for creating and -maintaining documentation. - -The involvement of the community takes many forms, from authoring to bug reports -to just plain complaining when the documentation could be more complete or -easier to use. - -This document is aimed at authors and potential authors of documentation for -Python. More specifically, it is for people contributing to the standard -documentation and developing additional documents using the same tools as the -standard documents. This guide will be less useful for authors using the Python -documentation tools for topics other than Python, and less useful still for -authors not using the tools at all. - -If your interest is in contributing to the Python documentation, but you don't -have the time or inclination to learn reStructuredText and the markup structures -documented here, there's a welcoming place for you among the Python contributors -as well. Any time you feel that you can clarify existing documentation or -provide documentation that's missing, the existing documentation team will -gladly work with you to integrate your text, dealing with the markup for you. -Please don't let the material in this document stand between the documentation -and your desire to help out! - -.. _style-guide: - -Style guide -=========== - -Use of whitespace ------------------ - -All reST files use an indentation of 3 spaces; no tabs are allowed. The -maximum line length is 80 characters for normal text, but tables, deeply -indented code samples and long links may extend beyond that. Code example -bodies should use normal Python 4-space indentation. - -Make generous use of blank lines where applicable; they help group things -together. - -A sentence-ending period may be followed by one or two spaces; while reST -ignores the second space, it is customarily put in by some users, for example -to aid Emacs' auto-fill mode. - -Footnotes ---------- - -Footnotes are generally discouraged, though they may be used when they are the -best way to present specific information. When a footnote reference is added at -the end of the sentence, it should follow the sentence-ending punctuation. The -reST markup should appear something like this:: - - This sentence has a footnote reference. [#]_ This is the next sentence. - -Footnotes should be gathered at the end of a file, or if the file is very long, -at the end of a section. The docutils will automatically create backlinks to -the footnote reference. - -Footnotes may appear in the middle of sentences where appropriate. - -Capitalization --------------- - -.. sidebar:: Sentence case - - Sentence case is a set of capitalization rules used in English - sentences: the first word is always capitalized and other words are - only capitalized if there is a specific rule requiring it. - -In the Python documentation, the use of sentence case in section titles is -preferable, but consistency within a unit is more important than -following this rule. If you add a section to a chapter where most -sections are in title case, you can either convert all titles to -sentence case or use the dominant style in the new section title. - -Sentences that start with a word for which specific rules require -starting it with a lower case letter should be avoided. - -.. note:: - - Sections that describe a library module often have titles in the - form of "modulename --- Short description of the module." In this - case, the description should be capitalized as a stand-alone - sentence. - -Many special names are used in the Python documentation, including the names of -operating systems, programming languages, standards bodies, and the like. Most -of these entities are not assigned any special markup, but the preferred -spellings are given here to aid authors in maintaining the consistency of -presentation in the Python documentation. - -Other terms and words deserve special mention as well; these conventions should -be used to ensure consistency throughout the documentation: - -CPU - For "central processing unit." Many style guides say this should be - spelled out on the first use (and if you must use it, do so!). For - the Python documentation, this abbreviation should be avoided since - there's no reasonable way to predict which occurrence will be the - first seen by the reader. It is better to use the word "processor" - instead. - -POSIX - The name assigned to a particular group of standards. This is always - uppercase. - -Python - The name of our favorite programming language is always capitalized. - -reST - For "reStructuredText," an easy to read, plaintext markup syntax - used to produce Python documentation. When spelled out, it is - always one word and both forms start with a lower case 'r'. - -Unicode - The name of a character coding system. This is always written - capitalized. - -Unix - The name of the operating system developed at AT&T Bell Labs in the early - 1970s. - -Affirmative Tone ----------------- - -The documentation focuses on affirmatively stating what the language does and -how to use it effectively. - -Except for certain security or segfault risks, the docs should avoid -wording along the lines of "feature x is dangerous" or "experts only". These -kinds of value judgments belong in external blogs and wikis, not in the core -documentation. - -Bad example (creating worry in the mind of a reader): - - Warning: failing to explicitly close a file could result in lost data or - excessive resource consumption. Never rely on reference counting to - automatically close a file. - -Good example (establishing confident knowledge in the effective use of the -language): - - A best practice for using files is use a try/finally pair to explicitly - close a file after it is used. Alternatively, using a with-statement can - achieve the same effect. This assures that files are flushed and file - descriptor resources are released in a timely manner. - -Economy of Expression ---------------------- - -More documentation is not necessarily better documentation. Err on the side -of being succinct. - -It is an unfortunate fact that making documentation longer can be an impediment -to understanding and can result in even more ways to misread or misinterpret the -text. Long descriptions full of corner cases and caveats can create the -impression that a function is more complex or harder to use than it actually is. - -Security Considerations (and Other Concerns) --------------------------------------------- - -Some modules provided with Python are inherently exposed to security issues -(e.g. shell injection vulnerabilities) due to the purpose of the module -(e.g. :mod:`ssl`). Littering the documentation of these modules with red -warning boxes for problems that are due to the task at hand, rather than -specifically to Python's support for that task, doesn't make for a good -reading experience. - -Instead, these security concerns should be gathered into a dedicated -"Security Considerations" section within the module's documentation, and -cross-referenced from the documentation of affected interfaces with a note -similar to ``"Please refer to the :ref:`security-considerations` section -for important information on how to avoid common mistakes."``. - -Similarly, if there is a common error that affects many interfaces in a -module (e.g. OS level pipe buffers filling up and stalling child processes), -these can be documented in a "Common Errors" section and cross-referenced -rather than repeated for every affected interface. - -Code Examples -------------- - -Short code examples can be a useful adjunct to understanding. Readers can often -grasp a simple example more quickly than they can digest a formal description in -prose. - -People learn faster with concrete, motivating examples that match the context of -a typical use case. For instance, the :meth:`str.rpartition` method is better -demonstrated with an example splitting the domain from a URL than it would be -with an example of removing the last word from a line of Monty Python dialog. - -The ellipsis for the :py:data:`sys.ps2` secondary interpreter prompt should only -be used sparingly, where it is necessary to clearly differentiate between input -lines and output lines. Besides contributing visual clutter, it makes it -difficult for readers to cut-and-paste examples so they can experiment with -variations. - -Code Equivalents ----------------- - -Giving pure Python code equivalents (or approximate equivalents) can be a useful -adjunct to a prose description. A documenter should carefully weigh whether the -code equivalent adds value. - -A good example is the code equivalent for :func:`all`. The short 4-line code -equivalent is easily digested; it re-emphasizes the early-out behavior; and it -clarifies the handling of the corner-case where the iterable is empty. In -addition, it serves as a model for people wanting to implement a commonly -requested alternative where :func:`all` would return the specific object -evaluating to False whenever the function terminates early. - -A more questionable example is the code for :func:`itertools.groupby`. Its code -equivalent borders on being too complex to be a quick aid to understanding. -Despite its complexity, the code equivalent was kept because it serves as a -model to alternative implementations and because the operation of the "grouper" -is more easily shown in code than in English prose. - -An example of when not to use a code equivalent is for the :func:`oct` function. -The exact steps in converting a number to octal doesn't add value for a user -trying to learn what the function does. - -Audience --------- - -The tone of the tutorial (and all the docs) needs to be respectful of the -reader's intelligence. Don't presume that the readers are stupid. Lay out the -relevant information, show motivating use cases, provide glossary links, and do -your best to connect-the-dots, but don't talk down to them or waste their time. - -The tutorial is meant for newcomers, many of whom will be using the tutorial to -evaluate the language as a whole. The experience needs to be positive and not -leave the reader with worries that something bad will happen if they make a -misstep. The tutorial serves as guide for intelligent and curious readers, -saving details for the how-to guides and other sources. - -Be careful accepting requests for documentation changes from the rare but vocal -category of reader who is looking for vindication for one of their programming -errors ("I made a mistake, therefore the docs must be wrong ..."). Typically, -the documentation wasn't consulted until after the error was made. It is -unfortunate, but typically no documentation edit would have saved the user from -making false assumptions about the language ("I was surprised by ..."). +This document describes the custom reStructuredText markup introduced by Sphinx +to support Python documentation and how it should be used. .. _rst-primer: @@ -405,14 +137,14 @@ Hyperlinks ---------- External links -^^^^^^^^^^^^^^ +'''''''''''''' Use ```Link text `_`` for inline web links. If the link text should be the web address, you don't need special markup at all, the parser finds links and mail addresses in ordinary text. Internal links -^^^^^^^^^^^^^^ +'''''''''''''' Internal linking is done via a special reST role, see the section on specific markup, :ref:`doc-ref-role`. @@ -461,7 +193,8 @@ A directive is a generic block of explicit markup. Besides roles, it is one of the extension mechanisms of reST, and Sphinx makes heavy use of it. Basically, a directive consists of a name, arguments, options and content. (Keep -this terminology in mind, it is used in the next chapter describing custom +this terminology in mind, it is used in `:ref:`the next section +` describing custom directives.) Looking at this example, :: @@ -1507,336 +1240,3 @@ default. They are set in the build configuration file :file:`conf.py`. .. [1] There is a standard ``include`` directive, but it raises errors if the file is not found. This one only emits a warning. - - -.. _building-doc: - -Building the documentation -========================== - -.. highlight:: bash - -The toolset used to build the docs is written in Python and is called Sphinx_. -Sphinx is maintained separately and is not included in this tree. Also needed -are blurb_, a tool to create :file:`Misc/NEWS` on demand; and -python-docs-theme_, the Sphinx theme for the Python documentation. - -To build the documentation, follow the instructions from one of the sections -below. You can view the documentation after building the HTML by pointing -a browser at the file :file:`Doc/build/html/index.html`. - -You are expected to have installed the latest stable version of -Sphinx_ and blurb_ on your system or in a virtualenv_ (which can be -created using ``make venv``), so that the Makefile can find the -``sphinx-build`` command. You can also specify the location of -``sphinx-build`` with the ``SPHINXBUILD`` :command:`make` variable. - - -.. _building-using-make: - -Using make / make.bat ---------------------- - -**On Unix**, run the following from the root of your :ref:`repository clone -` to build the output as HTML:: - - cd Doc - make venv - make html - -or alternatively ``make -C Doc/ venv html``. ``htmlview`` can be used -instead of ``html`` to conveniently open the docs in a browser once the -build completes. - -You can also use ``make help`` to see a list of targets supported by -:command:`make`. Note that ``make check`` is automatically run when -you submit a :doc:`pull request `, so you should make -sure that it runs without errors. - -**On Windows**, a :file:`make.bat` batchfile tries to emulate :command:`make` -as closely as possible, but the venv target is not implemented, so you will -probably want to make sure you are working in a virtual environment before -proceeding, otherwise all dependencies will be automatically installed on your -system. - -When ready, run the following from the root of your :ref:`repository clone -` to build the output as HTML:: - - cd Doc - make html - -You can also use ``make help`` to see a list of targets supported by -:file:`make.bat`. - -See also :file:`Doc/README.rst` for more information. - -Using sphinx-build ------------------- - -Sometimes we directly want to execute the sphinx-build tool instead of through -``make`` (although the latter is still the preferred way). In this case, you can -use the following command line from the ``Doc`` directory (make sure to install -Sphinx_, blurb_ and python-docs-theme_ packages from PyPI):: - - sphinx-build -b . build/ - -where ```` is one of html, text, latex, or htmlhelp (for explanations -see the make targets above). - -.. _translating: - -Translating -=========== - -Python documentation translations are governed by :PEP:`545`. -They are built by `docsbuild-scripts -`__ and hosted on -docs.python.org. There are several documentation translations already -in production; others are works in progress. - -+-----------------+-------------------------------+----------------------------+ -| Language | Contact | Links | -+=================+===============================+============================+ -| Arabic (ar) | `Abdur-Rahmaan Janhangeer | `GitHub `_ | -| | `_ | | -+-----------------+-------------------------------+----------------------------+ -| Bengali as | `Kushal Das `_ | `GitHub `_ | -| spoken in | | | -| India (bn_IN) | | | -+-----------------+-------------------------------+----------------------------+ -| French (fr) | `Julien Palard (@JulienPalard)| `GitHub `_ | -| | `_ | | -+-----------------+-------------------------------+----------------------------+ -| Hindi as spoken | | `GitHub `_ | -| in India (hi_IN)| | | -+-----------------+-------------------------------+----------------------------+ -| Hungarian (hu) | `Tamás Bajusz (@gbtami) | `GitHub `_ | -| | `_ | `Mailing List `_ | -+-----------------+-------------------------------+----------------------------+ -| Indonesian (id) | `Oon Arfiandwi `_ | `GitHub `_ | -+-----------------+-------------------------------+----------------------------+ -| Italian (it) | | `mail `_ | -+-----------------+-------------------------------+----------------------------+ -| Japanese (ja) | `Kinebuchi Tomohiko | `GitHub `_ | -| | (@cocoatomo) `_| `Doc `_ | -+-----------------+-------------------------------+----------------------------+ -| Korean (ko) | | `GitHub `_ | -| | | `Doc `_ | -+-----------------+-------------------------------+----------------------------+ -| Marathi (mr) | `Sanket Garade | `GitHub `_ | -| | `_ | | -+-----------------+-------------------------------+----------------------------+ -| Lithuanian (lt) | | `mail `_ | -+-----------------+-------------------------------+----------------------------+ -| Persian (fa) | `Komeil Parseh (@mmdbalkhi) | `GitHub `_ | -| | `_ | | -+-----------------+-------------------------------+----------------------------+ -| Polish (pl) | | `GitHub `_ | -| | | `Translations `_ | -| | | `Doc `_ | -| | | `mail `_ | -+-----------------+-------------------------------+----------------------------+ -| Portuguese (pt) | Gustavo Toffo | | -+-----------------+-------------------------------+----------------------------+ -| Portuguese | Marco Rougeth | `GitHub `_ | -| as spoken | | `Wiki `_ | -| in Brasil | | `Telegram `_ | -| (pt-br) | | `article `_| -+-----------------+-------------------------------+----------------------------+ -| Russian (ru) | | `mail `_ | -+-----------------+-------------------------------+----------------------------+ -| Simplified | `Shengjing Zhu `_ | `Transifex `_ | -| Chinese | | `GitHub `_ | -| (zh-cn) | | `Doc `_ | -+-----------------+-------------------------------+----------------------------+ -| Spanish (es) | Raúl Cumplido | `GitHub `_ | -+-----------------+-------------------------------+----------------------------+ -| Traditional | `王威翔 Matt Wang | `GitHub `_ | -| Chinese | `_, | `Doc `_ | -| (zh-tw) | Josix Wang | | -+-----------------+-------------------------------+----------------------------+ -| Turkish (tr) | `Ege Akman (@egeakman) | `GitHub `_ | -| | `_ | | -+-----------------+-------------------------------+----------------------------+ - -.. _article_pt_br: https://rgth.co/blog/python-ptbr-cenario-atual/ -.. _gh_cocoatomo: https://github.com/cocoatomo -.. _gh_gbtami: https://github.com/gbtami -.. _gh_kushal: https://github.com/Kushal997-das -.. _gh_mdk: https://github.com/JulienPalard -.. _gh_mmdbalkhi: https://github.com/mmdbalkhi -.. _gh_oonid: https://github.com/oonid -.. _gh_osdotsystem: https://github.com/Abdur-rahmaanJ -.. _gh_zhsj: https://github.com/zhsj -.. _gh_mattwang44: https://github.com/mattwang44 -.. _email_egeakman: mailto:egeakmanegeakman@hotmail.com -.. _email_garade: mailto:garade@pm.me -.. _chat_pt_br: https://t.me/pybr_i18n -.. _doc_ja: https://docs.python.org/ja/ -.. _doc_ko: https://docs.python.org/ko/ -.. _doc_pl: https://docs.python.org/pl/ -.. _doc_zh_cn: https://docs.python.org/zh-cn/ -.. _doc_zh_tw: https://docs.python.org/zh-tw/ -.. _github_ar: https://github.com/Abdur-rahmaanJ/python-docs-ar -.. _github_bn_in: https://github.com/python/python-docs-bn-in -.. _github_es: https://github.com/python/python-docs-es -.. _github_fa: https://github.com/mmdbalkhi/python-docs-fa -.. _github_fr: https://github.com/python/python-docs-fr -.. _github_hi_in: https://github.com/CuriousLearner/python-docs-hi-in -.. _github_hu: https://github.com/python/python-docs-hu -.. _github_id: https://github.com/python/python-docs-id -.. _github_ja: https://github.com/python/python-docs-ja -.. _github_ko: https://github.com/python/python-docs-ko -.. _github_mr: https://github.com/sanketgarade/python-doc-mr -.. _github_pl: https://github.com/python/python-docs-pl -.. _github_pt_br: https://github.com/python/python-docs-pt-br -.. _github_tr: https://github.com/python-docs-tr/python-docs-tr -.. _github_zh_cn: https://github.com/python/python-docs-zh-cn -.. _github_zh_tw: https://github.com/python/python-docs-zh-tw -.. _list_hu: https://mail.python.org/pipermail/python-hu -.. _mail_it: https://mail.python.org/pipermail/doc-sig/2019-April/004114.html -.. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html -.. _mail_pl: https://mail.python.org/pipermail/doc-sig/2019-April/004106.html -.. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html -.. _tx_pl: https://www.transifex.com/python-doc/python-newest/ -.. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/ -.. _wiki_pt_br: https://python.org.br/traducao/ - -Starting a new translation --------------------------- - -First subscribe to the `translation mailing list `_, -and introduce yourself and the translation you're starting. Translations -fall under the aegis of the `PSF Translation Workgroup `_ - -Then you can bootstrap your new translation by using our `cookiecutter -`__. - -The important steps look like this: - -- Create the GitHub repo (anywhere) with the right hierarchy (using the - cookiecutter). -- Gather people to help you translate. You can't do it alone. -- You can use any tool to translate, as long as you can synchronize with git. - Some use Transifex, and some use only GitHub. You can choose another - way if you like; it's up to you. -- Ensure we update this page to reflect your work and progress, either via a - PR or by asking on the `translation mailing list `_. -- When ``bugs.html``, ``tutorial``, and ``library/functions`` are 100% - completed, ask on the `translation mailing list `_ for - your language to be added in the language picker on docs.python.org. - - -PEP 545 summary: ----------------- - -Here are the essential points of :PEP:`545`: - -- Each translation is assigned an appropriate lowercased language tag, - with an optional region subtag, and joined with a dash, like - ``pt-br`` or ``fr``. - -- Each translation is under CC0 and marked as such in the README (as in - the cookiecutter). - -- Translation files are hosted on - ``https://github.com/python/python-docs-{LANGUAGE_TAG}`` (not - mandatory to start a translation, but mandatory to land on - ``docs.python.org``). - -- Translations having completed ``tutorial/``, ``library/stdtypes`` - and ``library/functions`` are hosted on - ``https://docs.python.org/{LANGUAGE_TAG}/{VERSION_TAG}/``. - - -How to get help ---------------- - -Discussions about translations occur on the `translation mailing list `_, -and there's a `Libera.Chat IRC `_ channel, -``#python-doc``. - - -Translation FAQ ---------------- - -Which version of the Python documentation should be translated? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Consensus is to work on current stable. You can then propagate your -translation from one branch to another using `pomerge -`__. - - -Are there some tools to help in managing the repo? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Here's what we're using: - -- `pomerge `__ to propagate translations - from one file to others. -- `pospell `__ to check for typos in ``.po`` files. -- `powrap `__ to rewrap the ``.po`` files - before committing. This helps keep git diffs short. -- `potodo `__ to list what needs to be translated. - - -How is a coordinator elected? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There is no election; each translation has to sort this out. Here are some suggestions. - -- Coordinator requests are to be public on the `translation mailing list `_. -- If the given language has a native core dev, the core dev has their - say on the choice. -- Anyone who wants to become coordinator for their native language and shows - motivation by translating and building a community will be named - coordinator. -- In case of concurrency between two persons, no one will sort this out - for you. It is up to you two to organize a local election or whatever is - needed to sort this out. -- If a coordinator becomes inactive or unreachable for a long - period of time, someone else can ask for a takeover on the `translation mailing list `_. - - -The entry for my translation is missing/not up to date on this page -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Ask on the `translation mailing list `_, or better, make a PR on the `devguide -`__. - - -I have a translation, but it's not in git. What should I do? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You can ask for help on the `translation mailing list `_, and -the team will help you create an appropriate repository. You can still use tools like transifex, -if you like. - - -My git hierarchy does not match yours. Can I keep it? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -No, inside the ``github.com/python`` organization we’ll all have the -exact same hierarchy so bots will be able to build all of our -translations. So you may have to convert from one hierarchy to another. -Ask for help on the `translation mailing list `_ if you’re -not sure on how to do it. - - -What hierarchy should I use in my GitHub repository? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As for every project, we have a *branch* per version. We store ``.po`` -files in the root of the repository using the ``gettext_compact=0`` -style. - - - -.. _docutils: https://docutils.sourceforge.io/ -.. _python-docs-theme: https://pypi.org/project/python-docs-theme/ -.. _Sphinx: https://www.sphinx-doc.org/ -.. _virtualenv: https://virtualenv.pypa.io/ -.. _blurb: https://pypi.org/project/blurb/ -.. _translation_wg: https://wiki.python.org/psf/TranslationWG/Charter -.. _translation_ml: https://mail.python.org/mailman3/lists/translation.python.org/ diff --git a/documentation/start-documenting.rst b/documentation/start-documenting.rst new file mode 100644 index 0000000000..8a0db24045 --- /dev/null +++ b/documentation/start-documenting.rst @@ -0,0 +1,151 @@ +.. _start-documenting: +.. _documenting: + +=============== +Getting Started +=============== + +.. highlight:: rest + +The Python language has a substantial body of documentation, much of it +contributed by various authors. The markup used for the Python documentation is +`reStructuredText`_, developed by the `docutils`_ project, amended by custom +directives and using a toolset named `Sphinx`_ to post-process the HTML output. + +The documentation in HTML, PDF or EPUB format is generated from text files +written using the :ref:`reStructuredText format ` and contained in the +:ref:`CPython Git repository `. + +.. _reStructuredText: https://docutils.sourceforge.io/rst.html + +.. note:: + + If you're interested in contributing to Python's documentation, there's no + need to write reStructuredText if you're not so inclined; plain text + contributions are more than welcome as well. Send an e-mail to + docs@python.org or open an issue on the :ref:`tracker `. + + +Introduction +============ + +Python's documentation has long been considered to be good for a free +programming language. There are a number of reasons for this, the most +important being the early commitment of Python's creator, Guido van Rossum, to +providing documentation on the language and its libraries, and the continuing +involvement of the user community in providing assistance for creating and +maintaining documentation. + +The involvement of the community takes many forms, from authoring to bug reports +to just plain complaining when the documentation could be more complete or +easier to use. + +This section is aimed at authors and potential authors of documentation for +Python. More specifically, it is for people contributing to the standard +documentation and developing additional documents using the same tools as the +standard documents. This guide will be less useful for authors using the Python +documentation tools for topics other than Python, and less useful still for +authors not using the tools at all. + +If your interest is in contributing to the Python documentation, but you don't +have the time or inclination to learn reStructuredText and the markup structures +documented here, there's a welcoming place for you among the Python contributors +as well. Any time you feel that you can clarify existing documentation or +provide documentation that's missing, the existing documentation team will +gladly work with you to integrate your text, dealing with the markup for you. +Please don't let the material in this section stand between the documentation +and your desire to help out! + + +.. _building-doc: + +Building the documentation +========================== + +.. highlight:: bash + +The toolset used to build the docs is written in Python and is called Sphinx_. +Sphinx is maintained separately and is not included in this tree. Also needed +are blurb_, a tool to create :file:`Misc/NEWS` on demand; and +python-docs-theme_, the Sphinx theme for the Python documentation. + +To build the documentation, follow the instructions from one of the sections +below. You can view the documentation after building the HTML by pointing +a browser at the file :file:`Doc/build/html/index.html`. + +You are expected to have installed the latest stable version of +Sphinx_ and blurb_ on your system or in a virtualenv_ (which can be +created using ``make venv``), so that the Makefile can find the +``sphinx-build`` command. You can also specify the location of +``sphinx-build`` with the ``SPHINXBUILD`` :command:`make` variable. + + +.. _building-using-make: + +Using make / make.bat +--------------------- + +**On Unix**, run the following from the root of your :ref:`repository clone +` to build the output as HTML:: + + cd Doc + make venv + make html + +or alternatively ``make -C Doc/ venv html``. ``htmlview`` can be used +instead of ``html`` to conveniently open the docs in a browser once the +build completes. + +You can also use ``make help`` to see a list of targets supported by +:command:`make`. Note that ``make check`` is automatically run when +you submit a :ref:`pull request `, so you should make +sure that it runs without errors. + +**On Windows**, a :file:`make.bat` batchfile tries to emulate :command:`make` +as closely as possible, but the venv target is not implemented, so you will +probably want to make sure you are working in a virtual environment before +proceeding, otherwise all dependencies will be automatically installed on your +system. + +When ready, run the following from the root of your :ref:`repository clone +` to build the output as HTML:: + + cd Doc + make html + +You can also use ``make help`` to see a list of targets supported by +:file:`make.bat`. + +See also :file:`Doc/README.rst` for more information. + +Using sphinx-build +------------------ + +Sometimes we directly want to execute the sphinx-build tool instead of through +``make`` (although the latter is still the preferred way). In this case, you can +use the following command line from the ``Doc`` directory (make sure to install +Sphinx_, blurb_ and python-docs-theme_ packages from PyPI):: + + sphinx-build -b . build/ + +where ```` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). + + +.. _docutils: https://docutils.sourceforge.io/ +.. _python-docs-theme: https://pypi.org/project/python-docs-theme/ +.. _Sphinx: https://www.sphinx-doc.org/ +.. _virtualenv: https://virtualenv.pypa.io/ +.. _blurb: https://pypi.org/project/blurb/ + + +Style Guide +=========== + +Moved to :doc:`style-guide` + + +Translating +=========== + +Moved to :doc:`translating` diff --git a/documentation/style-guide.rst b/documentation/style-guide.rst new file mode 100644 index 0000000000..4ebe9f4dc9 --- /dev/null +++ b/documentation/style-guide.rst @@ -0,0 +1,223 @@ +.. _style-guide: + +=========== +Style Guide +=========== + +.. highlight:: rest + +This document describes the style guide for our documentation. + + +Use of whitespace +================= + +All reST files use an indentation of 3 spaces; no tabs are allowed. The +maximum line length is 80 characters for normal text, but tables, deeply +indented code samples and long links may extend beyond that. Code example +bodies should use normal Python 4-space indentation. + +Make generous use of blank lines where applicable; they help group things +together. + +A sentence-ending period may be followed by one or two spaces; while reST +ignores the second space, it is customarily put in by some users, for example +to aid Emacs' auto-fill mode. + +Footnotes +========= + +Footnotes are generally discouraged, though they may be used when they are the +best way to present specific information. When a footnote reference is added at +the end of the sentence, it should follow the sentence-ending punctuation. The +reST markup should appear something like this:: + + This sentence has a footnote reference. [#]_ This is the next sentence. + +Footnotes should be gathered at the end of a file, or if the file is very long, +at the end of a section. The docutils will automatically create backlinks to +the footnote reference. + +Footnotes may appear in the middle of sentences where appropriate. + +Capitalization +============== + +.. sidebar:: Sentence case + + Sentence case is a set of capitalization rules used in English + sentences: the first word is always capitalized and other words are + only capitalized if there is a specific rule requiring it. + +In the Python documentation, the use of sentence case in section titles is +preferable, but consistency within a unit is more important than +following this rule. If you add a section to a chapter where most +sections are in title case, you can either convert all titles to +sentence case or use the dominant style in the new section title. + +Sentences that start with a word for which specific rules require +starting it with a lower case letter should be avoided. + +.. note:: + + Sections that describe a library module often have titles in the + form of "modulename --- Short description of the module." In this + case, the description should be capitalized as a stand-alone + sentence. + +Many special names are used in the Python documentation, including the names of +operating systems, programming languages, standards bodies, and the like. Most +of these entities are not assigned any special markup, but the preferred +spellings are given here to aid authors in maintaining the consistency of +presentation in the Python documentation. + +Other terms and words deserve special mention as well; these conventions should +be used to ensure consistency throughout the documentation: + +CPU + For "central processing unit." Many style guides say this should be + spelled out on the first use (and if you must use it, do so!). For + the Python documentation, this abbreviation should be avoided since + there's no reasonable way to predict which occurrence will be the + first seen by the reader. It is better to use the word "processor" + instead. + +POSIX + The name assigned to a particular group of standards. This is always + uppercase. + +Python + The name of our favorite programming language is always capitalized. + +reST + For "reStructuredText," an easy to read, plaintext markup syntax + used to produce Python documentation. When spelled out, it is + always one word and both forms start with a lower case 'r'. + +Unicode + The name of a character coding system. This is always written + capitalized. + +Unix + The name of the operating system developed at AT&T Bell Labs in the early + 1970s. + +Affirmative Tone +================ + +The documentation focuses on affirmatively stating what the language does and +how to use it effectively. + +Except for certain security or segfault risks, the docs should avoid +wording along the lines of "feature x is dangerous" or "experts only". These +kinds of value judgments belong in external blogs and wikis, not in the core +documentation. + +Bad example (creating worry in the mind of a reader): + + Warning: failing to explicitly close a file could result in lost data or + excessive resource consumption. Never rely on reference counting to + automatically close a file. + +Good example (establishing confident knowledge in the effective use of the +language): + + A best practice for using files is use a try/finally pair to explicitly + close a file after it is used. Alternatively, using a with-statement can + achieve the same effect. This assures that files are flushed and file + descriptor resources are released in a timely manner. + +Economy of Expression +===================== + +More documentation is not necessarily better documentation. Err on the side +of being succinct. + +It is an unfortunate fact that making documentation longer can be an impediment +to understanding and can result in even more ways to misread or misinterpret the +text. Long descriptions full of corner cases and caveats can create the +impression that a function is more complex or harder to use than it actually is. + +Security Considerations (and Other Concerns) +============================================ + +Some modules provided with Python are inherently exposed to security issues +(e.g. shell injection vulnerabilities) due to the purpose of the module +(e.g. :mod:`ssl`). Littering the documentation of these modules with red +warning boxes for problems that are due to the task at hand, rather than +specifically to Python's support for that task, doesn't make for a good +reading experience. + +Instead, these security concerns should be gathered into a dedicated +"Security Considerations" section within the module's documentation, and +cross-referenced from the documentation of affected interfaces with a note +similar to ``"Please refer to the :ref:`security-considerations` section +for important information on how to avoid common mistakes."``. + +Similarly, if there is a common error that affects many interfaces in a +module (e.g. OS level pipe buffers filling up and stalling child processes), +these can be documented in a "Common Errors" section and cross-referenced +rather than repeated for every affected interface. + +Code Examples +============= + +Short code examples can be a useful adjunct to understanding. Readers can often +grasp a simple example more quickly than they can digest a formal description in +prose. + +People learn faster with concrete, motivating examples that match the context of +a typical use case. For instance, the :meth:`str.rpartition` method is better +demonstrated with an example splitting the domain from a URL than it would be +with an example of removing the last word from a line of Monty Python dialog. + +The ellipsis for the :py:data:`sys.ps2` secondary interpreter prompt should only +be used sparingly, where it is necessary to clearly differentiate between input +lines and output lines. Besides contributing visual clutter, it makes it +difficult for readers to cut-and-paste examples so they can experiment with +variations. + +Code Equivalents +================ + +Giving pure Python code equivalents (or approximate equivalents) can be a useful +adjunct to a prose description. A documenter should carefully weigh whether the +code equivalent adds value. + +A good example is the code equivalent for :func:`all`. The short 4-line code +equivalent is easily digested; it re-emphasizes the early-out behavior; and it +clarifies the handling of the corner-case where the iterable is empty. In +addition, it serves as a model for people wanting to implement a commonly +requested alternative where :func:`all` would return the specific object +evaluating to False whenever the function terminates early. + +A more questionable example is the code for :func:`itertools.groupby`. Its code +equivalent borders on being too complex to be a quick aid to understanding. +Despite its complexity, the code equivalent was kept because it serves as a +model to alternative implementations and because the operation of the "grouper" +is more easily shown in code than in English prose. + +An example of when not to use a code equivalent is for the :func:`oct` function. +The exact steps in converting a number to octal doesn't add value for a user +trying to learn what the function does. + +Audience +======== + +The tone of the tutorial (and all the docs) needs to be respectful of the +reader's intelligence. Don't presume that the readers are stupid. Lay out the +relevant information, show motivating use cases, provide glossary links, and do +your best to connect-the-dots, but don't talk down to them or waste their time. + +The tutorial is meant for newcomers, many of whom will be using the tutorial to +evaluate the language as a whole. The experience needs to be positive and not +leave the reader with worries that something bad will happen if they make a +misstep. The tutorial serves as guide for intelligent and curious readers, +saving details for the how-to guides and other sources. + +Be careful accepting requests for documentation changes from the rare but vocal +category of reader who is looking for vindication for one of their programming +errors ("I made a mistake, therefore the docs must be wrong ..."). Typically, +the documentation wasn't consulted until after the error was made. It is +unfortunate, but typically no documentation edit would have saved the user from +making false assumptions about the language ("I was surprised by ..."). diff --git a/documentation/translating.rst b/documentation/translating.rst new file mode 100644 index 0000000000..fff7a53e4e --- /dev/null +++ b/documentation/translating.rst @@ -0,0 +1,253 @@ +.. _translating: + +=========== +Translating +=========== + +.. highlight:: rest + +Python documentation translations are governed by :PEP:`545`. +They are built by `docsbuild-scripts +`__ and hosted on +docs.python.org. There are several documentation translations already +in production; others are works in progress. + ++-----------------+-------------------------------+----------------------------+ +| Language | Contact | Links | ++=================+===============================+============================+ +| Arabic (ar) | `Abdur-Rahmaan Janhangeer | `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ +| Bengali as | `Kushal Das `_ | `GitHub `_ | +| spoken in | | | +| India (bn_IN) | | | ++-----------------+-------------------------------+----------------------------+ +| French (fr) | `Julien Palard (@JulienPalard)| `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ +| Hindi as spoken | | `GitHub `_ | +| in India (hi_IN)| | | ++-----------------+-------------------------------+----------------------------+ +| Hungarian (hu) | `Tamás Bajusz (@gbtami) | `GitHub `_ | +| | `_ | `Mailing List `_ | ++-----------------+-------------------------------+----------------------------+ +| Indonesian (id) | `Oon Arfiandwi `_ | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ +| Italian (it) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Japanese (ja) | `Kinebuchi Tomohiko | `GitHub `_ | +| | (@cocoatomo) `_| `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Korean (ko) | | `GitHub `_ | +| | | `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Marathi (mr) | `Sanket Garade | `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ +| Lithuanian (lt) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Persian (fa) | `Komeil Parseh (@mmdbalkhi) | `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ +| Polish (pl) | | `GitHub `_ | +| | | `Translations `_ | +| | | `Doc `_ | +| | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Portuguese (pt) | Gustavo Toffo | | ++-----------------+-------------------------------+----------------------------+ +| Portuguese | Marco Rougeth | `GitHub `_ | +| as spoken | | `Wiki `_ | +| in Brasil | | `Telegram `_ | +| (pt-br) | | `article `_| ++-----------------+-------------------------------+----------------------------+ +| Russian (ru) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Simplified | `Shengjing Zhu `_ | `Transifex `_ | +| Chinese | | `GitHub `_ | +| (zh-cn) | | `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Spanish (es) | Raúl Cumplido | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ +| Traditional | `王威翔 Matt Wang | `GitHub `_ | +| Chinese | `_, | `Doc `_ | +| (zh-tw) | Josix Wang | | ++-----------------+-------------------------------+----------------------------+ +| Turkish (tr) | `Ege Akman (@egeakman) | `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ + +.. _article_pt_br: https://rgth.co/blog/python-ptbr-cenario-atual/ +.. _gh_cocoatomo: https://github.com/cocoatomo +.. _gh_gbtami: https://github.com/gbtami +.. _gh_kushal: https://github.com/Kushal997-das +.. _gh_mdk: https://github.com/JulienPalard +.. _gh_mmdbalkhi: https://github.com/mmdbalkhi +.. _gh_oonid: https://github.com/oonid +.. _gh_osdotsystem: https://github.com/Abdur-rahmaanJ +.. _gh_zhsj: https://github.com/zhsj +.. _gh_mattwang44: https://github.com/mattwang44 +.. _email_egeakman: mailto:egeakmanegeakman@hotmail.com +.. _email_garade: mailto:garade@pm.me +.. _chat_pt_br: https://t.me/pybr_i18n +.. _doc_ja: https://docs.python.org/ja/ +.. _doc_ko: https://docs.python.org/ko/ +.. _doc_pl: https://docs.python.org/pl/ +.. _doc_zh_cn: https://docs.python.org/zh-cn/ +.. _doc_zh_tw: https://docs.python.org/zh-tw/ +.. _github_ar: https://github.com/Abdur-rahmaanJ/python-docs-ar +.. _github_bn_in: https://github.com/python/python-docs-bn-in +.. _github_es: https://github.com/python/python-docs-es +.. _github_fa: https://github.com/mmdbalkhi/python-docs-fa +.. _github_fr: https://github.com/python/python-docs-fr +.. _github_hi_in: https://github.com/CuriousLearner/python-docs-hi-in +.. _github_hu: https://github.com/python/python-docs-hu +.. _github_id: https://github.com/python/python-docs-id +.. _github_ja: https://github.com/python/python-docs-ja +.. _github_ko: https://github.com/python/python-docs-ko +.. _github_mr: https://github.com/sanketgarade/python-doc-mr +.. _github_pl: https://github.com/python/python-docs-pl +.. _github_pt_br: https://github.com/python/python-docs-pt-br +.. _github_tr: https://github.com/python-docs-tr/python-docs-tr +.. _github_zh_cn: https://github.com/python/python-docs-zh-cn +.. _github_zh_tw: https://github.com/python/python-docs-zh-tw +.. _list_hu: https://mail.python.org/pipermail/python-hu +.. _mail_it: https://mail.python.org/pipermail/doc-sig/2019-April/004114.html +.. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html +.. _mail_pl: https://mail.python.org/pipermail/doc-sig/2019-April/004106.html +.. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html +.. _tx_pl: https://www.transifex.com/python-doc/python-newest/ +.. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/ +.. _wiki_pt_br: https://python.org.br/traducao/ + +Starting a new translation +========================== + +First subscribe to the `translation mailing list `_, +and introduce yourself and the translation you're starting. Translations +fall under the aegis of the `PSF Translation Workgroup `_ + +Then you can bootstrap your new translation by using our `cookiecutter +`__. + +The important steps look like this: + +- Create the GitHub repo (anywhere) with the right hierarchy (using the + cookiecutter). +- Gather people to help you translate. You can't do it alone. +- You can use any tool to translate, as long as you can synchronize with git. + Some use Transifex, and some use only GitHub. You can choose another + way if you like; it's up to you. +- Ensure we update this page to reflect your work and progress, either via a + PR or by asking on the `translation mailing list `_. +- When ``bugs.html``, ``tutorial``, and ``library/functions`` are 100% + completed, ask on the `translation mailing list `_ for + your language to be added in the language picker on docs.python.org. + + +PEP 545 summary +=============== + +Here are the essential points of :PEP:`545`: + +- Each translation is assigned an appropriate lowercased language tag, + with an optional region subtag, and joined with a dash, like + ``pt-br`` or ``fr``. + +- Each translation is under CC0 and marked as such in the README (as in + the cookiecutter). + +- Translation files are hosted on + ``https://github.com/python/python-docs-{LANGUAGE_TAG}`` (not + mandatory to start a translation, but mandatory to land on + ``docs.python.org``). + +- Translations having completed ``tutorial/``, ``library/stdtypes`` + and ``library/functions`` are hosted on + ``https://docs.python.org/{LANGUAGE_TAG}/{VERSION_TAG}/``. + + +How to get help +=============== + +Discussions about translations occur on the `translation mailing list `_, +and there's a `Libera.Chat IRC `_ channel, +``#python-doc``. + + +Translation FAQ +=============== + +Which version of the Python documentation should be translated? +--------------------------------------------------------------- + +Consensus is to work on current stable. You can then propagate your +translation from one branch to another using `pomerge +`__. + + +Are there some tools to help in managing the repo? +-------------------------------------------------- + +Here's what we're using: + +- `pomerge `__ to propagate translations + from one file to others. +- `pospell `__ to check for typos in ``.po`` files. +- `powrap `__ to rewrap the ``.po`` files + before committing. This helps keep git diffs short. +- `potodo `__ to list what needs to be translated. + + +How is a coordinator elected? +----------------------------- + +There is no election; each translation has to sort this out. Here are some suggestions. + +- Coordinator requests are to be public on the `translation mailing list `_. +- If the given language has a native core dev, the core dev has their + say on the choice. +- Anyone who wants to become coordinator for their native language and shows + motivation by translating and building a community will be named + coordinator. +- In case of concurrency between two persons, no one will sort this out + for you. It is up to you two to organize a local election or whatever is + needed to sort this out. +- If a coordinator becomes inactive or unreachable for a long + period of time, someone else can ask for a takeover on the `translation mailing list `_. + + +The entry for my translation is missing/not up to date on this page +------------------------------------------------------------------- + +Ask on the `translation mailing list `_, or better, make a PR on the `devguide +`__. + + +I have a translation, but it's not in git. What should I do? +------------------------------------------------------------ + +You can ask for help on the `translation mailing list `_, and +the team will help you create an appropriate repository. You can still use tools like transifex, +if you like. + + +My git hierarchy does not match yours. Can I keep it? +----------------------------------------------------- + +No, inside the ``github.com/python`` organization we’ll all have the +exact same hierarchy so bots will be able to build all of our +translations. So you may have to convert from one hierarchy to another. +Ask for help on the `translation mailing list `_ if you’re +not sure on how to do it. + + +What hierarchy should I use in my GitHub repository? +---------------------------------------------------- + +As for every project, we have a *branch* per version. We store ``.po`` +files in the root of the repository using the ``gettext_compact=0`` +style. + +.. _translation_wg: https://wiki.python.org/psf/TranslationWG/Charter +.. _translation_ml: https://mail.python.org/mailman3/lists/translation.python.org/ diff --git a/fixingissues.rst b/getting-started/fixing-issues.rst similarity index 96% rename from fixingissues.rst rename to getting-started/fixing-issues.rst index 1486275787..70d33dd77f 100644 --- a/fixingissues.rst +++ b/getting-started/fixing-issues.rst @@ -1,5 +1,7 @@ +.. _fixing-issues: .. _fixingissues: +================================= Fixing "easy" Issues (and Beyond) ================================= diff --git a/help.rst b/getting-started/getting-help.rst similarity index 96% rename from help.rst rename to getting-started/getting-help.rst index 5de9c51a0a..60284d955a 100644 --- a/help.rst +++ b/getting-started/getting-help.rst @@ -1,3 +1,4 @@ +.. _getting-help: .. _help: Where to Get Help @@ -15,7 +16,7 @@ question. Discourse --------- -Python has a hosted `Discourse`_ instance. This forum has many different +Python has a hosted :ref:`Discourse ` instance. This forum has many different categories and most core development discussions take place in the open forum categories for `PEPs`_ and `Core Development`_ . Most categories are open for all users to read and post with the exception of @@ -25,7 +26,7 @@ categories, such as `Core Workflow `_. .. seealso:: - `Discourse `_ + :ref:`Discourse ` on how to get started. diff --git a/gitbootcamp.rst b/getting-started/git-boot-camp.rst similarity index 99% rename from gitbootcamp.rst rename to getting-started/git-boot-camp.rst index b76271a2b0..259fc1f1f3 100644 --- a/gitbootcamp.rst +++ b/getting-started/git-boot-camp.rst @@ -1,10 +1,11 @@ -.. highlight:: console - +.. _git-boot-camp: .. _gitbootcamp: Git Bootcamp and Cheat Sheet ============================ +.. highlight:: console + .. note:: This section provides instructions on common tasks in CPython's @@ -184,6 +185,8 @@ rename your local branch as follows:: git branch -u upstream/main main +.. _commit-changes: + Staging and Committing Files ---------------------------- diff --git a/getting-started/index.rst b/getting-started/index.rst new file mode 100644 index 0000000000..420f1231c9 --- /dev/null +++ b/getting-started/index.rst @@ -0,0 +1,12 @@ +=============== +Getting Started +=============== + +.. toctree:: + :maxdepth: 5 + + setup-building + fixing-issues + git-boot-camp + pull-request-lifecycle + getting-help diff --git a/pullrequest.rst b/getting-started/pull-request-lifecycle.rst similarity index 97% rename from pullrequest.rst rename to getting-started/pull-request-lifecycle.rst index a10df969b0..5f2a4262db 100644 --- a/pullrequest.rst +++ b/getting-started/pull-request-lifecycle.rst @@ -1,12 +1,15 @@ +.. _pull-request-lifecycle: .. _patch: +.. _pullrequest: +=========================== Lifecycle of a Pull Request =========================== .. highlight:: bash Introduction ------------- +============ CPython uses a workflow based on pull requests. What this means is that you create a branch in Git, make your changes, push those changes @@ -17,7 +20,7 @@ the official CPython repository (``upstream``). .. _pullrequest-quickguide: Quick Guide ------------ +=========== `Clear communication`_ is key to contributing to any project, especially an `Open Source`_ project like CPython. @@ -67,7 +70,7 @@ Here is a quick overview of how you can contribute to CPython: .. _pullrequest-steps: Step-by-step Guide ------------------- +================== You should have already :ref:`set up your system `, :ref:`got the source code `, and :ref:`built Python `. @@ -89,7 +92,7 @@ You should have already :ref:`set up your system `, make patchcheck ./python -m test - (Learn more about :ref:`patchcheck` and about :doc:`runtests`) + (Learn more about :ref:`patchcheck` and about :ref:`runtests`) * Once you are satisfied with the changes, add the files and commit them:: @@ -143,7 +146,7 @@ You should have already :ref:`set up your system `, .. _resolving-merge-conflicts: Resolving Merge Conflicts -''''''''''''''''''''''''' +------------------------- When merging changes from different branches (or variants of a branch on different repos), the two branches may contain incompatible changes to one @@ -172,7 +175,7 @@ for a detailed technical explanation. .. _good-prs: Making Good PRs ---------------- +=============== When creating a pull request for submission, there are several things that you should do to help ensure that your pull request is accepted. @@ -216,7 +219,7 @@ additions/changes should be included. .. _patchcheck: ``patchcheck`` --------------- +============== ``patchcheck`` is a simple automated patch checklist that guides a developer through the common patch generation checks. To run ``patchcheck``: @@ -256,7 +259,7 @@ making a complete patch. .. _good-commits: Making Good Commits -------------------- +=================== Each feature or bugfix should be addressed by a single pull request, and for each pull request there may be several commits. In particular: @@ -296,7 +299,7 @@ request. .. _cla: Licensing ---------- +========= To accept your change we must have your formal approval for distributing your work under the `PSF license`_. Therefore, you need to sign a @@ -327,7 +330,7 @@ Here are the steps needed in order to sign the CLA: Submitting ----------- +========== Once you are satisfied with your work you will want to commit your changes to your branch. In general you can run ``git commit -a`` and @@ -368,7 +371,7 @@ The commits will be squashed when the pull request is merged. Converting an Existing Patch from b.p.o to GitHub -------------------------------------------------- +================================================= When a patch exists in the `issue tracker`_ that should be converted into a GitHub pull request, please first ask the original patch author to prepare @@ -384,7 +387,7 @@ on how to properly add the co-author info. See also :ref:`Applying a Patch to Git `. Reviewing ---------- +========= To begin with, please be patient! There are many more people submitting pull requests than there are people capable of reviewing @@ -409,7 +412,7 @@ thus iterate until a satisfactory solution has emerged. How to Review a Pull Request -'''''''''''''''''''''''''''' +---------------------------- One of the bottlenecks in the Python development process is the lack of code reviews. @@ -450,7 +453,7 @@ code and leave comments in the pull request or issue tracker. 'merge-ready', you should always make sure the entire test suite passes. Leaving a Pull Request Review on GitHub ---------------------------------------- +======================================= When you review a pull request, you should provide additional details and context of your review process. @@ -466,7 +469,7 @@ Instead of simply "approving" the pull request, leave comments. For example: so will make it easier for the PR author to find the good in your comments. Dismissing Review from Another Core Developer ---------------------------------------------- +============================================= A core developer can dismiss another core developer's review if they confirmed that the requested changes have been made. When a core developer has assigned @@ -475,7 +478,7 @@ the PR, and their review should not be dismissed. Committing/Rejecting --------------------- +==================== Once your pull request has reached an acceptable state (and thus considered "accepted"), it will either be merged or rejected. If it is rejected, please @@ -491,7 +494,7 @@ it is warranted. Crediting ---------- +========= Non-trivial contributions are credited in the ``Misc/ACKS`` file (and, most often, in a contribution's news entry as well). You may be diff --git a/setup.rst b/getting-started/setup-building.rst similarity index 99% rename from setup.rst rename to getting-started/setup-building.rst index 4c3d2e52ba..ba1707a08c 100644 --- a/setup.rst +++ b/getting-started/setup-building.rst @@ -1,8 +1,9 @@ +.. _setup-building: .. _setup: -=============== -Getting Started -=============== +================== +Setup and Building +================== .. highlight:: console diff --git a/gh-labels.rst b/gh-labels.rst deleted file mode 100644 index 417c766bbf..0000000000 --- a/gh-labels.rst +++ /dev/null @@ -1,170 +0,0 @@ -.. _github-labels: - -GitHub Labels -============= - -We're using labels on GitHub to categorize issues and pull requests. -Many labels are shared for both use cases, while some are dedicated -only to one. Below is a possibly inexhaustive list, but it should get -you going. For a full list, see `here `_. - -General purpose labels ----------------------- - -type-behavior - Used for issues/PRs that address unintentional behavior, but do not - pose significant security concerns. Generally, bugfixes will be attached - to a specific issue where the unintended behavior was first reported. - -type-documentation - Used for issues/PRs that exclusively involve changes to - the documentation. Documentation includes ``*.rst`` files, docstrings, - and code comments. - -type-enhancement - Used for issues/PRs that provide additional functionality - or capabilities beyond the existing specifications. - -type-performance - Used for issues/PRs that provide performance optimizations. - -type-security - Used for issues/PRs that involve critical security issues. Less severe - security concerns can instead use the type-bugfix label. - -type-tests - Used for issues/PRs that exclusively involve changes to the tests. - -OS-Mac / OS-Windows - Used for issues/PRs involving changes which only have an effect upon - a specific operating system. - -spam - Used for issues/PRs that don't include enough eggs or bacon. - -Labels specific to issues -------------------------- - -Priority -^^^^^^^^ - -release-blocker - The highest priority of an issue. If unaddressed, will cause the - release manager to hold releasing a new version of Python. - -deferred-blocker - A release blocker that was pushed one or more releases into the - future. Possibly a temporary workaround was employed, or the version - of Python the issue is affecting is still in alpha or beta stages - of development. - -Component -^^^^^^^^^ - -library - Used for issues involving Python modules in the ``Lib/`` dir. - -docs - Used for issues involving documentation in the ``Doc/`` dir. - -interpreter-core - Used for issues in interpreter core (``Objects/``, ``Python/``, - ``Grammar/``, and ``Parser/`` dirs). - -extension-modules - Used for issues involving C modules in the ``Modules/`` dir. - -tests - Used for issues involving only Python's regression test suite, i.e. - files in the ``Lib/test/`` dir. - -Other -^^^^^ - -new - Denotes that the issue hasn't been looked at by triagers or core - developers yet. - -easy - Denotes that the issue is a good candidate for a newcomer to address. - - -Labels specific to PRs ----------------------- - -DO-NOT-MERGE - Used on PRs to prevent miss-islington from being able - to automatically merge the pull request. This label is appropriate when a PR - has a non-trivial conflict with the branch it is being merged into. - -expert-asyncio - Used for PRs which involve changes to the asyncio module - or other asynchronous frameworks that utilize it. - -invalid - Used manually for PRs that do not meet basic requirements and - automatically added by bedevere when PR authors attempt to merge maintenance - branches into the main branch. During events such as the October - Hacktoberfest, this label will prevent the PR from counting toward the - author's contributions. - -needs backport to X.Y - Used for PRs which are appropriate to backport to - branches prior to main. Generally, backports to the maintenance branches - are primarily bugfixes and documentation clarifications. Backports to the - security branches are strictly reserved for PRs involving security fixes, such as - crashes, privilege escalation, and DoS. The use of this label will cause - miss-islington to attempt to automatically merge the PR into the branches - specified. - -skip issue - Used for PRs which involve trivial changes, such as typo fixes, - comment changes, and section rephrases. The majority of PRs require - an issue to be attached to, but if there are no code changes and the - section being modified retains the same meaning, this label might be - appropriate. - -skip news - Similar to the skip issue label, this label is used for PRs which - involve trivial changes, backports, or already have a relevant news entry - in another PR. Any potentially impactful changes should have a - corresponding news entry, but for trivial changes it's commonly at the - discretion of the PR author if they wish to opt-out of making one. - -sprint - Used for PRs authored during an in-person sprint, such as - at PyCon, EuroPython, or other official Python events. The label is - used to prioritize the review of those PRs during the sprint. - -stale - Used for PRs that include changes which are no longer relevant, or when the - author hasn't responded to feedback in a long period of time, or when the - reviewer is unresponsive. This label helps core developers quickly identify - PRs that are candidates for closure or require a ping to the author or - reviewer. - -awaiting review - Used for PRs that haven't been reviewed by anyone yet. - -awaiting core review - Used when the PR is authored by a core developer or when a non-core - developer has reviewed the PR, even if they requested changes. - Note that reviewers could have been added manually by a triager or core - developer, or included automatically through use of the `CODEOWNERS - `_ - file. - -awaiting changes - A reviewer required changes to proceed with the PR. - -awaiting change review - The PR author made requested changes, and they are waiting for review. - -awaiting merge - The PR has been approved by a core developer and is ready to merge. - -test-with-buildbots - Used on PRs to test the latest commit with the buildbot fleet. Generally for - PRs with large code changes requiring more testing before merging. This - may take multiple hours to complete. Triagers can also stop a stuck build - using the web interface. diff --git a/index.rst b/index.rst index 0ffb6477ab..441c55d605 100644 --- a/index.rst +++ b/index.rst @@ -6,7 +6,7 @@ Python Developer's Guide This guide is a comprehensive resource for :ref:`contributing ` to Python_ -- for both new and experienced contributors. It is -:ref:`maintained ` by the same +:ref:`maintained ` by the same community that maintains Python. We welcome your contributions to Python! .. _quick-reference: @@ -42,7 +42,7 @@ instructions please see the :ref:`setup guide `. and the platform-specific pages for :ref:`UNIX `, :ref:`Mac OS `, and :ref:`Windows `. -4. :doc:`Run the tests `:: +4. :ref:`Run the tests `:: ./python -m test -j3 @@ -60,7 +60,7 @@ instructions please see the :ref:`setup guide `. 6. Once you fixed the issue, run the tests, run ``make patchcheck``, and if everything is ok, commit. -7. Push the branch on your fork on GitHub and :doc:`create a pull request +7. Push the branch on your fork on GitHub and :ref:`create a pull request `. Include the issue number using ``gh-NNNN`` in the pull request description. For example:: @@ -85,55 +85,14 @@ contributing to Python: * `Issue tracker`_ * `Buildbot status`_ -* :doc:`help` +* :ref:`help` * PEPs_ (Python Enhancement Proposals) -* :doc:`gitbootcamp` - -.. _branchstatus: +* :ref:`gitbootcamp` Status of Python branches ------------------------- -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| Branch | Schedule | Status | First release | End-of-life | Release manager | -+==================+==============+=============+================+================+=======================+ -| main | TBA | features | *2023-10-03* | *2028-10* | Thomas Wouters | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.11 | :pep:`664` | bugfix | *2022-10-03* | *2027-10* | Pablo Galindo Salgado | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.10 | :pep:`619` | bugfix | 2021-10-04 | *2026-10* | Pablo Galindo Salgado | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.9 | :pep:`596` | security | 2020-10-05 | *2025-10* | Łukasz Langa | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.8 | :pep:`569` | security | 2019-10-14 | *2024-10* | Łukasz Langa | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.7 | :pep:`537` | security | 2018-06-27 | *2023-06-27* | Ned Deily | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ - -.. Remember to update the end-of-life table in devcycle.rst. - -Dates in *italic* are scheduled and can be adjusted. - -The main branch is currently the future Python 3.12, and is the only -branch that accepts new features. The latest release for each Python -version can be found on the `download page `_. - -Status: - -:features: new features, bugfixes, and security fixes are accepted. -:prerelease: feature fixes, bugfixes, and security fixes are accepted for the - upcoming feature release. -:bugfix: bugfixes and security fixes are accepted, new binaries are still - released. (Also called **maintenance** mode or **stable** release) -:security: only security fixes are accepted and no more binaries are released, - but new source-only versions can be released -:end-of-life: release cycle is frozen; no further changes can be pushed to it. - -See also the :ref:`devcycle` page for more information about branches. - -By default, the end-of-life is scheduled 5 years after the first release, -but can be adjusted by the release manager of each branch. All Python 2 -versions have reached end-of-life. +Moved to :ref:`versions` .. _contributing: @@ -156,36 +115,20 @@ Core developers and contributors alike will find the following guides useful: Guide for contributing to Python: -+------------------------+---------------------+-----------------------+---------------------+ -| New Contributors | Documentarians | Triagers | Core Developers | -+========================+=====================+=======================+=====================+ -| :doc:`setup` | :doc:`docquality` | :doc:`tracker` | :doc:`coredev` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`help` | :doc:`documenting` | :doc:`triaging` | :doc:`developers` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`pullrequest` | :ref:`style-guide` | :ref:`helptriage` | :doc:`committing` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`runtests` | :ref:`rst-primer` | :doc:`experts` | :doc:`devcycle` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`fixingissues` | :ref:`translating` | | :doc:`motivations` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`communication` | | | :ref:`office hour` | -+------------------------+---------------------+-----------------------+---------------------+ -| :doc:`gitbootcamp` | | | | -+------------------------+---------------------+-----------------------+---------------------+ - -Advanced tasks and topics for once you are comfortable: - -* :doc:`silencewarnings` -* Fixing issues found by the :doc:`buildbots ` -* :doc:`coverity` -* Helping out with reviewing `open pull requests`_. - See :ref:`how to review a Pull Request `. -* :doc:`fixingissues` - -It is **recommended** that the above documents be read as needed. New -contributors will build understanding of the CPython workflow by reading the -sections mentioned in this table. You +======================== =================== ======================= ======================= +Contributors Documentarians Triagers Core Developers +======================== =================== ======================= ======================= +:ref:`setup` :ref:`docquality` :ref:`tracker` :ref:`responsibilities` +:ref:`help` :ref:`documenting` :ref:`triaging` :ref:`developers` +:ref:`pullrequest` :ref:`style-guide` :ref:`helptriage` :ref:`committing` +:ref:`runtests` :ref:`rst-primer` :ref:`experts` :ref:`devcycle` +:ref:`fixingissues` :ref:`translating` :ref:`labels` :ref:`motivations` +:ref:`communication` :ref:`gh-faq` :ref:`office hour` +:ref:`gitbootcamp` :ref:`triage-team` :ref:`experts` +:ref:`devcycle` +======================== =================== ======================= ======================= + +We **recommend** that the documents in this guide be read as needed. You can stop where you feel comfortable and begin contributing immediately without reading and understanding these documents all at once. If you do choose to skip around within the documentation, be aware that it is written assuming preceding @@ -206,8 +149,8 @@ developer's toolkit. While these kinds of change are much rarer than those described above, they do happen and that process is also described as part of this guide: -* :doc:`stdlibchanges` -* :doc:`langchanges` +* :ref:`stdlibchanges` +* :ref:`langchanges` Other Interpreter Implementations @@ -242,15 +185,15 @@ Key Resources * `Issue tracker`_ * `Meta tracker `_ (issue tracker for the issue tracker) - * :doc:`experts` + * :ref:`experts` * `Buildbot status`_ * Source code * `Browse online `_ * `Snapshot of the *main* branch `_ * `Daily OS X installer `_ * PEPs_ (Python Enhancement Proposals) -* :doc:`help` -* :doc:`developers` +* :ref:`help` +* :ref:`developers` .. _resources: @@ -259,16 +202,16 @@ Additional Resources -------------------- * Anyone can clone the sources for this guide. See - :ref:`helping-with-the-developers-guide`. + :ref:`devguide`. * Help with ... - * :doc:`exploring` - * :doc:`grammar` - * :doc:`parser` - * :doc:`compiler` - * :doc:`garbage_collector` + * :ref:`exploring` + * :ref:`grammar` + * :ref:`parser` + * :ref:`compiler` + * :ref:`garbage_collector` * Tool support - * :doc:`gdb` - * :doc:`clang` + * :ref:`gdb` + * :ref:`clang` * Various tools with configuration files as found in the `Misc directory`_ * Information about editors and their configurations can be found in the `wiki `_ @@ -297,41 +240,15 @@ Full Table of Contents .. toctree:: :maxdepth: 3 - setup - help - pullrequest - runtests - coverage - docquality - documenting - silencewarnings - fixingissues - tracker - triaging - communication - porting - coredev - developers - committing - devcycle - buildbots - stdlibchanges - langchanges - experts - gdb - exploring - grammar - parser - compiler - garbage_collector - extensions - c-api - coverity - clang - buildworker - motivations - gitbootcamp - appendix + getting-started/index + developer-workflow/index + triage/index + documentation/index + testing/index + core-developers/index + internals/index + advanced-tools/index + versions .. _Buildbot status: https://www.python.org/dev/buildbot/ .. _Misc directory: https://github.com/python/cpython/tree/main/Misc diff --git a/compiler.rst b/internals/compiler.rst similarity index 98% rename from compiler.rst rename to internals/compiler.rst index 2b3e4dd2ca..0e31e1be47 100644 --- a/compiler.rst +++ b/internals/compiler.rst @@ -1,12 +1,13 @@ .. _compiler: -Design of CPython's Compiler -============================ +=============== +Compiler Design +=============== .. highlight:: none Abstract --------- +======== In CPython, the compilation from source code to bytecode involves several steps: @@ -24,7 +25,7 @@ to read some source to have an exact understanding of all details. Parsing -------- +======= As of Python 3.9, Python's parser is a PEG parser of a somewhat unusual design (since its input is a stream of tokens rather than a @@ -34,11 +35,11 @@ The grammar file for Python can be found in :file:`Grammar/python.gram`. The definitions for literal tokens (such as ``:``, numbers, etc.) can be found in :file:`Grammar/Tokens`. Various C files, including :file:`Parser/parser.c` are generated from -these (see :doc:`grammar`). +these (see :ref:`grammar`). Abstract Syntax Trees (AST) ---------------------------- +=========================== .. _compiler-ast-trees: @@ -127,7 +128,7 @@ initializes the *name*, *args*, *body*, and *attributes* fields. Memory Management ------------------ +================= Before discussing the actual implementation of the compiler, a discussion of how memory is handled is in order. To make memory management simple, an arena @@ -161,7 +162,7 @@ the arena about it by calling ``PyArena_AddPyObject()``. Source Code to AST ------------------- +================== The AST is generated from source code using the function ``_PyParser_ASTFromString()`` or ``_PyParser_ASTFromFile()`` @@ -287,7 +288,7 @@ number is passed as the last parameter to each ``stmt_ty`` function. Control Flow Graphs -------------------- +=================== A *control flow graph* (often referenced by its acronym, CFG) is a directed graph that models the flow of a program. A node of a CFG is @@ -333,7 +334,7 @@ following the edges. AST to CFG to Bytecode ----------------------- +====================== With the AST created, the next step is to create the CFG. The first step is to convert the AST to Python bytecode without having jump targets @@ -449,7 +450,7 @@ handled by calling ``assemble()``. Introducing New Bytecode ------------------------- +======================== Sometimes a new feature requires a new opcode. But adding new bytecode is not as simple as just suddenly introducing new bytecode in the AST -> @@ -507,7 +508,7 @@ for recompiling generated C files. Code Objects ------------- +============ The result of ``PyAST_CompileObject()`` is a ``PyCodeObject`` which is defined in :file:`Include/code.h`. And with that you now have executable Python bytecode! @@ -518,7 +519,7 @@ statement in ``_PyEval_EvalFrameDefault()``. Important Files ---------------- +=============== + Parser/ @@ -655,7 +656,7 @@ Important Files Known Compiler-related Experiments ----------------------------------- +================================== This section lists known experiments involving the compiler (including bytecode). @@ -675,7 +676,7 @@ thanks to having to support both classic and new-style classes. References ----------- +========== .. [Wang97] Daniel C. Wang, Andrew W. Appel, Jeff L. Korn, and Chris S. Serra. `The Zephyr Abstract Syntax Description Language.`_ diff --git a/exploring.rst b/internals/exploring.rst similarity index 97% rename from exploring.rst rename to internals/exploring.rst index 7d5f908a2f..f27c150c0e 100644 --- a/exploring.rst +++ b/internals/exploring.rst @@ -1,7 +1,8 @@ .. _exploring: -Exploring CPython's Internals -============================= +======================== +Exploring the Internals +======================== This is a quick guide for people who are interested in learning more about CPython's internals. It provides a summary of the source code structure @@ -9,7 +10,7 @@ and contains references to resources providing a more in-depth view. CPython Source Code Layout --------------------------- +========================== This guide gives an overview of CPython's code structure. It serves as a summary of file locations for modules and builtins. @@ -49,7 +50,7 @@ Some exceptions: Additional References ---------------------- +===================== For over 20 years the CPython code base has been changing and evolving. Here's a sample of resources about the architecture of CPython aimed at diff --git a/garbage_collector.rst b/internals/garbage-collector.rst similarity index 97% rename from garbage_collector.rst rename to internals/garbage-collector.rst index 72a054d0b8..e79a746837 100644 --- a/garbage_collector.rst +++ b/internals/garbage-collector.rst @@ -1,14 +1,17 @@ +.. _garbage-collector: .. _gc: +.. _garbage_collector: -Design of CPython's Garbage Collector -===================================== +======================== +Garbage Collector Design +======================== :Author: Pablo Galindo Salgado .. highlight:: none Abstract --------- +======== The main garbage collection algorithm used by CPython is reference counting. The basic idea is that CPython counts how many different places there are that have a reference to an @@ -53,7 +56,7 @@ unreachable. This is the cyclic garbage collector, usually called just Garbage Collector (GC), even though reference counting is also a form of garbage collection. Memory layout and object structure ----------------------------------- +================================== Normally the C structure supporting a regular Python object looks as follows: @@ -118,7 +121,7 @@ the type is immutable, a ``tp_clear`` implementation must also be provided. Identifying reference cycles ----------------------------------------------- +============================ The algorithm that CPython uses to detect those reference cycles is implemented in the ``gc`` module. The garbage collector **only focuses** @@ -181,7 +184,7 @@ of that object when the algorithm starts. This is because the algorithm needs to modify the reference count to do the computations and in this way the interpreter will not modify the real reference count field. -.. figure:: images/python-cyclic-gc-1-new-page.png +.. figure:: /_static/python-cyclic-gc-1-new-page.png The GC then iterates over all containers in the first list and decrements by one the ``gc_ref`` field of any other object that container is referencing. Doing @@ -190,7 +193,7 @@ using the C API or inherited by a superclass) to know what objects are reference each container. After all the objects have been scanned, only the objects that have references from outside the “objects to scan” list will have ``gc_refs > 0``. -.. figure:: images/python-cyclic-gc-2-new-page.png +.. figure:: /_static/python-cyclic-gc-2-new-page.png Notice that having ``gc_refs == 0`` does not imply that the object is unreachable. This is because another object that is reachable from the outside (``gc_refs > 0``) @@ -204,13 +207,13 @@ tentatively unreachable list. The following image depicts the state of the lists moment when the GC processed the ``link_3`` and ``link_4`` objects but has not processed ``link_1`` and ``link_2`` yet. -.. figure:: images/python-cyclic-gc-3-new-page.png +.. figure:: /_static/python-cyclic-gc-3-new-page.png Then the GC scans the next ``link_1`` object. Because it has ``gc_refs == 1``, the gc does not do anything special because it knows it has to be reachable (and is already in what will become the reachable list): -.. figure:: images/python-cyclic-gc-4-new-page.png +.. figure:: /_static/python-cyclic-gc-4-new-page.png When the GC encounters an object which is reachable (``gc_refs > 0``), it traverses its references using the ``tp_traverse`` slot to find all the objects that are @@ -225,7 +228,7 @@ objects that have already been visited once (by unsetting the ``PREV_MASK_COLLEC flag) so that if an object that has already been processed is referenced by some other object, the GC does not process it twice. -.. figure:: images/python-cyclic-gc-5-new-page.png +.. figure:: /_static/python-cyclic-gc-5-new-page.png Notice that an object that was marked as "tentatively unreachable" and was later moved back to the reachable list will be visited again by the garbage collector @@ -241,7 +244,7 @@ number of objects, number of pointers, or the lengths of pointer chains. Apart the GC algorithms require. Why moving unreachable objects is better -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------- It sounds logical to move the unreachable objects under the premise that most objects are usually reachable, until you think about it: the reason it pays isn't actually @@ -266,7 +269,7 @@ unbounded number of moves across an unbounded number of later collections. The o time the cost can be higher is the first time the chain is scanned. Destroying unreachable objects ------------------------------- +============================== Once the GC knows the list of unreachable objects, a very delicate process starts with the objective of completely destroying these objects. Roughly, the process @@ -297,7 +300,7 @@ follows these steps in order: objects. Optimization: generations -------------------------- +========================= In order to limit the time each garbage collection takes, the GC uses a popular optimization: generations. The main idea behind this concept is the assumption that @@ -368,7 +371,7 @@ specifically in a generation by calling ``gc.collect(generation=NUM)``. Collecting the oldest generation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +-------------------------------- In addition to the various configurable thresholds, the GC only triggers a full collection of the oldest generation if the ratio ``long_lived_pending / long_lived_total`` @@ -387,7 +390,7 @@ in the total number of objects (the effect of which can be summarized thusly: grows, but we do fewer and fewer of them"). Optimization: reusing fields to save memory -------------------------------------------- +=========================================== In order to save memory, the two linked list pointers in every object with GC support are reused for several purposes. This is a common optimization known @@ -436,7 +439,7 @@ of ``PyGC_Head`` discussed in the `Memory layout and object structure`_ section: ``NEXT_MASK_UNREACHABLE`` flag) are employed. Optimization: delay tracking containers ---------------------------------------- +======================================= Certain types of containers cannot participate in a reference cycle, and so do not need to be tracked by the garbage collector. Untracking these objects diff --git a/internals/index.rst b/internals/index.rst new file mode 100644 index 0000000000..ce04c665c1 --- /dev/null +++ b/internals/index.rst @@ -0,0 +1,11 @@ +=================== +CPython's Internals +=================== + +.. toctree:: + :maxdepth: 5 + + exploring + parser + compiler + garbage-collector diff --git a/parser.rst b/internals/parser.rst similarity index 98% rename from parser.rst rename to internals/parser.rst index c52590bc31..951b62198b 100644 --- a/parser.rst +++ b/internals/parser.rst @@ -1,14 +1,15 @@ .. _parser: -Guide to CPython's Parser -========================= +=================== +Guide to the Parser +=================== :Author: Pablo Galindo Salgado .. highlight:: none Abstract --------- +======== The Parser in CPython is currently a `PEG (Parser Expression Grammar) `_ parser. The first @@ -24,7 +25,7 @@ modifying the grammar file and developers rarely need to interact with the parser generator itself other than use it to generate the parser. How PEG Parsers Work --------------------- +==================== .. _how-peg-parsers-work: @@ -73,7 +74,7 @@ table-based. Key ideas -~~~~~~~~~ +--------- .. important:: Don't try to reason about a PEG grammar in the same way you would to with an EBNF @@ -92,7 +93,7 @@ Key ideas Consequences or the ordered choice operator -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------------- .. _consequences-of-ordered-choice: @@ -143,7 +144,7 @@ In this case, if the input string doesn't have an ``'else' block``, the first al will fail and the second will be attempted without said part. Syntax ------- +====== The grammar consists of a sequence of rules of the form: :: @@ -159,7 +160,7 @@ If the return type is omitted, then a ``void *`` is returned in C and an ``Any`` in Python. Grammar Expressions -~~~~~~~~~~~~~~~~~~~ +------------------- ``# comment`` ''''''''''''' @@ -288,7 +289,7 @@ alternative won’t be considered, even if some_rule or ``)`` fail to be parsed. Left recursion -~~~~~~~~~~~~~~ +-------------- PEG parsers normally do not support left recursion but CPython's parser generator implements a technique similar to the one described in Medeiros et al. @@ -306,7 +307,7 @@ and "hidden left-recursion" like:: rule: 'optional'? rule '@' some_other_rule Variables in the Grammar -~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------ A sub-expression can be named by preceding it with an identifier and an ``=`` sign. The name can then be used in the action (see below), like this: :: @@ -314,7 +315,7 @@ A sub-expression can be named by preceding it with an identifier and an rule_name[return_type]: '(' a=some_other_rule ')' { a } Grammar actions -~~~~~~~~~~~~~~~ +--------------- .. _peg-grammar-actions: @@ -514,7 +515,7 @@ A similar grammar written to target Python AST objects: Pegen ------ +===== Pegen is the parser generator used in CPython to produce the final PEG parser used by the interpreter. It is the program that can be used to read the python grammar located in :file:`Grammar/Python.gram` and produce the final C @@ -531,7 +532,7 @@ The source code for Pegen lives at :file:`Tools/peg_generator/pegen` but normall with the parser generator are executed from the main makefile. How to regenerate the parser -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------- Once you have made the changes to the grammar files, to regenerate the ``C`` parser (the one used by the interpreter) just execute: :: @@ -546,7 +547,7 @@ use the Visual Studio project files to regenerate the parser or to execute: :: The generated parser file is located at :file:`Parser/parser.c`. How to regenerate the meta-parser -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +--------------------------------- The meta-grammar (the grammar that describes the grammar for the grammar files themselves) is located at :file:`Tools/peg_generator/pegen/metagrammar.gram`. @@ -563,7 +564,7 @@ to regenerate the parser or to execute: :: Grammatical elements and rules -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------ Pegen has some special grammatical elements and rules: @@ -582,7 +583,7 @@ Pegen has some special grammatical elements and rules: (like any rule in PEG). Tokenization -~~~~~~~~~~~~ +------------ It is common among PEG parser frameworks that the parser does both the parsing and the tokenization, but this does not happen in Pegen. The reason is that the Python language needs a custom tokenizer @@ -604,7 +605,7 @@ How tokens are generated and the rules governing this is completely up to the to and the parser just receives tokens from it. Memoization -~~~~~~~~~~~ +----------- As described previously, to avoid exponential time complexity in the parser, memoization is used. @@ -630,7 +631,7 @@ memoization (check the :file:`Parser/pegen.c` file for more information) but it needs to be manually activated. Automatic variables -~~~~~~~~~~~~~~~~~~~ +------------------- To make writing actions easier, Pegen injects some automatic variables in the namespace available when writing actions. In the C parser, some of these automatic variable names are: @@ -641,7 +642,7 @@ when writing actions. In the C parser, some of these automatic variable names ar location variables are taken from the location information of the current token. Hard and Soft keywords -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- .. note:: In the grammar files, keywords are defined using **single quotes** (e.g. `'class'`) while soft @@ -705,7 +706,7 @@ as well as soft keywords: try to define them in places where there is not a lot of alternatives. Error handling -~~~~~~~~~~~~~~ +-------------- When a pegen-generated parser detects that an exception is raised, it will **automatically stop parsing**, no matter what the current state of the parser @@ -722,7 +723,7 @@ error messages. parser finishes without returning anything. How Syntax errors are reported -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------ As described previously in the :ref:`how PEG parsers work section `, PEG parsers don't have a defined concept of where @@ -799,7 +800,7 @@ displayed when the error is reported. will be reported there instead of the ``$`` character. Generating AST objects -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- The output of the C parser used by CPython that is generated by the :file:`Grammar/Python.gram` grammar file is a Python AST object (using C @@ -828,7 +829,7 @@ custom helper in :file:`Parser/action_helpers.c` and expose it in the If the parsing succeeds, the parser **must** return a **valid** AST object. Testing -------- +======= There are three files that contain tests for the grammar and the parser: @@ -843,10 +844,10 @@ Tests for the parser generator itself can be found in the :file:`Lib/test/test_p Debugging generated parsers ---------------------------- +=========================== Making experiments -~~~~~~~~~~~~~~~~~~ +------------------ As the generated C parser is the one used by Python, this means that if something goes wrong when adding some new rules to the grammar you cannot correctly compile and execute Python anymore. This makes it a bit challenging @@ -870,7 +871,7 @@ better understand some complex situations. Verbose mode -~~~~~~~~~~~~ +------------ When Python is compiled in debug mode (by adding ``--with-pydebug`` when running the configure step in Linux or by adding ``-d`` when calling the :file:`PCbuild/build.bat` script in Windows), it is possible to activate a **very** verbose @@ -907,7 +908,7 @@ indicates what alternative within that rule is being attempted. References ----------- +========== .. [1] Ford, Bryan https://pdos.csail.mit.edu/~baford/packrat/thesis/ diff --git a/requirements.txt b/requirements.txt index c5f1246b3e..a30e3d07c0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,3 +2,4 @@ Sphinx==5.0.2 furo>=2022.6.4 sphinx_copybutton>=0.3.3 sphinx-lint<1 +sphinxext-rediraffe diff --git a/buildbots.rst b/testing/buildbots.rst similarity index 96% rename from buildbots.rst rename to testing/buildbots.rst index 979d4632ea..e9ccbcab0b 100644 --- a/buildbots.rst +++ b/testing/buildbots.rst @@ -1,11 +1,12 @@ .. _buildbots: -Continuous Integration +====================== +Working with Buildbots ====================== .. highlight:: bash -To assert that there are no regressions in the :doc:`development and maintenance +To assert that there are no regressions in the :ref:`development and maintenance branches `, Python has a set of dedicated machines (called *buildbots* or *build workers*) used for continuous integration. They span a number of hardware/operating system combinations. Furthermore, each machine hosts @@ -26,7 +27,7 @@ important that you get acquainted with the way these results are presented, and how various kinds of failures can be explained and diagnosed. In case of trouble ------------------- +================== Please read this page in full. If your questions aren't answered here and you need assistance with the buildbots, a good way to get help is to either: @@ -36,7 +37,7 @@ need assistance with the buildbots, a good way to get help is to either: * contact the release manager of the branch you have issues with. Buildbot failures on Pull Requests ----------------------------------- +================================== The ``bedevere-bot`` on GitHub will put a message on your merged Pull Request if building your commit on a stable buildbot worker fails. Take care to @@ -48,7 +49,7 @@ complete so they are done periodically. This is why it's important for you to be able to check the results yourself, too. Checking results of automatic builds ------------------------------------- +==================================== There are three ways of visualizing recent build results: @@ -89,7 +90,7 @@ branch, it often happens that a single build is scheduled for all these changesets. Stability ---------- +========= A subset of the buildbots are marked "stable". They are taken into account when making a new release. The rule is that all stable builders must be free of @@ -103,9 +104,9 @@ prevent some tests from succeeding (or even terminating at all), but introducing additional failures should generally not be an option. Flags-dependent failures ------------------------- +======================== -Sometimes, while you have run the :doc:`whole test suite ` before +Sometimes, while you have run the :ref:`whole test suite ` before committing, you may witness unexpected failures on the buildbots. One source of such discrepancies is if different flags have been passed to the test runner or to Python itself. To reproduce, make sure you use the same flags as the @@ -119,7 +120,7 @@ the failing build's tests. For example:: ``-m test``. Ordering-dependent failures ---------------------------- +=========================== Sometimes the failure is even subtler, as it relies on the order in which the tests are run. The buildbots *randomize* test order (by using the ``-r`` @@ -200,7 +201,7 @@ good luck! Transient failures ------------------- +================== While we try to make the test suite as reliable as possible, some tests do not reach a perfect level of reproducibility. Some of them will sometimes @@ -224,7 +225,7 @@ implementation, or by making its parameters - such as a timeout - more robust. Custom builders ---------------- +=============== .. highlight:: console diff --git a/coverage.rst b/testing/coverage.rst similarity index 98% rename from coverage.rst rename to testing/coverage.rst index d2e4e7f156..2573716eb9 100644 --- a/coverage.rst +++ b/testing/coverage.rst @@ -1,5 +1,6 @@ .. _coverage: +====================== Increase Test Coverage ====================== @@ -36,7 +37,7 @@ implicit testing by other code that happens to use the module. Common Gotchas -"""""""""""""" +============== Please realize that coverage reports on modules already imported before coverage data starts to be recorded will be wrong. Typically you can tell a module falls @@ -54,7 +55,7 @@ stick with whitebox testing in order to properly exercise the code. Measuring Coverage -"""""""""""""""""" +================== It should be noted that a quirk of running coverage over Python's own stdlib is that certain modules are imported as part of interpreter startup. Those modules @@ -256,15 +257,15 @@ times. Filing the Issue -"""""""""""""""" +================ Once you have increased coverage, you need to create an issue on the -`issue tracker`_ and submit a :doc:`pull request `. On the +`issue tracker`_ and submit a :ref:`pull request `. On the issue set the "Components" to "Test" and "Versions" to the version of Python you worked on (i.e., the in-development version). Measuring coverage of C code with gcov and lcov -""""""""""""""""""""""""""""""""""""""""""""""" +=============================================== It's also possible to measure the function, line and branch coverage of Python's C code. Right now only GCC with `gcov`_ is supported. In order to diff --git a/testing/index.rst b/testing/index.rst new file mode 100644 index 0000000000..4a7247c985 --- /dev/null +++ b/testing/index.rst @@ -0,0 +1,12 @@ +===================== +Testing and Buildbots +===================== + +.. toctree:: + :maxdepth: 5 + + run-write-tests + silence-warnings + coverage + buildbots + new-buildbot-worker diff --git a/buildworker.rst b/testing/new-buildbot-worker.rst similarity index 97% rename from buildworker.rst rename to testing/new-buildbot-worker.rst index e91d22f68c..53c186ea73 100644 --- a/buildworker.rst +++ b/testing/new-buildbot-worker.rst @@ -1,8 +1,9 @@ - +.. _new-buildbot-worker: .. _buildworker: -Running a buildbot worker -========================= +==================== +New Buildbot Workers +==================== .. highlight:: bash @@ -14,7 +15,7 @@ supported by corporations. Even the corporate sponsored buildbots, however, tend to exist because some individual championed them, made them a reality, and is committed to maintaining them. -Anyone can contribute a buildbot to the fleet. This chapter describes how +Anyone can contribute a buildbot to the fleet. This document describes how to go about setting up a buildbot worker, getting it added, and some hints about buildbot maintenance. @@ -35,7 +36,7 @@ contribute. Preparing for buildbot worker setup ------------------------------------ +=================================== Since the goal is to build Python from source, the system will need to have everything required to do normal python development: a compiler, a linker, and @@ -53,10 +54,10 @@ the "buildbot worker" step below. Setting up the buildbot worker ------------------------------- +============================== Conventional always-on machines -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------------- You need a recent version of the `buildbot `_ software, and you will probably want a separate 'buildbot' user to run the buildbot @@ -195,7 +196,7 @@ idea. Latent workers -^^^^^^^^^^^^^^ +-------------- We also support running `latent workers `_ @@ -242,7 +243,7 @@ buildmaster administrators with the new AMI ID. Buildbot worker operation -------------------------- +========================= Most of the time, running a worker is a "set and forget" operation, depending on the level of involvement you want to have in resolving bugs @@ -273,7 +274,7 @@ to resolve the issue. Required Ports --------------- +============== The worker operates as a *client* to the *buildmaster*. This means that all network connections are *outbound*. This is true also for the network @@ -303,7 +304,7 @@ using either ``localhost`` or ``127.0.0.1``. Required Resources ------------------- +================== Based on the last time we did a `survey `_ on @@ -321,7 +322,7 @@ suite. Security Considerations ------------------------ +======================= We only allow builds to be triggered against commits to the `CPython repository on GitHub `_. diff --git a/runtests.rst b/testing/run-write-tests.rst similarity index 97% rename from runtests.rst rename to testing/run-write-tests.rst index 20d102a4fd..fb72bb5694 100644 --- a/runtests.rst +++ b/testing/run-write-tests.rst @@ -1,7 +1,9 @@ +.. _run-write-tests: .. _runtests: -Running & Writing Tests -======================= +========================= +Running and Writing Tests +========================= .. note:: @@ -11,7 +13,7 @@ Running & Writing Tests on new features not available in earlier versions of Python. Running -------- +======= The shortest, simplest way of running the test suite is the following command from the root directory of your checkout (after you have @@ -96,7 +98,7 @@ above. Unexpected Skips -^^^^^^^^^^^^^^^^ +---------------- Sometimes when running the test suite, you will see "unexpected skips" reported. These represent cases where an entire test module has been @@ -114,7 +116,7 @@ settings on some platforms will disallow some tests) Writing -------- +======= Writing tests for Python is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as @@ -134,7 +136,8 @@ you have to take to make your tests robust and portable. Benchmarks ----------- +========== + Benchmarking is useful to test that a change does not degrade performance. `The Python Benchmark Suite `_ diff --git a/silencewarnings.rst b/testing/silence-warnings.rst similarity index 93% rename from silencewarnings.rst rename to testing/silence-warnings.rst index a5f21ed0d3..e32da9a9c3 100644 --- a/silencewarnings.rst +++ b/testing/silence-warnings.rst @@ -1,5 +1,7 @@ +.. _silence-warnings: .. _silencewarnings: +==================================== Silence Warnings From the Test Suite ==================================== diff --git a/gh-faq.rst b/triage/github-bpo-faq.rst similarity index 88% rename from gh-faq.rst rename to triage/github-bpo-faq.rst index 42b0fb61d4..d69d04fc9c 100644 --- a/gh-faq.rst +++ b/triage/github-bpo-faq.rst @@ -1,15 +1,19 @@ -GitHub issues for BPO users +.. _github-bpo-faq: +.. _gh-faq: + +=========================== +GitHub Issues for BPO Users =========================== Here are some frequently asked questions about how to do things in GitHub issues that you used to be able to do on `bpo`_. -Before you ask your own question, make sure you read :doc:`tracker` -and :doc:`triaging` (specifically including :doc:`gh-labels`) as those +Before you ask your own question, make sure you read :ref:`tracker` +and :ref:`triaging` (specifically including :ref:`gh-labels`) as those pages include a lot of introductory material. How to format my comments nicely? ---------------------------------- +================================= There is a wonderful `beginner guide to writing and formatting on GitHub `_. @@ -27,13 +31,13 @@ If you still insist on pasting it in your comment, do it like this:: How to attach files to an issue? --------------------------------- +================================ Drag them into the comment field, wait until the file uploads, and GitHub will automatically put a link to your file in your comment text. How to link to file paths in the repository when writing comments? ------------------------------------------------------------------- +================================================================== Use Markdown links. If you link to the default GitHub path, the file will link to the latest current version on the given branch. @@ -42,13 +46,13 @@ You can get a permanent link to a given revision of a given file by `pressing "y" `_. How to do advanced searches? ----------------------------- +============================ Use the `GitHub search syntax`_ or the interactive `advanced search`_ form that generates search queries for you. Where is the "nosy list"? -------------------------- +========================= Subscribe another person to the issue by tagging them in the comment with ``@username``. @@ -65,7 +69,7 @@ this information during the transfer, we list the previous members of this list in the first message on the migrated issue. How to add issue dependencies? ------------------------------- +============================== Add a checkbox list like this in the issue description:: @@ -79,7 +83,7 @@ closed. More details in the `official GitHub documentation `_. What on Earth is a "mannequin"? -------------------------------- +=============================== For issues migrated to GitHub from `bpo`_ where the authors or commenters are not core developers, we opted not to link to their GitHub accounts @@ -94,22 +98,22 @@ name in their `bpo`_ profile, we use that. Otherwise, their classic `bpo`_ username is used instead. Where did the "Resolution" field go? ------------------------------------- +==================================== Based on historical data we found it not being used very often. Where did the "Low", "High", and "Critical" priorities go? ----------------------------------------------------------- +========================================================== Based on historical data we found those not being used very often. How to find a random issue? ---------------------------- +=========================== This is not supported by GitHub. Where are regression labels? ----------------------------- +============================ We rarely updated this information and it turned out not to be particularly useful outside of the change log. diff --git a/triage/index.rst b/triage/index.rst new file mode 100644 index 0000000000..b2ae1c5737 --- /dev/null +++ b/triage/index.rst @@ -0,0 +1,12 @@ +=================== +Issues and Triaging +=================== + +.. toctree:: + :maxdepth: 5 + + issue-tracker + triaging + labels + github-bpo-faq + triage-team diff --git a/tracker.rst b/triage/issue-tracker.rst similarity index 57% rename from tracker.rst rename to triage/issue-tracker.rst index 87f9d8acdf..6a0ae22d65 100644 --- a/tracker.rst +++ b/triage/issue-tracker.rst @@ -1,9 +1,11 @@ -============== -Issue Tracking -============== - +.. _issue-tracker: .. _tracker: +============= +Issue Tracker +============= + + Using the Issue Tracker ======================= @@ -22,8 +24,8 @@ If you would like to file an issue about this devguide, please do so at the migrated to the current `issue tracker`_ on GitHub. If you're familiar with ``bpo`` and would like to learn more about GitHub - issues, please read this page, and the :doc:`triaging` page as they - provide good introductory material. There is also a :doc:`gh-faq` + issues, please read this page, and the :ref:`triaging` page as they + provide good introductory material. There is also a :ref:`gh-faq` document to answer some of the more popular questions. Checking if a bug already exists @@ -113,110 +115,8 @@ As a reminder, issues closed by a core developer have already been carefully considered. Please do not reopen a closed issue. An issue can be closed with reason either as ``complete`` or ``not planned``. -.. _helptriage: - -Helping Triage Issues -===================== - -Once you know your way around how Python's source files are -structured and you are comfortable working with patches, a great way to -contribute is to help triage issues. Do realize, though, that experience -working on Python is needed in order to effectively help triage. - -Around the clock, new issues are being opened on the `issue tracker`_ and -existing issues are being updated. Every issue needs to be triaged to make -sure various things are in proper order. Even without special privileges you -can help with this process. - -Classifying Reports -------------------- - -For bugs, an issue needs to: - -* clearly explain the bug so it can be reproduced -* include all relevant platform details -* state what version(s) of Python are affected by the bug. - -These are things you can help with once you have experience developing for -Python: - -* try reproducing the bug: For instance, if a bug is not clearly explained - enough for you to reproduce it then there is a good chance a core developer - won't be able to either. -* see if the issue happens on a different Python version: It is always helpful - to know if a bug not only affects the in-development version of Python, but - whether it also affects other versions in maintenance mode. -* write a unit test: If the bug lacks a unit test that should end up in - Python's test suite, having that written can be very helpful. - -This is all helpful as it allows triagers (i.e., -:ref:`people with the Developer role on the issue tracker `) to -properly classify an issue so it can be handled by the right core developers in -a timely fashion. - -Reviewing Patches ------------------ - -If an issue has a pull request attached that has not been reviewed, you can help -by making sure the patch: - -* follows the style guides -* applies cleanly to an up-to-date clone -* is a good solution to the problem it is trying to solve -* includes proper tests -* includes proper documentation changes -* submitter is listed in ``Misc/ACKS``, either already or the patch adds them - -Doing all of this allows core developers and :ref:`triagers ` to more -quickly look for subtle issues that only people with extensive experience -working on Python's code base will notice. - -Finding an Issue You Can Help With ----------------------------------- - -If you want to help triage issues, you might also want to search for issues -in modules which you have a working knowledge. Search for the name of a module -in the issue tracker or use the `advanced search`_ query builder to search for -specific kinds of issues (e.g. the "Windows" label if you are a Windows -developer, "Extension Modules" if you are familiar with C, etc.). - - -.. _devrole: - -Gaining the "Triager" Role on the Issue Tracker -=============================================== - -When you have consistently shown the ability to properly -help triage issues without guidance, you may request that you -be given the "Triager" role on the `issue tracker`_. You can make the request -to any person who already has the Triager role. If they decide you are ready -to gain the extra privileges on the tracker they will then act as a mentor to -you until you are ready to do things entirely on your own. There is no set rule -as to how many issues you need to have helped with before or how long you have -been participating. The key requirements are that you show the desire to -help, you are able to work well with others (especially those already with the -Triager role), and that have a firm grasp of how to do things on the issue -tracker properly on your own. - -Gaining the Triager role will allow you to set any value on any issue in the -tracker, releasing you from the burden of having to ask others to set values on -an issue for you in order to properly triage something. This will not only help -speed up and simplify your work in helping out, but also help lessen the -workload for everyone by gaining your help. - -Sub-pages related to the Issue Tracker -====================================== - -.. toctree:: - :maxdepth: 1 - - gh-labels - gh-faq - .. seealso:: - | *Issues with Python and documentation* - `The Python issue tracker `_ Where to report issues about Python. diff --git a/triaging.rst b/triage/labels.rst similarity index 64% rename from triaging.rst rename to triage/labels.rst index 9428717a77..78a5ac9b52 100644 --- a/triaging.rst +++ b/triage/labels.rst @@ -1,102 +1,188 @@ -.. _triaging: - -Triaging an Issue -================= - -This section of the devguide documents the `issue tracker`_ for users -and developers. - -Contributors with the Triager role on the `issue tracker`_ can triage issues -directly without any assistance. - -Additionally, this section provides an overview of the Python triage team. - -Python triage team ------------------- - -The Python triage team is a group dedicated towards improving workflow -efficiency through thoughtful review and triage of open issues and pull -requests. This helps contributors receive timely feedback and enables core -developers to focus on reviewed items which reduces their workload. The -expectations of this role expand upon the "Triager" role on the -`issue tracker`_. The responsibilities listed below are primarily centered -around the Python GitHub repositories. This extends beyond CPython, and, as -needed, to other repos such as devguide and core-workflow. - -Responsibilities include: - -* PR/issue management - - Reviewing PRs - - Assisting contributors - - Notifying appropriate core developers -* Applying appropriate labels to PRs/Issues - - Skip news - - Skip issue - - Good first issue - - Other categorizations - -Although triagers have the power to close PRs, they should generally not do so -without first consulting a core developer. By having triagers and core developers work together, -the author receives a careful consideration of their PR. This encourages future -contributions, regardless of whether their PR is accepted or closed. - -Nonetheless, triagers should feel free to close a PR if they judge that the -chance of the PR being merged would be exceedingly low, even if substantial -revisions were made to the PR. This includes (but is not limited to) the -following: - -* PRs proposing solely cosmetic changes -* PRs proposing changes to deprecated modules -* PRs that are no longer relevant. This includes: - - PRs proposing fixes for bugs that can no longer be reproduced - - PRs proposing changes that have been rejected by Python core developers - elsewhere (e.g. in an issue or a PEP rejection notice) - -If a triager has any doubt about whether to close a PR, they should consult a core -developer before taking any action. - -Triagers can also make use of the ``invalid`` and ``stale`` labels to suggest that a -PR may be suitable for closure. For more information, see the -:ref:`GitHub PR labels ` section. - -Note that it is of paramount importance to treat every contributor to the Python -project kindly and with respect. Regardless of whether they're entirely new -or a veteran core developer, they're actively choosing to voluntarily donate their -time towards the improvement of Python. As is the case with any member of -the Python Software Foundation, always follow the `PSF Code of Conduct`_. - -Becoming a member of the Python triage team -------------------------------------------- - -Any Python core developers are welcome to invite a Python contributor to the -Python triage team. Triagers will be responsible to handle not just issues, but -also pull requests, and even managing backports. A Python triager has access to -more repositories than just CPython. - -Any existing active contributor to the Python repository on GitHub can -transition into becoming a Python triager. They can request this to any core -developer, and the core developer can pass the request to the `Python -organization admin -`_ -on GitHub. The request can be made confidentially via a DM in Discourse, or -publicly by opening an `issue in the core-workflow repository -`_. - -For every new triager, it would be great to announce them in the python-committers -mailing list and core-workflow category in Discourse. `Example announcement -`_. +.. _labels: +.. _gh-labels: + +============= +GitHub Labels +============= + +We're using labels on GitHub to categorize issues and pull requests. +Many labels are shared for both use cases, while some are dedicated +only to one. Below is a possibly inexhaustive list, but it should get +you going. For a full list, see `here `_. + +General purpose labels +====================== + +type-behavior + Used for issues/PRs that address unintentional behavior, but do not + pose significant security concerns. Generally, bugfixes will be attached + to a specific issue where the unintended behavior was first reported. + +type-documentation + Used for issues/PRs that exclusively involve changes to + the documentation. Documentation includes ``*.rst`` files, docstrings, + and code comments. + +type-enhancement + Used for issues/PRs that provide additional functionality + or capabilities beyond the existing specifications. + +type-performance + Used for issues/PRs that provide performance optimizations. + +type-security + Used for issues/PRs that involve critical security issues. Less severe + security concerns can instead use the type-bugfix label. + +type-tests + Used for issues/PRs that exclusively involve changes to the tests. + +OS-Mac / OS-Windows + Used for issues/PRs involving changes which only have an effect upon + a specific operating system. + +spam + Used for issues/PRs that don't include enough eggs or bacon. + +Labels specific to issues +========================= + +Priority +-------- + +release-blocker + The highest priority of an issue. If unaddressed, will cause the + release manager to hold releasing a new version of Python. + +deferred-blocker + A release blocker that was pushed one or more releases into the + future. Possibly a temporary workaround was employed, or the version + of Python the issue is affecting is still in alpha or beta stages + of development. + +Component +--------- + +library + Used for issues involving Python modules in the ``Lib/`` dir. + +docs + Used for issues involving documentation in the ``Doc/`` dir. + +interpreter-core + Used for issues in interpreter core (``Objects/``, ``Python/``, + ``Grammar/``, and ``Parser/`` dirs). + +extension-modules + Used for issues involving C modules in the ``Modules/`` dir. + +tests + Used for issues involving only Python's regression test suite, i.e. + files in the ``Lib/test/`` dir. + +Other +----- + +new + Denotes that the issue hasn't been looked at by triagers or core + developers yet. + +easy + Denotes that the issue is a good candidate for a newcomer to address. + + +Labels specific to PRs +====================== + +DO-NOT-MERGE + Used on PRs to prevent miss-islington from being able + to automatically merge the pull request. This label is appropriate when a PR + has a non-trivial conflict with the branch it is being merged into. + +expert-asyncio + Used for PRs which involve changes to the asyncio module + or other asynchronous frameworks that utilize it. + +invalid + Used manually for PRs that do not meet basic requirements and + automatically added by bedevere when PR authors attempt to merge maintenance + branches into the main branch. During events such as the October + Hacktoberfest, this label will prevent the PR from counting toward the + author's contributions. + +needs backport to X.Y + Used for PRs which are appropriate to backport to + branches prior to main. Generally, backports to the maintenance branches + are primarily bugfixes and documentation clarifications. Backports to the + security branches are strictly reserved for PRs involving security fixes, such as + crashes, privilege escalation, and DoS. The use of this label will cause + miss-islington to attempt to automatically merge the PR into the branches + specified. + +skip issue + Used for PRs which involve trivial changes, such as typo fixes, + comment changes, and section rephrases. The majority of PRs require + an issue to be attached to, but if there are no code changes and the + section being modified retains the same meaning, this label might be + appropriate. + +skip news + Similar to the skip issue label, this label is used for PRs which + involve trivial changes, backports, or already have a relevant news entry + in another PR. Any potentially impactful changes should have a + corresponding news entry, but for trivial changes it's commonly at the + discretion of the PR author if they wish to opt-out of making one. + +sprint + Used for PRs authored during an in-person sprint, such as + at PyCon, EuroPython, or other official Python events. The label is + used to prioritize the review of those PRs during the sprint. + +stale + Used for PRs that include changes which are no longer relevant, or when the + author hasn't responded to feedback in a long period of time, or when the + reviewer is unresponsive. This label helps core developers quickly identify + PRs that are candidates for closure or require a ping to the author or + reviewer. + +awaiting review + Used for PRs that haven't been reviewed by anyone yet. + +awaiting core review + Used when the PR is authored by a core developer or when a non-core + developer has reviewed the PR, even if they requested changes. + Note that reviewers could have been added manually by a triager or core + developer, or included automatically through use of the `CODEOWNERS + `_ + file. + +awaiting changes + A reviewer required changes to proceed with the PR. + +awaiting change review + The PR author made requested changes, and they are waiting for review. + +awaiting merge + The PR has been approved by a core developer and is ready to merge. + +test-with-buildbots + Used on PRs to test the latest commit with the buildbot fleet. Generally for + PRs with large code changes requiring more testing before merging. This + may take multiple hours to complete. Triagers can also stop a stuck build + using the web interface. + .. _github-pr-labels: GitHub Labels for PRs -''''''''''''''''''''' +===================== An important component of triaging PRs for the CPython repo involves appropriately categorizing them through the usage of labels. For this -purpose we're using :doc:`gh-labels`. +purpose we're using :ref:`gh-labels`. Applying labels for Issues --------------------------- +========================== The major elements found in an issue report include: @@ -110,7 +196,7 @@ The major elements found in an issue report include: * History Title -''''' +----- A brief description of the issue. Review whether the title is too generic or specifies an incorrect term or library. @@ -118,7 +204,7 @@ specifies an incorrect term or library. IDLE, doc, or asyncio. Type -'''' +---- Describes the type of issue. If an issue does not fit within any specific type, please do not set a type. @@ -150,13 +236,13 @@ specific type, please do not set a type. +----------------+----------------------------------------------------------+ Stage -''''' +----- A needed next action to advance the issue. The *stage* on GitHub issues is determined by presence of a linked PR and whether the issue is still open or closed. It is the PR that holds code review-related labels. Components -'''''''''' +---------- The area or Python library affected by the issue. A single issue can apply multiple component labels. @@ -184,7 +270,7 @@ One or more components may be selected for an issue: +-------------------+------------------------------------------------------+ Versions -'''''''' +-------- The known versions of Python that the issue affects and should be fixed for. Thus if an issue for a new feature is assigned for e.g., Python 3.8 but is not @@ -192,7 +278,7 @@ applied before Python 3.8.0 is released, this label should be updated to say ``python-3.9`` as the version and drop ``python-3.8``. Priority -'''''''' +-------- What is the severity and urgency? +------------------+--------------------------------------------------------+ @@ -209,14 +295,15 @@ What is the severity and urgency? | | an alpha release. | +------------------+--------------------------------------------------------+ -As a guideline, whether a bug is a *release blocker* for the current `release -schedule`_ is decided by the release manager. Triagers may recommend this -priority and should notify the release manager by tagging them in a comment -using ``@username``. If needed, consult the `release schedule`_ and the -release's associated PEP for the release manager's name. +As a guideline, whether a bug is a *release blocker* for the current +:ref:`release schedule ` is decided by the release manager. +Triagers may recommend this priority and should notify the release manager by +tagging them in a comment using ``@username``. If needed, consult the +:ref:`release schedule ` and the release's associated PEP for the +release manager's name. Keywords -'''''''' +-------- Various informational flags about the issue. Multiple values are possible. +---------------+------------------------------------------------------------+ @@ -227,7 +314,7 @@ Various informational flags about the issue. Multiple values are possible. +---------------+------------------------------------------------------------+ Nosy List -''''''''' +--------- A list of people who may be interested in an issue. This used to be a feature of the old issue tracker. On GitHub issues the @@ -243,7 +330,7 @@ decided this issue is not for you, you might click the *🔕 Unsubscribe* button in the sidebar. Assignees -''''''''' +--------- Who is expected to take the next step in resolving the issue. It is acceptable to assign an issue to someone if the issue cannot move @@ -255,7 +342,7 @@ Note that in order to assign an issue to someone, that person **must** be a team member, likely a Triager or a core developer. Dependencies -'''''''''''' +------------ The issue requires the listed issue(s) to be resolved first before it can move forward. This is achieved using checkbox lists in the initial issue description comment. Long story short, if you add this:: @@ -272,12 +359,12 @@ More details in the `official GitHub documentation `_. Superseder -'''''''''' +---------- The issue is a duplicate of the listed issue(s). To make GitHub mark an issue as duplicate, write "Duplicate of #xxxx" in a comment. Status -'''''' +------ +---------------+------------------------------------------------------------+ | Status | Description | @@ -288,13 +375,13 @@ Status +---------------+------------------------------------------------------------+ Linked pull requests -'''''''''''''''''''' +-------------------- A link might be added manually using the cog icon next to this field. Most commonly though, if the PR includes "Fixes #xxx" in its description, the link will be added automatically. Generating Special Links in a Comment -------------------------------------- +===================================== Using the following abbreviations in a comment will automatically generate a link to relevant web pages. @@ -310,63 +397,17 @@ a link to relevant web pages. | | generates a link to changeset ```` on GitHub. | +-------------------------------------------------------------+-------------------------------------------------------+ -Checklist for Triaging ----------------------- - -* Read the issue comment(s). -* Review and set classification fields - - Title: should be concise with specifics which are helpful to someone - scanning a list of issue titles. (Optional, if possible) Add a - prefix at the start of the title to indicate the module, e.g. IDLE, - doc, or async. - - Type - - Stage - - Components: multiple items may be set - - Versions: set if known, leave blank if unsure. Multiple items may be set. -* Review and set process fields - - Status - - Superseder - - Assignees - - Nosy List - - Priority - - Keywords -* (Optional) Leave a brief comment about the proposed next action needed. If - there is a long message list, a summary can be very helpful. - - -.. _CPython: https://github.com/python/cpython/ .. _Doc: https://github.com/python/cpython/tree/main/Doc/ .. _Grammar: https://github.com/python/cpython/tree/main/Grammar/ .. _Lib: https://github.com/python/cpython/tree/main/Lib/ -.. _Lib/lib2to3: https://github.com/python/cpython/tree/main/Lib/lib2to3/ -.. _Lib/ctypes: https://github.com/python/cpython/tree/main/Lib/ctypes/ -.. _Lib/distutils: https://github.com/python/cpython/tree/main/Lib/distutils/ .. _Lib/doctest.py: https://github.com/python/cpython/blob/main/Lib/doctest.py -.. _Lib/idlelib: https://github.com/python/cpython/tree/main/Lib/idlelib/ -.. _Lib/io.py: https://github.com/python/cpython/blob/main/Lib/io.py -.. _Lib/re.py: https://github.com/python/cpython/blob/main/Lib/re.py .. _Lib/test: https://github.com/python/cpython/tree/main/Lib/test/ .. _Lib/test/regrtest.py: https://github.com/python/cpython/blob/main/Lib/test/regrtest.py .. _Lib/test/support: https://github.com/python/cpython/tree/main/Lib/test/support/ -.. _Lib/tkinter: https://github.com/python/cpython/tree/main/Lib/tkinter/ .. _Lib/unittest: https://github.com/python/cpython/tree/main/Lib/unittest/ -.. _Lib/xml: https://github.com/python/cpython/tree/main/Lib/xml/ .. _Modules: https://github.com/python/cpython/tree/main/Modules/ -.. _Modules/_io: https://github.com/python/cpython/tree/main/Modules/_io/ -.. _Modules/_sre.c: https://github.com/python/cpython/blob/main/Modules/_sre.c .. _Objects: https://github.com/python/cpython/tree/main/Objects/ -.. _Objects/unicodeobject.c: https://github.com/python/cpython/blob/main/Objects/unicodeobject.c .. _Parser: https://github.com/python/cpython/tree/main/Parser/ .. _Python: https://github.com/python/cpython/tree/main/Python/ -.. _Tools: https://github.com/python/cpython/tree/main/Tools/ -.. _Tools/demo: https://github.com/python/cpython/tree/main/Tools/demo/ -.. _Developer's guide: https://github.com/python/devguide/ -.. _GSoC: https://summerofcode.withgoogle.com/ -.. _issue tracker: https://devguide.python.org/tracker/ -.. _GitHub pull requests: https://github.com/python/cpython/pulls -.. _Python source code repositories: https://github.com/python/cpython/ .. _Reporting security issues in Python: https://www.python.org/dev/security/ .. _python-ideas: https://mail.python.org/mailman3/lists/python-ideas.python.org -.. _release schedule: https://devguide.python.org/#status-of-python-branches -.. _PSF Code of Conduct: https://www.python.org/psf/conduct/ -.. _PEP 3121: https://www.python.org/dev/peps/pep-3121/ diff --git a/triage/triage-team.rst b/triage/triage-team.rst new file mode 100644 index 0000000000..36994cd72e --- /dev/null +++ b/triage/triage-team.rst @@ -0,0 +1,100 @@ +.. _triage-team: +.. _triagers: + +=========== +Triage Team +=========== + +The Python triage team is a group dedicated towards improving workflow +efficiency through thoughtful review and triage of open issues and pull +requests. This helps contributors receive timely feedback and enables core +developers to focus on reviewed items which reduces their workload. The +expectations of this role expand upon the "Triager" role on the +:ref:`issue tracker `. The responsibilities listed below are primarily centered +around the Python GitHub repositories. This extends beyond CPython, and, as +needed, to other repos such as devguide and core-workflow. + +Responsibilities include: + +* PR/issue management + - Reviewing PRs + - Assisting contributors + - Notifying appropriate core developers +* Applying appropriate labels to PRs/Issues + - Skip news + - Skip issue + - Good first issue + - Other categorizations + +Although triagers have the power to close PRs, they should generally not do so +without first consulting a core developer. By having triagers and core developers work together, +the author receives a careful consideration of their PR. This encourages future +contributions, regardless of whether their PR is accepted or closed. + +Nonetheless, triagers should feel free to close a PR if they judge that the +chance of the PR being merged would be exceedingly low, even if substantial +revisions were made to the PR. This includes (but is not limited to) the +following: + +* PRs proposing solely cosmetic changes +* PRs proposing changes to deprecated modules +* PRs that are no longer relevant. This includes: + - PRs proposing fixes for bugs that can no longer be reproduced + - PRs proposing changes that have been rejected by Python core developers + elsewhere (e.g. in an issue or a PEP rejection notice) + +If a triager has any doubt about whether to close a PR, they should consult a core +developer before taking any action. + +Triagers can also make use of the ``invalid`` and ``stale`` labels to suggest that a +PR may be suitable for closure. For more information, see the +:ref:`GitHub PR labels ` section. + +Note that it is of paramount importance to treat every contributor to the Python +project kindly and with respect. Regardless of whether they're entirely new +or a veteran core developer, they're actively choosing to voluntarily donate their +time towards the improvement of Python. As is the case with any member of +the Python Software Foundation, always follow the `PSF Code of Conduct`_. + +.. _PSF Code of Conduct: https://www.python.org/psf/conduct/ + + +Becoming a member of the Python triage team +=========================================== + +All Python core developers are welcome to invite a Python contributor to the +Python triage team. Triagers will be responsible to handle not just issues, but +also pull requests, and even managing backports. A Python triager has access to +more repositories than just CPython. + +When you have consistently shown the ability to properly +help triage issues without guidance, you may request that you +be given the "Triager" role on the :ref:`issue tracker `. You can make the request +to any core developer. If they decide you are ready +to gain the extra privileges on the tracker they will then act as a mentor to +you until you are ready to do things entirely on your own. There is no set rule +as to how many issues you need to have helped with before or how long you have +been participating. The key requirements are that you show the desire to +help, you are able to work well with others (especially those already with the +Triager role), and that have a firm grasp of how to do things on the issue +tracker properly on your own. + +Gaining the Triager role will allow you to set any value on any issue in the +tracker, releasing you from the burden of having to ask others to set values on +an issue for you in order to properly triage something. This will not only help +speed up and simplify your work in helping out, but also help lessen the +workload for everyone by gaining your help. + +Any existing active contributor to the Python repository on GitHub can +transition into becoming a Python triager. They can request this to any core +developer, either confidentially via a DM in Discourse, or +publicly by opening an `issue in the core-workflow repository +`_. +If the core devloper decides you are ready to gain the extra privileges on the +tracker, they will ask a :ref:`Python organization admin ` +to invite you to the Python organisation, and then act as a mentor to you until +you are ready to do things entirely on your own. + +For every new triager, it would be great to announce them in the python-committers +mailing list and core-workflow category in Discourse. `Example announcement +`_. diff --git a/triage/triaging.rst b/triage/triaging.rst new file mode 100644 index 0000000000..493429fe6d --- /dev/null +++ b/triage/triaging.rst @@ -0,0 +1,105 @@ +.. _triaging: + +================= +Triaging an Issue +================= + +This section of the devguide documents the :ref:`issue tracker ` for +users and developers. + +Contributors with the Triager role on the issue tracker can triage issues +directly without any assistance. + + +Checklist for Triaging +====================== + +* Read the issue comment(s). +* Review and set classification fields + - Title: should be concise with specifics which are helpful to someone + scanning a list of issue titles. (Optional, if possible) Add a + prefix at the start of the title to indicate the module, e.g. IDLE, + doc, or async. + - Type + - Stage + - Components: multiple items may be set + - Versions: set if known, leave blank if unsure. Multiple items may be set. +* Review and set process fields + - Status + - Superseder + - Assignees + - Nosy List + - Priority + - Keywords +* (Optional) Leave a brief comment about the proposed next action needed. If + there is a long message list, a summary can be very helpful. + + +.. _helptriage: + +Helping Triage Issues +===================== + +Once you know your way around how Python's source files are +structured and you are comfortable working with patches, a great way to +contribute is to help triage issues. Do realize, though, that experience +working on Python is needed in order to effectively help triage. + +Around the clock, new issues are being opened on the :ref:`issue tracker +` and existing issues are being updated. Every issue needs to be +triaged to make sure various things are in proper order. Even without special +privileges you can help with this process. + +Classifying Reports +------------------- + +For bugs, an issue needs to: + +* clearly explain the bug so it can be reproduced +* include all relevant platform details +* state what version(s) of Python are affected by the bug. + +These are things you can help with once you have experience developing for +Python: + +* try reproducing the bug: For instance, if a bug is not clearly explained + enough for you to reproduce it then there is a good chance a core developer + won't be able to either. +* see if the issue happens on a different Python version: It is always helpful + to know if a bug not only affects the in-development version of Python, but + whether it also affects other versions in maintenance mode. +* write a unit test: If the bug lacks a unit test that should end up in + Python's test suite, having that written can be very helpful. + +This is all helpful as it allows triagers (i.e., +:ref:`people with the Developer role on the issue tracker `) to +properly classify an issue so it can be handled by the right core developers in +a timely fashion. + +Reviewing Patches +----------------- + +If an issue has a pull request attached that has not been reviewed, you can help +by making sure the patch: + +* follows the style guides +* applies cleanly to an up-to-date clone +* is a good solution to the problem it is trying to solve +* includes proper tests +* includes proper documentation changes +* submitter is listed in ``Misc/ACKS``, either already or the patch adds them + +Doing all of this allows core developers and :ref:`triagers` to more +quickly look for subtle issues that only people with extensive experience +working on Python's code base will notice. + +Finding an Issue You Can Help With +---------------------------------- + +If you want to help triage issues, you might also want to search for issues +in modules which you have a working knowledge. Search for the name of a module +in the issue tracker or use the `advanced search`_ query builder to search for +specific kinds of issues (e.g. the "Windows" label if you are a Windows +developer, "Extension Modules" if you are familiar with C, etc.). + +.. _advanced search: https://github.com/search/advanced diff --git a/versions.rst b/versions.rst new file mode 100644 index 0000000000..301bdead22 --- /dev/null +++ b/versions.rst @@ -0,0 +1,64 @@ +.. _versions: +.. _branchstatus: + +========================= +Status of Python Versions +========================= + +The main branch is currently the future Python 3.12, and is the only +branch that accepts new features. The latest release for each Python +version can be found on the `download page `_. + + +Supported Versions +================== + +Dates shown in *italic* are scheduled and can be adjusted. + +====== ========== ======== ============= ============ ===================== +Branch Schedule Status First release End-of-life Release manager +====== ========== ======== ============= ============ ===================== +main TBA features *2023-10-03* *2028-10* Thomas Wouters +3.11 :pep:`664` bugfix *2022-10-03* *2027-10* Pablo Galindo Salgado +3.10 :pep:`619` bugfix 2021-10-04 *2026-10* Pablo Galindo Salgado +3.9 :pep:`596` security 2020-10-05 *2025-10* Łukasz Langa +3.8 :pep:`569` security 2019-10-14 *2024-10* Łukasz Langa +3.7 :pep:`537` security 2018-06-27 *2023-06-27* Ned Deily +====== ========== ======== ============= ============ ===================== + + +Unsupported versions +==================== + +====== ========== =========== ============= =========== ================================ +Branch Schedule Status First release End-of-life Release manager +====== ========== =========== ============= =========== ================================ +3.6 :pep:`494` end-of-life 2016-12-23 2021-12-23 Ned Deily +3.5 :pep:`478` end-of-life 2015-09-13 2020-09-30 Larry Hastings +3.4 :pep:`429` end-of-life 2014-03-16 2019-03-18 Larry Hastings +3.3 :pep:`398` end-of-life 2012-09-29 2017-09-29 Georg Brandl, Ned Deily (3.3.7+) +3.2 :pep:`392` end-of-life 2011-02-20 2016-02-20 Georg Brandl +3.1 :pep:`375` end-of-life 2009-06-27 2012-04-09 Benjamin Peterson +3.0 :pep:`361` end-of-life 2008-12-03 2009-06-27 Barry Warsaw +2.7 :pep:`373` end-of-life 2010-07-03 2020-01-01 Benjamin Peterson +2.6 :pep:`361` end-of-life 2008-10-01 2013-10-29 Barry Warsaw +====== ========== =========== ============= =========== ================================ + + +Status key +========== + +:features: new features, bugfixes, and security fixes are accepted. +:prerelease: feature fixes, bugfixes, and security fixes are accepted for the + upcoming feature release. +:bugfix: bugfixes and security fixes are accepted, new binaries are still + released. (Also called **maintenance** mode or **stable** release) +:security: only security fixes are accepted and no more binaries are released, + but new source-only versions can be released +:end-of-life: release cycle is frozen; no further changes can be pushed to it. + +See also the :ref:`devcycle` page for more information about branches. + +By default, the end-of-life is scheduled 5 years after the first release, +but can be adjusted by the release manager of each branch. All Python 2 +versions have reached end-of-life.