Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-104773: PEP 594: Remove cgi and cgitb modules #104775

Merged
merged 2 commits into from
May 24, 2023
Merged

gh-104773: PEP 594: Remove cgi and cgitb modules #104775

merged 2 commits into from
May 24, 2023

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented May 22, 2023 •
edited
Loading

@bedevere-bot bedevere-bot mentioned this pull request May 22, 2023
Closed
19 tasks
@vstinner
Copy link
Member Author

vstinner commented May 23, 2023 •
edited
Loading

PR updated to fix the doc: Warning: (...)/Doc/library/security_warnings.rst:12: WARNING: undefined label: cgi-security.

@vstinner vstinner changed the title gh-104773: PEP 594: Remove cgi and cgitb modules gh-104773: PEP 594: Remove cgi and cgitb modules May 23, 2023
@vstinner vstinner force-pushed the remove_cgi branch 2 times, most recently from 7a4a015 to 7cbc500 Compare May 23, 2023 00:36
@vstinner
Copy link
Member Author

Update: fix test_pyclbr.

@vstinner
Copy link
Member Author

It seems like cgi.escape() was used in Python 2. There is still compatibility code in the wild using it. Example with distlib/compat.py:

try:
    from html import escape
except ImportError:
    from cgi import escape

src: https://github.com/pypa/distlib/blob/05375908c1b2d6b0e74bdeb574569d3609db9f56/distlib/compat.py#L470-L474

@vstinner
Copy link
Member Author

Using import cgi|from cgi regex, I found a bunch of PyPI top 5,000 projects (PYPI-2023-04-13) using cgi and cgitb modules.

Affected projects (91):

  • Cheetah3-3.2.6.post1
  • CherryPy (18.8.0)
  • Cython (0.29.34)
  • FormEncode (2.0.1)
  • Mako (1.2.4)
  • PTable (0.9.2)
  • Paste (3.5.2)
  • PasteDeploy (3.0.1)
  • PasteScript (3.3.0)
  • PyVISA (1.13.0)
  • Pyro4 (4.82)
  • ShopifyAPI (12.3.0)
  • Tempita (0.5.2)
  • Twisted (22.10.0)
  • WMI (1.5.1)
  • WebHelpers (1.3)
  • WebOb (1.8.7)
  • WebTest (3.0.0)
  • Whoosh (2.7.4)
  • alexapy (1.26.5)
  • aliyun-python-sdk-core (2.13.36)
  • ansible (7.4.0)
  • asana (3.2.0)
  • astropy (5.2.2)
  • asyncio (3.4.3)
  • atlassian-jwt-auth (17.0.0)
  • boltons (23.0.0)
  • boto (2.49.0)
  • braintree (4.19.0)
  • chispa (0.9.2)
  • cinemagoer (2022.12.27)
  • conda (4.3.16)
  • coreapi (2.3.3)
  • cssutils (2.6.0)
  • devpi-client (6.0.4)
  • distlib (0.3.6)
  • docxtpl (0.16.6)
  • eli5 (0.13.0)
  • emails (0.6)
  • eventlet (0.33.3)
  • falcon (3.1.1)
  • feedparser (6.0.10)
  • fissix (21.11.13)
  • gevent (22.10.2)
  • google-apitools (0.5.32)
  • gsutil (5.23)
  • gviz_api (1.10.0)
  • htmlmin (0.1.12)
  • ibm-cos-sdk-core (2.13.0)
  • isort (5.12.0)
  • jenkinsapi (0.3.13)
  • json-ref-dict (0.7.1)
  • json2html (1.3.0)
  • jusText (3.0.0)
  • lxml (4.9.2)
  • mechanize (0.4.8)
  • mercurial (6.4.1)
  • mozlog (7.1.1)
  • nibabel (5.1.0)
  • nltk (3.8.1)
  • numpy (1.24.2)
  • oauth (1.0.1)
  • oauth2client (4.1.3)
  • odfpy (1.4.1)
  • oletools (0.60.1)
  • os_sys (2.1.4)
  • pdfminer (20191125)
  • peppercorn (0.6)
  • pex (2.1.131)
  • prov (2.0.0)
  • pyLDAvis (3.4.0)
  • pyrocko (2023.3.27)
  • pysaml2 (7.4.1)
  • pystache (0.6.0)
  • python-mimeparse (1.6.0)
  • python-openid (2.2.5)
  • python3-openid (3.2.0)
  • pyvmomi (8.0.0.1.2)
  • scapy (2.5.0)
  • sentry-sdk (1.19.1)
  • setuptools (67.6.1)
  • simplekml (1.3.6)
  • speedtest-cli (2.1.3)
  • supervisor (4.2.5)
  • tableauserverclient (0.25)
  • tqdm (4.65.0)
  • twine (4.0.2)
  • uri (2.0.1)
  • web.py (0.62)
  • webapp2 (2.5.2)
  • zeep (4.2.1)
Code: 
Cheetah3-3.2.6.post1/Cheetah/Filters.py: # These substitutions are copied from cgi.escape().
Cheetah3-3.2.6.post1/Cheetah/Template.py: import cgi  # Used by .webInput() if the template is a CGI script.
Cheetah3-3.2.6.post1/Cheetah/Tests/Regressions.py: from cgi import escape as html_escape
CherryPy-18.8.0/cherrypy/lib/covercp.py: import cgi
CherryPy-18.8.0/cherrypy/lib/httputil.py: from cgi import parse_header
Cython-0.29.34/Cython/Tempita/_tempita.py: import cgi
FormEncode-2.0.1/formencode/fieldstorage.py: import cgi
FormEncode-2.0.1/formencode/htmlgen.py: from cgi import escape
FormEncode-2.0.1/formencode/rewritingparser.py: from cgi import escape
FormEncode-2.0.1/formencode/validators.py: import cgi
Mako-1.2.4/examples/wsgi/run_wsgi.py: import cgi
PTable-0.9.2/prettytable/_compact.py: from cgi import escape
Paste-3.5.2/paste/auth/open_id.py: import cgi
Paste-3.5.2/paste/cgitb_catcher.py: import cgitb
Paste-3.5.2/paste/debug/doctest_webapp.py: import cgi
Paste-3.5.2/paste/debug/prints.py: import cgi
Paste-3.5.2/paste/debug/profile.py: import cgi
Paste-3.5.2/paste/debug/watchthreads.py: import cgi
Paste-3.5.2/paste/debug/wdg_validate.py: import cgi
Paste-3.5.2/paste/exceptions/errormiddleware.py: import cgi
Paste-3.5.2/paste/request.py: import cgi
Paste-3.5.2/paste/util/multidict.py: import cgi
Paste-3.5.2/tests/cgiapp_data/form.cgi: import cgi
Paste-3.5.2/tests/test_fixture.py: import cgi
Paste-3.5.2/tests/test_multidict.py: import cgi
Paste-3.5.2/tests/test_wsgiwrappers.py: import cgi
PasteDeploy-3.0.1/src/paste/deploy/paster_templates/paste_deploy/+package+/sampleapp.py_tmpl: import cgi
PasteScript-3.3.0/paste/script/copydir.py: import cgi as html
PasteScript-3.3.0/paste/script/exe.py: # Maybe import cgitb or something?
PasteScript-3.3.0/paste/script/testapp.py: import cgi as html
PyVISA-1.13.0/pyvisa/thirdparty/prettytable.py: from cgi import escape
Pyro4-4.82/src/Pyro4/utils/httpgateway.py: import cgi
ShopifyAPI-12.3.0/shopify/collection.py: import cgi
Tempita-0.5.2/tempita/__init__.py: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger15.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger16.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger17.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger18.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19a.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19b.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19c.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger20.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger21.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger22.py: import cgi
Twisted-22.10.0/src/twisted/web/http.py: import cgi
Twisted-22.10.0/src/twisted/web/test/test_cgi.py: import cgi
WMI-1.5.1/wmiweb.py: from cgi import escape
WebHelpers-1.3/webhelpers/util.py: from cgi import parse_qs
WebHelpers-1.3/webhelpers/util.py: import cgi
WebOb-1.8.7/src/webob/compat.py: from cgi import FieldStorage as _cgi_FieldStorage
WebOb-1.8.7/src/webob/compat.py: from cgi import FieldStorage as cgi_FieldStorage
WebOb-1.8.7/src/webob/compat.py: from cgi import escape
WebOb-1.8.7/src/webob/compat.py: from cgi import parse_header
WebOb-1.8.7/src/webob/compat.py: import cgi
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_in_wsgiref.py: import cgi
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_request.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_request.py: import cgi
WebOb-1.8.7/tests/test_request.py: import cgi
WebOb-1.8.7/tests/test_request.py: import cgi
WebTest-3.0.0/tests/test_forms.py: import cgi
Whoosh-2.7.4/src/whoosh/compat.py: import cgi
alexapy-1.26.5/alexapy/aiohttp/helpers.py: import cgi
aliyun-python-sdk-core-2.13.36/aliyunsdkcore/vendored/requests/utils.py: import cgi
ansible-7.4.0/ansible_collections/community/general/plugins/modules/campfire.py: import cgi
asana-3.2.0/tests/test_client_attachments.py: import cgi
astropy-5.2.2/cextern/cfitsio/docs/changes.txt: that even fits files returned from cgi queries which have the wrong
asyncio-3.4.3/examples/crawl.py: import cgi
atlassian-jwt-auth-17.0.0/atlassian_jwt_auth/key.py: import cgi
boltons-23.0.0/boltons/tableutils.py: from cgi import escape as html_escape
boltons-23.0.0/boltons/tableutils.py: from cgi import escape as html_escape
boto-2.49.0/boto/ecs/item.py: import cgi
braintree-4.19.0/braintree/apple_pay_gateway.py: from cgi import escape
chispa-0.9.2/chispa/prettytable.py: from cgi import escape
cinemagoer-2022.12.27/imdb/helpers.py: from cgi import escape
cinemagoer-2022.12.27/imdb/parser/http/piculet.py: from cgi import escape as html_escape
conda-4.3.16/conda/gateways/adapters/ftp.py: import cgi
coreapi-2.3.3/coreapi/codecs/download.py: import cgi
cssutils-2.6.0/cssutils/_fetchgae.py: import cgi
cssutils-2.6.0/cssutils/tests/test_util.py: import cgi
cssutils-2.6.0/encutils/__init__.py: import cgi
cssutils-2.6.0/tools/try.py: import cgi
devpi-client-6.0.4/testing/simpypi.py: from cgi import escape  # type: ignore
distlib-0.3.6/tests/test_database.py: import cgi
docxtpl-0.16.6/docxtpl/listing.py: from cgi import escape
docxtpl-0.16.6/docxtpl/richtext.py: from cgi import escape
docxtpl-0.16.6/docxtpl/template.py: from cgi import escape  # noqa: F401
eli5-0.13.0/eli5/formatters/html.py: from cgi import escape  # type: ignore
emails-0.6/emails/loader/helpers.py: import cgi
eventlet-0.33.3/tests/wsgi_test.py: import cgi
falcon-3.1.1/falcon/asgi/multipart.py: import cgi
falcon-3.1.1/falcon/media/multipart.py: import cgi
falcon-3.1.1/falcon/testing/helpers.py: import cgi
falcon-3.1.1/falcon/vendor/mimeparse/mimeparse.py: import cgi
feedparser-6.0.10/feedparser/encodings.py: import cgi
fissix-21.11.13/fissix/tests/test_util.py: self.assertEqual(str(node), "import cgi\nbar()\n\n")
gevent-22.10.2/examples/webproxy.py: from cgi import escape
gevent-22.10.2/src/greentest/2.7/test_httpservers.py: import cgi
gevent-22.10.2/src/greentest/2.7pypy/test_httpservers.py: import cgi
gevent-22.10.2/src/greentest/3.7pypy/test_httpservers.py: import cgi
google-apitools-0.5.32/apitools/base/protorpclite/test_util.py: import cgi
gsutil-5.23/gslib/vendored/boto/boto/ecs/item.py: import cgi
gsutil-5.23/gslib/vendored/oauth2client/oauth2client/contrib/appengine.py: import cgi
gviz_api-1.10.0/gviz_api.py: import cgi as html  # Only used for .escape()
htmlmin-0.1.12/htmlmin/escape.py: from cgi import escape
htmlmin-0.1.12/htmlmin/main.py: import cgi
ibm-cos-sdk-core-2.13.0/ibm_botocore/utils.py: import cgi
isort-5.12.0/tests/unit/profiles/test_django.py: """import cgi
jenkinsapi-0.3.13/examples/low_level/post_watcher.py: import cgi
jenkinsapi-0.3.13/jenkinsapi/utils/simple_post_logger.py: import cgi
json-ref-dict-0.7.1/json_ref_dict/loader.py: import cgi
json2html-1.3.0/json2html/jsonconv.py: from cgi import escape as html_escape
jusText-3.0.0/justext/_compat.py: from cgi import escape
lxml-4.9.2/doc/s5/ep2008/atom.py: import cgi
lxml-4.9.2/src/lxml/doctestcompare.py: from cgi import escape as html_escape
lxml-4.9.2/src/lxml/html/diff.py: from cgi import escape as html_escape
mechanize-0.4.8/examples/forms/echo.cgi: import cgi
mercurial-6.4.1/contrib/hgweb.fcgi: # import cgitb; cgitb.enable()
mercurial-6.4.1/contrib/hgweb.wsgi: #import cgitb; cgitb.enable()
mercurial-6.4.1/hgweb.cgi: # import cgitb; cgitb.enable()
mercurial-6.4.1/tests/test-clone-cgi.t: > import cgitb
mercurial-6.4.1/tests/test-mq.t: > import cgitb
mercurial-6.4.1/tests/test-newcgi.t: > import cgitb
mercurial-6.4.1/tests/test-newcgi.t: > import cgitb
mercurial-6.4.1/tests/test-newercgi.t: > import cgitb
mercurial-6.4.1/tests/test-newercgi.t: > import cgitb
mercurial-6.4.1/tests/test-oldcgi.t: > import cgitb, os, sys
mercurial-6.4.1/tests/test-oldcgi.t: > import cgitb, sys
mercurial-6.4.1/tests/test-push-cgi.t: > import cgitb
mozlog-7.1.1/mozlog/formatters/html/html.py: from cgi import escape
nibabel-5.1.0/tools/dicomfs.wsgi: import cgi
nltk-3.8.1/nltk/tree/prettyprinter.py: from cgi import escape
numpy-1.24.2/tools/npy_tempita/__init__.py: from cgi import escape as html_escape
oauth-1.0.1/oauth/oauth.py: import cgi
oauth2client-4.1.3/oauth2client/contrib/appengine.py: import cgi
odfpy-1.4.1/contrib/html2odt/html2odt.py: from cgi import escape,parse_header
odfpy-1.4.1/contrib/html2odt/shtml2odt.py: from cgi import escape,parse_header
oletools-0.60.1/oletools/thirdparty/prettytable/prettytable.py: from cgi import escape
os_sys-2.1.4/bars/bars/notebook.py: from cgi import escape
os_sys-2.1.4/bars/notebook.py: from cgi import escape
os_sys-2.1.4/os_sys/download.py: import cgi
os_sys-2.1.4/server/core/handlers/wsgi.py: import cgi
os_sys-2.1.4/server/core/management/templates.py: import cgi
os_sys-2.1.4/server/http/multipartparser.py: import cgi
pdfminer-20191125/tools/pdf2html.cgi: #import cgitb; cgitb.enable()
pdfminer-20191125/tools/pdf2html.cgi: import cgi, logging, traceback, random
peppercorn-0.6/peppercorn/tests.py: from cgi import FieldStorage
pex-2.1.131/pex/vendor/_vendored/pip/pip/_internal/index/collector.py: import cgi
pex-2.1.131/pex/vendor/_vendored/pip/pip/_internal/network/download.py: import cgi
prov-2.0.0/src/prov/dot.py: from cgi import escape
pyLDAvis-3.4.0/pyLDAvis/lib/python3.11/site-packages/setuptools/_distutils/config.py: import cgi
pyrocko-2023.3.27/src/gf/server.py: from cgi import escape
pyrocko-2023.3.27/src/gf/server.py: import cgi
pysaml2-7.4.1/src/saml2/httputil.py: import cgi
pysaml2-7.4.1/src/saml2/pack.py: import cgi as html  # type: ignore[no-redef]
pystache-0.6.0/pystache/defaults.py: from cgi import escape
python-mimeparse-1.6.0/mimeparse.py: import cgi
python-openid-2.2.5/contrib/openid-parse: import cgi, re, subprocess, sys
python-openid-2.2.5/examples/consumer.py: import cgi
python-openid-2.2.5/examples/consumer.py: import cgitb
python-openid-2.2.5/examples/djopenid/server/views.py: import cgi
python-openid-2.2.5/examples/server.py: import cgi
python-openid-2.2.5/examples/server.py: import cgitb
python-openid-2.2.5/openid/consumer/consumer.py: import cgi
python-openid-2.2.5/openid/test/test_auth_request.py: import cgi
python-openid-2.2.5/openid/test/test_consumer.py: import cgi
python-openid-2.2.5/openid/test/test_message.py: import cgi
python-openid-2.2.5/openid/test/test_server.py: import cgi
python3-openid-3.2.0/contrib/openid-parse: import cgi, re, subprocess, sys
python3-openid-3.2.0/examples/consumer.py: import cgitb
python3-openid-3.2.0/examples/djopenid/server/views.py: import cgi
python3-openid-3.2.0/examples/server.py: import cgitb
pyvmomi-8.0.0.1.2/pyVim/sso.py: from cgi import escape
scapy-2.5.0/scapy/compat.py: import cgi
sentry-sdk-1.19.1/sentry_sdk/integrations/pyramid.py: from webob.compat import cgi_FieldStorage  # type: ignore
sentry-sdk-1.19.1/sentry_sdk/utils.py: from cgi import parse_qs  # type: ignore
setuptools-67.6.1/setuptools/_distutils/config.py: import cgi
simplekml-1.3.6/simplekml/base.py: import cgi
speedtest-cli-2.1.3/speedtest.py: from cgi import parse_qs
supervisor-4.2.5/supervisor/compat.py: from cgi import escape
tableauserverclient-0.25/tableauserverclient/server/endpoint/datasources_endpoint.py: import cgi
tableauserverclient-0.25/tableauserverclient/server/endpoint/flows_endpoint.py: import cgi
tableauserverclient-0.25/tableauserverclient/server/endpoint/workbooks_endpoint.py: import cgi
tqdm-4.65.0/tqdm/notebook.py: from cgi import escape
twine-4.0.2/twine/commands/check.py: import cgi
uri-2.0.1/uri/compat.py: from cgi import escape
web.py-0.62/web/webapi.py: import cgi
webapp2-2.5.2/tests/request_test.py: import cgi
webapp2-2.5.2/tests/request_test.py: import cgi
webapp2-2.5.2/webapp2.py: import cgi
zeep-4.2.1/src/zeep/utils.py: import cgi

@hugovk hugovk self-requested a review May 23, 2023 09:44
hugovk
hugovk approved these changes May 23, 2023
View reviewed changes
Copy link
Member

@hugovk hugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹

@vstinner vstinner force-pushed the remove_cgi branch 2 times, most recently from c0001e7 to 734ae51 Compare May 24, 2023 07:38
@vstinner
Copy link
Member Author

test_importlib modified the environment on the Ubuntu job. I wrote a fix for that: #104840

@vstinner
pythongh-104773: PEP 594: Remove cgi and cgitb modules
10b3f34 
* Replace "cgi" with "!cgi" in the Sphinx documentation to avoid
  warnings on broken references.
* test_pyclbr no longer tests the cgi module.
@vstinner
Explain how to port existing code
5ad9cf6
@vstinner
Copy link
Member Author

I re-ran the Ubuntu job and test_importlib failed again.

test_importlib modified the environment on the Ubuntu job. I wrote a fix for that: #104840

I merged the test_importlib fix. I rebased my PR on top of it.

@vstinner vstinner enabled auto-merge (squash) May 24, 2023 08:57
@vstinner vstinner merged commit 08d5923 into python:main May 24, 2023
@vstinner vstinner deleted the remove_cgi branch May 24, 2023 09:05
@vstinner
Copy link
Member Author

Bye bye cgi! I added instructions on how to port existing code.

@JelleZijlstra JelleZijlstra mentioned this pull request May 25, 2023
Closed
@vstinner
Copy link
Member Author

@JulienPalard: Oh, I'm surprising that sphinx-lint missing this typo:

* :pep:`594`: Remove the :mod:`!cgi`` and :mod:`!cgitb` modules,
  deprecated in Python 3.11.

@vstinner vstinner mentioned this pull request May 31, 2023
Open
@Uzume Uzume mentioned this pull request May 18, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gpshead gpshead gpshead approved these changes

@hugovk hugovk hugovk approved these changes

@ethanfurman ethanfurman Awaiting requested review from ethanfurman ethanfurman is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@vstinner @gpshead @hugovk @bedevere-bot