Remove unused `# type: ignore`s #12531

DMRobertson · 2022-04-22T19:18:06Z

Over time we've begun to use newer versions of mypy, typeshed, stub
packages---and of course we've improved our own annotations. This makes
some type ignore comments no longer necessary. I have removed them.

There was one exception: a module that imports select.epoll. The
ignore is redundant on Linux, but I've kept it ignored for those of us
who work on the source tree using not-Linux. (#11771)

I'm more interested in the config line which enforces this. I want
unused ignores to be reported, because I think it's useful feedback when
annotating to know when you've fixed a problem you had to previously
ignore.

Over time we've begun to use newer versions of mypy, typeshed, stub packages---and of course we've improved our own annotations. This makes some type ignore comments no longer necessary. I have removed them. There was one exception: a module that imports `select.epoll`. The ignore is redundant on Linux, but I've kept it ignored for those of us who work on the source tree using not-Linux. (#11771) I'm more interested in the config line which enforces this. I want unused ignores to be reported, because I think it's useful feedback when annotating to know when you've fixed a problem you had to previously ignore.

synapse/app/_base.py

synapse/module_api/__init__.py

DMRobertson · 2022-04-22T19:38:11Z

tests/handlers/test_register.py

-        # Type ignore: mypy doesn't like us assigning to methods.
-        self.store.count_monthly_users = Mock(  # type: ignore[assignment]
+        self.store.count_monthly_users = Mock(


I don't know why mypy is now happy with this. I'm guessing that it thinks self.store is Any.

Given that this is just a test file I wasn't too worried about investigating further.

DMRobertson · 2022-04-22T21:11:29Z

mypy passes locally but fails in CI. That must be because I have --extras all installed locally, but not installed in CI.

Bugger. Maybe I should finally cave and pass through a list of extras to setup-python-poetry.

DMRobertson · 2022-04-25T11:31:38Z

.github/workflows/tests.yml

-    # This does a vanilla `poetry install` - no extras. I'm slightly anxious
-    # that we might skip some typechecks on code that uses extras. However,
-    # I think the right way to fix this is to mark any extras needed for
-    # typechecking as development dependencies. To detect this, we ought to
-    # turn up mypy's strictness: disallow unknown imports and be accept fewer
-    # uses of `Any`.
-    uses: "matrix-org/backend-meta/.github/workflows/python-poetry-ci.yml@v1"
+    uses: "matrix-org/backend-meta/.github/workflows/python-poetry-ci.yml@dmr/extras-for-typechecking"
+    with:
+      typechecking-extras: "all"


Looks like I was right to be anxious.

On reflection, I think the approach I described on the left was too idealistic.

Before merging the branch of backend-meta should be reset to point to the v1 tag.

clokep · 2022-04-25T13:00:07Z

I'm more interested in the config line which enforces this. I want unused ignores to be reported, because I think it's useful feedback when annotating to know when you've fixed a problem you had to previously ignore.

Thanks for tackling this! 👍

clokep · 2022-04-25T18:49:00Z

I think @squahtx already gave some feedback on this, so I'm assuming the review is back to them!

squahtx

It's great to see stale # type: ignores being removed. I'm largely happy with the changes, save for the two questions/comments above.

DMRobertson · 2022-04-25T19:45:14Z

It's great to see stale # type: ignores being removed. I'm largely happy with the changes, save for the two questions/comments above.

Ahh sorry, I think I lost track of this today. I'll get those sorted tomorrow.

whatever that is.

Now that matrix-org/backend-meta#6 is merged to the release/v1 branch.

squahtx

lgtm!

DMRobertson · 2022-04-27T13:01:59Z

Sytest failure seems to be matrix-org/sytest#1113.

Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574.

Twisted trunk job already does this. Missed in #12531.

Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574.

Twisted trunk job already does this. Missed in #12531.

* Allow unused ignores in "bleeding edge" CI Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574. * Use `--extras all` in latest deps mypy CI Twisted trunk job already does this. Missed in #12531. * changelog

@SimonBrandner

Synapse 1.59.0rc1 (2022-05-10) ============================== This release makes several changes that server administrators should be aware of: - Device name lookup over federation is now disabled by default. ([\#12616](#12616)) - The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](#12452), [\#12654](#12654)) See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details. Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](#12597)) Features -------- - Support [MSC3266](matrix-org/matrix-spec-proposals#3266) room summaries over federation. ([\#11507](#11507)) - Implement [changes](matrix-org/matrix-spec-proposals@4a77139) to [MSC2285 (hidden read receipts)](matrix-org/matrix-spec-proposals#2285). Contributed by @SimonBrandner. ([\#12168](#12168), [\#12635](#12635), [\#12636](#12636), [\#12670](#12670)) - Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py) to allow modules to change actions for existing push rules of local users. ([\#12406](#12406)) - Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](#12452)) - Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](#12654)) - Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](#12526)) - Implement [MSC3786](matrix-org/matrix-spec-proposals#3786): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](#12601)) - Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](#12619)) Bugfixes -------- - Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](#12273)) - Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](#12544)) - Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](#12570)) - Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](#12580)) - Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](#12594)) - Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](#12633)) - Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](#12657)) - Prevent memory leak from reoccurring when presence is disabled. ([\#12656](#12656)) Updates to the Docker image --------------------------- - Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](#12541)) - Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](#12573)) Improved Documentation ---------------------- - Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](#12536)) - Add missing linebreak to `pipx` install instructions. ([\#12579](#12579)) - Add information about the TCP replication module to docs. ([\#12621](#12621)) - Fixes to the formatting of `README.rst`. ([\#12627](#12627)) - Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](#12664)) Deprecations and Removals ------------------------- - Remove unstable identifiers from [MSC3069](matrix-org/matrix-spec-proposals#3069). ([\#12596](#12596)) - Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from [MSC2778](matrix-org/matrix-spec-proposals#2778). ([\#12597](#12597)) - Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](#12613)) Internal Changes ---------------- - Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](#12480)) - Immediately retry any requests that have backed off when a server comes back online. ([\#12500](#12500)) - Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](#12505)) - Consistently check if an object is a `frozendict`. ([\#12564](#12564)) - Protect module callbacks with read semantics against cancellation. ([\#12568](#12568)) - Improve comments and error messages around access tokens. ([\#12577](#12577)) - Improve docstrings for the receipts store. ([\#12581](#12581)) - Use constants for read-receipts in tests. ([\#12582](#12582)) - Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](#12587), [\#12663](#12663)) - Remove special-case for `twisted` logger from default log config. ([\#12589](#12589)) - Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](#12599)) - Add link to documentation in Grafana Dashboard. ([\#12602](#12602)) - Reduce log spam when running multiple event persisters. ([\#12610](#12610)) - Add extra debug logging to federation sender. ([\#12614](#12614)) - Prevent remote homeservers from requesting local user device names by default. ([\#12616](#12616)) - Add a consistency check on events which we read from the database. ([\#12620](#12620)) - Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](#12624)) - Remove unused code related to receipts. ([\#12632](#12632)) - Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](#12637)) - Move `pympler` back in to the `all` extras. ([\#12652](#12652)) - Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](#12665)) - Release script: confirm the commit to be tagged before tagging. ([\#12556](#12556)) - Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](#12612)) - Fix scripts-dev to pass typechecking. ([\#12356](#12356)) - Add some type hints to datastore. ([\#12485](#12485)) - Remove unused `# type: ignore`s. ([\#12531](#12531)) - Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](#12576)) - Remove redundant lines of config from `mypy.ini`. ([\#12608](#12608)) - Update to mypy 0.950. ([\#12650](#12650)) - Use `Concatenate` to better annotate `_do_execute`. ([\#12666](#12666)) - Use `ParamSpec` to refine type hints. ([\#12667](#12667)) - Fix mypy against latest pillow stubs. ([\#12671](#12671))

David Robertson added 4 commits April 22, 2022 20:09

Changelog

548846f

Fixup test_account_data_manager

dd2ca58

Fixup formatter

7491a1b

DMRobertson commented Apr 22, 2022

View reviewed changes

DMRobertson mentioned this pull request Apr 25, 2022

Allow extras to be installed for typechecking matrix-org/backend-meta#6

Merged

David Robertson added 2 commits April 25, 2022 11:49

Try installing extras before typechecking

02ea483

Fix test twisted import for olddeps

38ece60

DMRobertson commented Apr 25, 2022

View reviewed changes

DMRobertson marked this pull request as ready for review April 25, 2022 11:32

DMRobertson requested a review from a team as a code owner April 25, 2022 11:32

clokep requested review from squahtx and removed request for a team April 25, 2022 18:48

squahtx reviewed Apr 25, 2022

View reviewed changes

David Robertson added 2 commits April 27, 2022 12:45

use JsonMapping as return type of get_global

8388649

whatever that is.

Use v1 tag for backend-meta again

68e1cc8

Now that matrix-org/backend-meta#6 is merged to the release/v1 branch.

squahtx approved these changes Apr 27, 2022

View reviewed changes

DMRobertson merged commit 6463244 into develop Apr 27, 2022

DMRobertson deleted the dmr/typing/warn-unused-ignores branch April 27, 2022 13:03

DMRobertson mentioned this pull request Apr 28, 2022

CI run against latest deps is failing #12574

Closed

DMRobertson pushed a commit that referenced this pull request Apr 28, 2022

Allow unused ignores in "bleeding edge" CI

ce6ecdd

Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574.

DMRobertson pushed a commit that referenced this pull request Apr 28, 2022

Use --extras all in latest deps mypy CI

f282d5f

Twisted trunk job already does this. Missed in #12531.

DMRobertson pushed a commit that referenced this pull request Apr 28, 2022

Allow unused ignores in "bleeding edge" CI

4fadcad

Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574.

DMRobertson pushed a commit that referenced this pull request Apr 28, 2022

Use --extras all in latest deps mypy CI

505d861

Twisted trunk job already does this. Missed in #12531.

DMRobertson mentioned this pull request Apr 28, 2022

Allow unused ignores in "bleeding edge" CI #12576

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove unused `# type: ignore`s #12531

Remove unused `# type: ignore`s #12531

DMRobertson commented Apr 22, 2022

DMRobertson Apr 22, 2022

DMRobertson commented Apr 22, 2022

DMRobertson Apr 25, 2022

clokep commented Apr 25, 2022

clokep commented Apr 25, 2022

squahtx left a comment

DMRobertson commented Apr 25, 2022

squahtx left a comment

DMRobertson commented Apr 27, 2022

Remove unused # type: ignores #12531

Remove unused # type: ignores #12531

Conversation

DMRobertson commented Apr 22, 2022

DMRobertson Apr 22, 2022

Choose a reason for hiding this comment

DMRobertson commented Apr 22, 2022

DMRobertson Apr 25, 2022

Choose a reason for hiding this comment

clokep commented Apr 25, 2022

clokep commented Apr 25, 2022

squahtx left a comment

Choose a reason for hiding this comment

DMRobertson commented Apr 25, 2022

squahtx left a comment

Choose a reason for hiding this comment

DMRobertson commented Apr 27, 2022

Remove unused `# type: ignore`s #12531

Remove unused `# type: ignore`s #12531