From 37ec1ba202f8bcf2622257e54cc1298273036918 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 9 Jun 2021 02:29:15 -0500 Subject: [PATCH 1/3] Document how to see logger output when running the twisted tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit As shared by @erikjohnston ❤ https://github.com/matrix-org/synapse/pull/9247#discussion_r647237778 --- README.rst | 36 ++++++++++++++++++++++++------------ changelog.d/10148.doc | 1 + 2 files changed, 25 insertions(+), 12 deletions(-) create mode 100644 changelog.d/10148.doc diff --git a/README.rst b/README.rst index a14a687fd1b4..11631816a6b2 100644 --- a/README.rst +++ b/README.rst @@ -293,18 +293,6 @@ try installing the failing modules individually:: pip install -e "module-name" -Once this is done, you may wish to run Synapse's unit tests to -check that everything is installed correctly:: - - python -m twisted.trial tests - -This should end with a 'PASSED' result (note that exact numbers will -differ):: - - Ran 1337 tests in 716.064s - - PASSED (skips=15, successes=1322) - We recommend using the demo which starts 3 federated instances running on ports `8080` - `8082` ./demo/start.sh @@ -324,6 +312,30 @@ If you just want to start a single instance of the app and run it directly:: python -m synapse.app.homeserver --config-path homeserver.yaml +Running the unit tests +====================== + +After getting up and running, you may wish to run Synapse's unit tests to +check that everything is installed correctly:: + + python -m twisted.trial tests + +This should end with a 'PASSED' result (note that exact numbers will +differ):: + + Ran 1337 tests in 716.064s + + PASSED (skips=15, successes=1322) + +To run a specific test:: + + python -m twisted.trial tests.storage.test_redaction + python -m twisted.trial tests.handlers.test_presence.PresenceJoinTestCase.test_remote_gets_presence_when_local_user_joins + +To see logger output from the app while the tests run, set ``SYNAPSE_TEST_LOG_LEVEL`` +and the resulting logs will be in ``_trial_temp/test.log``:: + + SYNAPSE_TEST_LOG_LEVEL=DEBUG python -m twisted.trial tests Running the Integration Tests diff --git a/changelog.d/10148.doc b/changelog.d/10148.doc new file mode 100644 index 000000000000..5066392d409a --- /dev/null +++ b/changelog.d/10148.doc @@ -0,0 +1 @@ +Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. From 2cb407762d4679cfcb739acf11f8feb99144adf8 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 9 Jun 2021 17:09:23 -0500 Subject: [PATCH 2/3] Move test tips to contributing doc --- CONTRIBUTING.md | 10 ++++++++++ README.rst | 11 ++--------- changelog.d/{10148.doc => 10148.misc} | 0 3 files changed, 12 insertions(+), 9 deletions(-) rename changelog.d/{10148.doc => 10148.misc} (100%) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b6a70f7ffed4..f479e11458ba 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -163,6 +163,8 @@ was broken. They are slower than the linters but will typically catch more error ```sh source ./env/bin/activate trial tests +# The tests can also be run with this longer command +python -m twisted.trial tests ``` If you wish to only run *some* unit tests, you may specify @@ -179,6 +181,14 @@ If your tests fail, you may wish to look at the logs: less _trial_temp/test.log ``` +To see always see logger output from the app while the tests run, +set `SYNAPSE_TEST_LOG_LEVEL` and the resulting logs will be in `_trial_temp/test.log`: + +```sh +SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests +``` + + ## Run the integration tests. The integration tests are a more comprehensive suite of tests. They diff --git a/README.rst b/README.rst index 11631816a6b2..29f4cc9281bf 100644 --- a/README.rst +++ b/README.rst @@ -327,15 +327,8 @@ differ):: PASSED (skips=15, successes=1322) -To run a specific test:: - - python -m twisted.trial tests.storage.test_redaction - python -m twisted.trial tests.handlers.test_presence.PresenceJoinTestCase.test_remote_gets_presence_when_local_user_joins - -To see logger output from the app while the tests run, set ``SYNAPSE_TEST_LOG_LEVEL`` -and the resulting logs will be in ``_trial_temp/test.log``:: - - SYNAPSE_TEST_LOG_LEVEL=DEBUG python -m twisted.trial tests +For more tips on running the unit tests, like running a specific test or +show the logger output of the app, see the `CONTRIBUTING doc `_. Running the Integration Tests diff --git a/changelog.d/10148.doc b/changelog.d/10148.misc similarity index 100% rename from changelog.d/10148.doc rename to changelog.d/10148.misc From 6d02df511bfefbb119aaee5e8d7287883e3c96cb Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Thu, 10 Jun 2021 20:05:18 -0500 Subject: [PATCH 3/3] Standardize test command and how SYNAPSE_TEST_LOG_LEVEL actually works --- CONTRIBUTING.md | 7 ++----- README.rst | 4 ++-- 2 files changed, 4 insertions(+), 7 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f479e11458ba..a4e6688042e3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -163,8 +163,6 @@ was broken. They are slower than the linters but will typically catch more error ```sh source ./env/bin/activate trial tests -# The tests can also be run with this longer command -python -m twisted.trial tests ``` If you wish to only run *some* unit tests, you may specify @@ -175,14 +173,13 @@ source ./env/bin/activate trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_invite ``` -If your tests fail, you may wish to look at the logs: +If your tests fail, you may wish to look at the logs (the default log level is `ERROR`): ```sh less _trial_temp/test.log ``` -To see always see logger output from the app while the tests run, -set `SYNAPSE_TEST_LOG_LEVEL` and the resulting logs will be in `_trial_temp/test.log`: +To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`: ```sh SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests diff --git a/README.rst b/README.rst index 29f4cc9281bf..1c9f05cc85cf 100644 --- a/README.rst +++ b/README.rst @@ -318,7 +318,7 @@ Running the unit tests After getting up and running, you may wish to run Synapse's unit tests to check that everything is installed correctly:: - python -m twisted.trial tests + trial tests This should end with a 'PASSED' result (note that exact numbers will differ):: @@ -328,7 +328,7 @@ differ):: PASSED (skips=15, successes=1322) For more tips on running the unit tests, like running a specific test or -show the logger output of the app, see the `CONTRIBUTING doc `_. +to see the logging output, see the `CONTRIBUTING doc `_. Running the Integration Tests