Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PYTHON-4731 - Explicitly close all MongoClients opened during tests #1855

Merged
merged 31 commits into from
Sep 17, 2024

Conversation

NoahStapp
Copy link
Contributor

No description provided.

@NoahStapp NoahStapp requested a review from blink1073 September 12, 2024 14:59
@NoahStapp NoahStapp marked this pull request as draft September 12, 2024 15:51
@NoahStapp NoahStapp marked this pull request as ready for review September 12, 2024 21:16
@blink1073
Copy link
Member

test_cleanup_executors_on_client_del is failing because we have created a reference to the client through self.addCleanup(client.close) in this PR. I think that test needs to be special cased and not use the builtin methods, but instead ignore the resource warning in the test.

@blink1073
Copy link
Member

The other half of the fix is to remove source=self from the warnings in __del__, which add another reference.

@NoahStapp
Copy link
Contributor Author

test_cleanup_executors_on_client_del is failing because we have created a reference to the client through self.addCleanup(client.close) in this PR. I think that test needs to be special cased and not use the builtin methods, but instead ignore the resource warning in the test.

Yup that did it, fantastic catch!

@@ -1185,7 +1185,7 @@ def __getitem__(self, name: str) -> database.AsyncDatabase[_DocumentType]:
def __del__(self) -> None:
"""Check that this AsyncMongoClient has been closed and issue a warning if not."""
try:
if not self._closed:
if self._opened and not self._closed:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should remove source=self here to prevent holding a strong reference to the client.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, unless we think it's useful to provide the user with the instance of the unclosed client?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the time __del__ is called, there's no guarantees about the state of the object.

@NoahStapp NoahStapp requested a review from blink1073 September 16, 2024 13:15
@blink1073
Copy link
Member

test.asynchronous.test_bulk.AsyncTestBulk.test_numerous_inserts needs updating.

@blink1073
Copy link
Member

blink1073 commented Sep 16, 2024

Also test.test_bulk.TestBulk.test_numerous_inserts and test.test_transactions.TestTransactionsConvenientAPI.test_callback_not_retried_after_commit_timeout

@NoahStapp
Copy link
Contributor Author

test.asynchronous.test_bulk.AsyncTestBulk.test_numerous_inserts needs updating.

Missed one of the branches in async_client_context.init_client, fixed.

@blink1073
Copy link
Member

I think you might need to pull from master again to pick up #1853

Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All green!

@NoahStapp NoahStapp merged commit 7395102 into mongodb:master Sep 17, 2024
57 of 58 checks passed
@NoahStapp NoahStapp deleted the PYTHON-4731 branch September 17, 2024 13:22
client_options["password"] = db_pwd
client = AsyncMongoClient(uri, port, **client_options)
if client._options.connect:
await client.aconnect()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what motivated calling connect() here? It makes it so that _async_mongo_client() behaves differently than AsyncMongoClient() which I worry could cause issues down the line.

Copy link
Contributor Author

@NoahStapp NoahStapp Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several places that use an _async_mongo_client() wrapper expect the client it returns to already be connected. In the interest of minimizing the amount of changes, I wanted to keep that expectation where it existed.

Notably, simple_client() directly wraps AsyncMongoClient() instead without any of the extra wrapping we do in _async_mongo_client(). I plan to write up a new section of CONTRIBUTING.md to give guidelines for writing tests and which helper methods to use in creating clients.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants