Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Windows CI. #569

Merged
merged 1 commit into from
Nov 10, 2023
Merged

Added Windows CI. #569

merged 1 commit into from
Nov 10, 2023

Conversation

dblock
Copy link
Member

@dblock dblock commented Nov 9, 2023

Description

Added Windows CI.

Fixed failing tests:

2023-11-09T22:09:24.7178027Z _________ TestTransport.test_failed_connection_will_be_marked_as_dead _________
2023-11-09T22:09:24.7178656Z 
2023-11-09T22:09:24.7179274Z self = <DummyConnection: http://localhost:9200>, args = ('GET', '/', None, None)
2023-11-09T22:09:24.7180399Z kwargs = {'headers': None, 'ignore': (), 'timeout': None}
2023-11-09T22:09:24.7180874Z 
2023-11-09T22:09:24.7181305Z     def perform_request(self, *args: Any, **kwargs: Any) -> Any:
2023-11-09T22:09:24.7181986Z         self.calls.append((args, kwargs))
2023-11-09T22:09:24.7182516Z         if self.exception:
2023-11-09T22:09:24.7182927Z >           raise self.exception
2023-11-09T22:09:24.7183252Z 
2023-11-09T22:09:24.7183475Z test_opensearchpy\test_transport.py:56: 
2023-11-09T22:09:24.7184204Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2023-11-09T22:09:24.7184976Z opensearchpy\transport.py:409: in perform_request
2023-11-09T22:09:24.7185728Z     status, headers_response, data = connection.perform_request(
2023-11-09T22:09:24.7186585Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2023-11-09T22:09:24.7187167Z 
2023-11-09T22:09:24.7187803Z self = <DummyConnection: http://localhost:9200>, args = ('GET', '/', None, None)
2023-11-09T22:09:24.7188542Z kwargs = {'headers': None, 'ignore': (), 'timeout': None}
2023-11-09T22:09:24.7188833Z 
2023-11-09T22:09:24.7189083Z     def perform_request(self, *args: Any, **kwargs: Any) -> Any:
2023-11-09T22:09:24.7189510Z         self.calls.append((args, kwargs))
2023-11-09T22:09:24.7189829Z         if self.exception:
2023-11-09T22:09:24.7190097Z >           raise self.exception
2023-11-09T22:09:24.7190739Z E           opensearchpy.exceptions.ConnectionError: <exception str() failed>
2023-11-09T22:09:24.7191126Z 

The constructor ConnectionError("abandon ship") had wrong arguments (it takes 3), so as written str(ConnectionError("abandon ship")) will fail. On *nix it seems to re-raise the object as an exception as is, but on Windows it re-raises str(error) and fails with exception str() failed.


2023-11-09T22:09:24.7192548Z self = <test_opensearchpy.test_transport.TestTransport testMethod=test_failed_connection_will_be_marked_as_dead>
2023-11-09T22:09:24.7193065Z 
2023-11-09T22:09:24.7193457Z     def test_failed_connection_will_be_marked_as_dead(self) -> None:
2023-11-09T22:09:24.7193870Z         t: Any = Transport(
2023-11-09T22:09:24.7194214Z             [{"exception": ConnectionError("abandon ship")}] * 2,
2023-11-09T22:09:24.7194656Z             connection_class=DummyConnection,
2023-11-09T22:09:24.7194962Z         )
2023-11-09T22:09:24.7195128Z     
2023-11-09T22:09:24.7195451Z >       self.assertRaises(ConnectionError, t.perform_request, "GET", "/")
2023-11-09T22:09:24.7195790Z 
2023-11-09T22:09:24.7195909Z test_opensearchpy\test_transport.py:282: 
2023-11-09T22:09:24.7196306Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2023-11-09T22:09:24.7196752Z opensearchpy\transport.py:439: in perform_request
2023-11-09T22:09:24.7197106Z     self.mark_dead(connection)
2023-11-09T22:09:24.7197399Z opensearchpy\transport.py:367: in mark_dead
2023-11-09T22:09:24.7197764Z     self.connection_pool.mark_dead(connection)
2023-11-09T22:09:24.7198151Z opensearchpy\connection_pool.py:198: in mark_dead
2023-11-09T22:09:24.7198520Z     self.dead.put((now + timeout, connection))
2023-11-09T22:09:24.7198993Z C:\hostedtoolcache\windows\Python\3.11.6\x64\Lib\queue.py:150: in put
2023-11-09T22:09:24.7199416Z     self._put(item)
2023-11-09T22:09:24.7199722Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2023-11-09T22:09:24.7200031Z 
2023-11-09T22:09:24.7200117Z     def _put(self, item):
2023-11-09T22:09:24.7200375Z >       heappush(self.queue, item)
2023-11-09T22:09:24.7200974Z E       TypeError: '<' not supported between instances of 'DummyConnection' and 'DummyConnection'

This DummyConnection object is being compared to another in a heap. Not sure why it only hits lt on Windows, but if these objects are to be compared, they should have lt implemented. Probably a real bug that only shows up on Windows.

2023-11-09T22:09:24.7217026Z _______________ TestAIOHttpConnection.test_uses_given_ca_certs ________________
2023-11-09T22:09:24.7217385Z 
2023-11-09T22:09:24.7217754Z self = <test_opensearchpy.test_async.test_connection.TestAIOHttpConnection object at 0x00000204C20F63D0>
2023-11-09T22:09:24.7218578Z load_verify_locations = <MagicMock name='load_verify_locations' id='2219462067920'>
2023-11-09T22:09:24.7219510Z tmp_path = WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-runneradmin/pytest-0/test_uses_given_ca_certs0')
2023-11-09T22:09:24.7220102Z 
2023-11-09T22:09:24.7220242Z     @patch("ssl.SSLContext.load_verify_locations")
2023-11-09T22:09:24.7220606Z     async def test_uses_given_ca_certs(
2023-11-09T22:09:24.7220966Z         self, load_verify_locations: Any, tmp_path: Any
2023-11-09T22:09:24.7221324Z     ) -> None:
2023-11-09T22:09:24.7221555Z         path = tmp_path / "ca_certs.pem"
2023-11-09T22:09:24.7221857Z         path.touch()
2023-11-09T22:09:24.7222157Z         AIOHttpConnection(use_ssl=True, ca_certs=str(path))
2023-11-09T22:09:24.7222638Z >       load_verify_locations.assert_called_once_with(cafile=str(path))
2023-11-09T22:09:24.7222966Z 
2023-11-09T22:09:24.7223124Z test_opensearchpy\test_async\test_connection.py:264: 
2023-11-09T22:09:24.7223577Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2023-11-09T22:09:24.7223876Z 
2023-11-09T22:09:24.7224151Z _mock_self = <MagicMock name='load_verify_locations' id='2219462067920'>
2023-11-09T22:09:24.7224559Z args = ()
2023-11-09T22:09:24.7225290Z kwargs = {'cafile': 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_uses_given_ca_certs0\\ca_certs.pem'}
2023-11-09T22:09:24.7226183Z self = <MagicMock name='load_verify_locations' id='2219462067920'>
2023-11-09T22:09:24.7227581Z msg = "Expected 'load_verify_locations' to be called once. Called 3 times.\nCalls: [call(cadata=bytearray(b'0\\x82\\x01\\xca...dmin\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-runneradmin\\\\pytest-0\\\\test_uses_given_ca_certs0\\\\ca_certs.pem')]."
2023-11-09T22:09:24.7228728Z 
2023-11-09T22:09:24.7228898Z     def assert_called_once_with(_mock_self, *args, **kwargs):
2023-11-09T22:09:24.7229410Z         """assert that the mock was called exactly once and that that call was
2023-11-09T22:09:24.7229865Z         with the specified arguments."""
2023-11-09T22:09:24.7230163Z         self = _mock_self
2023-11-09T22:09:24.7230414Z         if not self.call_count == 1:
2023-11-09T22:09:24.7230873Z             msg = ("Expected '%s' to be called once. Called %s times.%s"
2023-11-09T22:09:24.7231346Z                    % (self._mock_name or 'mock',
2023-11-09T22:09:24.7231687Z                       self.call_count,
2023-11-09T22:09:24.7231997Z                       self._calls_repr()))
2023-11-09T22:09:24.7232307Z >           raise AssertionError(msg)
2023-11-09T22:09:24.7232965Z E           AssertionError: Expected 'load_verify_locations' to be called once. Called 3 times.

Looks like the SSL context is being created somewhere else than in AIOHttpConnection as a side-effect, causing the number of calls to load_verify_locations to be different. To fix I used a mock SSL context instead of patching the global method, therefore only counting the calls we makle.

Issues Resolved

Closes #347.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Nov 9, 2023

Codecov Report

Merging #569 (66373f1) into main (dcb79cc) will increase coverage by 0.00%.
Report is 1 commits behind head on main.
The diff coverage is 75.00%.

@@           Coverage Diff           @@
##             main     #569   +/-   ##
=======================================
  Coverage   71.69%   71.69%           
=======================================
  Files          87       87           
  Lines        7875     7886   +11     
=======================================
+ Hits         5646     5654    +8     
- Misses       2229     2232    +3     
Files Coverage Δ
opensearchpy/__init__.py 100.00% <ø> (ø)
opensearchpy/_async/helpers/document.py 57.38% <100.00%> (ø)
opensearchpy/_async/helpers/index.py 60.80% <100.00%> (ø)
opensearchpy/_async/http_aiohttp.py 83.45% <100.00%> (+0.12%) ⬆️
opensearchpy/connection/async_connections.py 95.91% <100.00%> (+0.17%) ⬆️
opensearchpy/connection_pool.py 89.74% <100.00%> (ø)
opensearchpy/helpers/actions.py 44.22% <ø> (ø)
opensearchpy/helpers/asyncsigner.py 95.65% <100.00%> (ø)
opensearchpy/helpers/field.py 93.37% <100.00%> (ø)
opensearchpy/helpers/index.py 64.50% <100.00%> (ø)
... and 7 more

@dblock dblock force-pushed the windows-ci branch 4 times, most recently from 5dab57d to 14d166a Compare November 9, 2023 23:22
Signed-off-by: dblock <dblock@amazon.com>
@VachaShah VachaShah merged commit 58b83d8 into opensearch-project:main Nov 10, 2023
63 checks passed
@dblock dblock deleted the windows-ci branch November 10, 2023 13:58
roma2023 pushed a commit to roma2023/opensearch-py that referenced this pull request Dec 28, 2023
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: roma2023 <romasaparhan19@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Tests failing on Windows
2 participants