Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_spec_example: send_to_address_and_wait(address, 400.0) hangs test #3666

Open
moodyjon opened this issue Sep 15, 2022 · 2 comments
Open

test_spec_example: send_to_address_and_wait(address, 400.0) hangs test #3666

moodyjon opened this issue Sep 15, 2022 · 2 comments
Labels
area: hub area: tests

Comments

@moodyjon
Copy link
Contributor

Similar to lbryio/hub#98, but no reconnect is involved. Extra hub tracing for notifications in moodyjon/hub was NOT enabled, so we don't see much detail.

https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405

3514
test_spec_example (integration.takeovers.test_resolve_command.ResolveClaimTakeovers) ... 2022-09-15 18:56:41,767 - asyncio - WARNING - Executing <Task pending coro=<CommandTestCase.asyncSetUp() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py:415> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/futures.py:351, <TaskWakeupMethWrapper object at 0x7f654c4b6910>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[_run_until_complete_cb() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:157] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:566> took 1.012 seconds
[3515](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3516)
2022-09-15 18:56:43,748 - elasticsearch - WARNING - GET http://localhost:9200/_template/9c608281e815443fb014651050bc2a4dclaims [status:404 request:0.002s]
[3516](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3517)
2022-09-15 18:56:45,324 - lbry.wallet.network - WARNING - wait reconnect+keepalive
[3517](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3518)
2022-09-15 18:56:45,419 - lbry.wallet.ledger - WARNING - initial status update address mqvGxpeAsSusqCGMe3jQRx6Wwcgq6BKaVY status None
[3518](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3519)
2022-09-15 18:56:45,419 - lbry.wallet.ledger - WARNING - initial status update address n2TPX6skMYygny9ivjqFF7QVGcS94GU8Xr status None
[3519](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3520)
2022-09-15 18:56:45,419 - lbry.wallet.ledger - WARNING - initial status update address mpmwDXYVYRNYzaRpUZM3rK2CD76icHdiGb status None
[3520](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3521)
2022-09-15 18:56:45,420 - lbry.wallet.ledger - WARNING - initial status update address mmyksKrFhekEQZa4LNPaFAAJKJ1WNM9nZH status None
[3521](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3522)
2022-09-15 18:56:45,420 - lbry.wallet.ledger - WARNING - initial status update address mmtZg7ni6RAKNqbnBcusPRrzFwnNZ5fp4Y status None
[3522](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3523)
2022-09-15 18:56:45,420 - lbry.wallet.ledger - WARNING - initial status update address moGtYHjqQJ3nHhMtJ6x2eiKwsENut92yV8 status None
[3523](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3524)
2022-09-15 18:56:45,421 - lbry.wallet.ledger - WARNING - initial status update address mq38kPcMgw3uGVeBfYCZSbA5xDBEp4LyD1 status None
[3524](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3525)
2022-09-15 18:56:45,421 - lbry.wallet.ledger - WARNING - initial status update address mhC5dDJmskpdoTYBRcrHZENTCQszuzgc7Y status None
[3525](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3526)
2022-09-15 18:56:45,421 - lbry.wallet.ledger - WARNING - initial status update address mwqB8bKUwcsjXTxmpEMk4f675C7J6F2JvL status None
[3526](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3527)
2022-09-15 18:56:45,422 - lbry.wallet.ledger - WARNING - initial status update address n39GG9ej6emTvY5AmdNxvXxBh1UR1NSvCm status None
[3527](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3528)
2022-09-15 18:56:45,422 - lbry.wallet.ledger - WARNING - initial status update address msaUzYxnQcsB3LzTJYFRDciY7JKo1R21iv status None
[3528](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3529)
2022-09-15 18:56:45,422 - lbry.wallet.ledger - WARNING - initial status update address mi2CJScakJVpS7d4AJt7z8r3V5HxKN2A2x status None
[3529](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3530)
2022-09-15 18:56:45,423 - lbry.wallet.ledger - WARNING - initial status update address mxsfuRsgXRZQCby7rR6AKsvB8JPJXPLrW8 status None
[3530](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3531)
2022-09-15 18:56:45,423 - lbry.wallet.ledger - WARNING - initial status update address mp6xvdVZdERXaticupeVvzqTxZWP6rr8rN status None
[3531](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3532)
2022-09-15 18:56:45,423 - lbry.wallet.ledger - WARNING - initial status update address myVY7t9RNWbYtNnHXVqFZjZSAycJXs4W9w status None
[3532](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3533)
2022-09-15 18:56:45,424 - lbry.wallet.ledger - WARNING - initial status update address mshLiHt2WLGyTBHPwyMq6RELLPm31ouFuF status None
[3533](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3534)
2022-09-15 18:56:45,424 - lbry.wallet.ledger - WARNING - initial status update address mzy1xwMiP82LLSoGzfSmmbaqAUk4EMZzAV status None
[3534](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3535)
2022-09-15 18:56:45,424 - lbry.wallet.ledger - WARNING - initial status update address n3WLFA5kwyVE9gYUVXwCbyRQq4BWJPbKfA status None
[3535](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3536)
2022-09-15 18:56:45,425 - lbry.wallet.ledger - WARNING - initial status update address mx3jGTAf6ngYLdTMWTF1bwmhdhnqiW9rZD status None
[3536](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3537)
2022-09-15 18:56:45,425 - lbry.wallet.ledger - WARNING - initial status update address mqvEwJxEdXDB4LGXhF4CQwT6ZkQfmvFcnW status None
[3537](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3538)
2022-09-15 18:56:45,480 - lbry.wallet.ledger - WARNING - initial status update address miWuDnqCRa6T3Xkpwy1GeNtcWxwYwtcvsD status None
[3538](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3539)
2022-09-15 18:56:45,480 - lbry.wallet.ledger - WARNING - initial status update address n3ZWyoczQNDnGfzk8Z2jAuf9uVrM79jvs5 status None
[3539](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3540)
2022-09-15 18:56:45,480 - lbry.wallet.ledger - WARNING - initial status update address n2gBXorN4AK5tE3HAxxfJmzGQ4Ms9YimE1 status None
[3540](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3541)
2022-09-15 18:56:45,481 - lbry.wallet.ledger - WARNING - initial status update address miEPidCpivRQ5W7mSbHXh2YZxeEhD5C36D status None
[3541](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3542)
2022-09-15 18:56:45,481 - lbry.wallet.ledger - WARNING - initial status update address mkJwZSXw9VqjH1fNVyp2JqoP3nFpjMAugE status None
[3542](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3543)
2022-09-15 18:56:45,481 - lbry.wallet.ledger - WARNING - initial status update address n1Cesg9NRCDZ8ipAMQuzW9ojYeHeCsEQ4o status None
[3543](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3544)
2022-09-15 18:56:45,707 - lbry.testcase - WARNING - before: done = False, tx_watch = []
[3544](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3545)
2022-09-15 18:56:45,757 - lbry.wallet.ledger - WARNING - status update address mqvGxpeAsSusqCGMe3jQRx6Wwcgq6BKaVY status 30e663346d037f2ceb2e50ccfa6d1975fe625841941e2575408aab746e0a0a99
[3545](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3546)
2022-09-15 18:56:45,774 - lbry.testcase - WARNING - after: done = False, tx_watch = []
[3546](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3547)
2022-09-15 18:56:45,782 - lbry.wallet.ledger - WARNING - updated address: mqvGxpeAsSusqCGMe3jQRx6Wwcgq6BKaVY history: b0f187bf634d0b7dd8f2beb85aff1188313f229bc5ceddac569281e332507c98:0:
[3547](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3548)
2022-09-15 18:56:45,814 - lbry.wallet.ledger - WARNING - initial status update address moCd6v5VZT3HzpbeQ2X61epmk3EkR51GYB status None
[3548](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3549)
2022-09-15 18:56:45,960 - lbry.wallet.ledger - WARNING - status update address mqvGxpeAsSusqCGMe3jQRx6Wwcgq6BKaVY status 6d886384f7604f1a6d325e8febb3b047a180d9995db5491ca80a19ae100374ea
[3549](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3550)
2022-09-15 18:56:45,990 - lbry.wallet.ledger - WARNING - updated address: mqvGxpeAsSusqCGMe3jQRx6Wwcgq6BKaVY history: b0f187bf634d0b7dd8f2beb85aff1188313f229bc5ceddac569281e332507c98:201:
[3550](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3551)
2022-09-15 18:56:53,322 - lbry.testcase - WARNING - before: done = False, tx_watch = []
[3551](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3552)
Stack for <Task pending coro=<Event.wait() running at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py:293> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c64b9d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[_wait.<locals>._on_completion() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:466] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:387> (most recent call last):
[3552](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3553)
Stack for <Task pending coro=<ConnectionManager._update() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/connection_manager.py:70> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654ca8a2d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/lbry/connection_manager.py:105> (most recent call last):
[3553](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3554)
Stack for <Task pending coro=<DiskSpaceManager.cleaning_loop() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/blob/disk_space_manager.py:66> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c5f2310>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[DiskSpaceManager.start.<locals>.<lambda>() at /home/runner/work/lbry-sdk/lbry-sdk/lbry/blob/disk_space_manager.py:72] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:351> (most recent call last):
[3554](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3555)
Stack for <Task pending coro=<BlobServer.start_server.<locals>._start_server() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/blob_exchange/server.py:186> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c4e3510>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/lbry/blob_exchange/server.py:188> (most recent call last):
[3555](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3556)
Stack for <Task pending coro=<SessionManager.serve() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py:599> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c5e1e90>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3556](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3557)
Stack for <Task pending coro=<ElasticNotifierClientProtocol.maintain_connection() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/notifier_protocol.py:54> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654cff1650>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3557](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3558)
Stack for <Task pending coro=<SessionManager._clear_stale_sessions() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py:358> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c418c10>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[_wait.<locals>._on_completion() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:466] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:387> (most recent call last):
[3558](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3559)
Stack for <Task pending coro=<ClientSession.keepalive_loop() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py:127> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c548890>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[_wait.<locals>._on_completion() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:466] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:351> (most recent call last):
[3559](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3560)
Stack for <Task pending coro=<ElasticSyncService.run_es_notifier() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/elastic_sync/service.py:65> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c92a550>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3560](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3561)
Stack for <Task pending coro=<HubMemPool.send_notifications_forever() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/mempool.py:236> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c4895d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3561](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3562)
Stack for <Task pending coro=<ReflectorServer.start_server.<locals>._start_server() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/stream/reflector/server.py:189> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654caf6850>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/lbry/stream/reflector/server.py:193> (most recent call last):
[3562](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3563)
Stack for <Task pending coro=<SessionManager._manage_servers() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py:304> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c56a0d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[_wait.<locals>._on_completion() at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:466] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:387> (most recent call last):
[3563](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3564)
Stack for <Task pending coro=<AnalyticsManager.run() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/extras/daemon/analytics.py:138> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c498090>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:351> (most recent call last):
[3564](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3565)
Stack for <Task pending coro=<BlockchainProcessorService.process_blocks_and_mempool_forever() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/scribe/service.py:1781> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c5d79d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3565](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3566)
Stack for <Task pending coro=<BlockchainReaderService.refresh_blocks_forever() running at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:261> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654f9e5ed0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py:58> (most recent call last):
[3566](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3567)
Stack for <Task pending coro=<Network.network_loop() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py:351> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c7093d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> cb=[Network.start.<locals>.loop_task_done_callback() at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py:222] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/tasks.py:351> (most recent call last):
[3567](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3568)
Stack for <Task pending coro=<RPCSession._receive_messages() running at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/rpc/session.py:406> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f654c6e70d0>()] created at /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py:395> created at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/rpc/session.py:181> (most recent call last):  File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py", line 293, in wait
[3568](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3569)
    await fut
[3569](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3570)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/connection_manager.py", line 70, in _update
[3570](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3571)
    await asyncio.sleep(0.1, loop=self.loop)
[3571](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3572)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/blob/disk_space_manager.py", line 66, in cleaning_loop
[3572](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3573)
    await asyncio.sleep(self.cleaning_interval)
[3573](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3574)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/blob_exchange/server.py", line 186, in _start_server
[3574](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3575)
    await server.serve_forever()
[3575](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3576)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py", line 599, in serve
[3576](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3577)
    self._manage_servers()
[3577](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3578)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/notifier_protocol.py", line 54, in maintain_connection
[3578](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3579)
    await self._lost_connection.wait()
[3579](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3580)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py", line 358, in _clear_stale_sessions
[3580](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3581)
    await sleep(session_timeout // 10)
[3581](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3582)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py", line 127, in keepalive_loop
[3582](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3583)
    await asyncio.sleep(max(0, max_idle - (now - self.last_send)))
[3583](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3584)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/elastic_sync/service.py", line 65, in run_es_notifier
[3584](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3585)
    await server.serve_forever()
[3585](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3586)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/mempool.py", line 236, in send_notifications_forever
[3586](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3587)
    (session_id, height_changed, hashXes) = await self._notification_q.get()
[3587](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3588)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/stream/reflector/server.py", line 189, in _start_server
[3588](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3589)
    await server.serve_forever()
[3589](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3590)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py", line 304, in _manage_servers
[3590](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3591)
    await self.session_event.wait()
[3591](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3592)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/extras/daemon/analytics.py", line 138, in run
[3592](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3593)
    await asyncio.sleep(1800)
[3593](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3594)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/scribe/service.py", line 1781, in process_blocks_and_mempool_forever
[3594](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3595)
    await asyncio.wait_for(self.blocks_event.wait(), self.wait_for_blocks_duration)
[3595](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3596)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py", line 261, in refresh_blocks_forever
[3596](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3597)
    await asyncio.sleep(self._refresh_interval)
[3597](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3598)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py", line 351, in network_loop
[3598](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3599)
    return_when=asyncio.FIRST_COMPLETED
[3599](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3600)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/rpc/session.py", line 406, in _receive_messages
[3600](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3601)
    message = await self.framer.receive_message()
[3601](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3602)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/service.py", line 261, in refresh_blocks_forever
[3602](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3603)
    await asyncio.sleep(self._refresh_interval)
[3603](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3604)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/service.py", line 110, in receive_es_notifications
[3604](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3605)
    self._es_height, self._es_block_hash = await self.es_notifications.get()
[3605](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3606)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/stream/stream_manager.py", line 167, in reflect_streams
[3606](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3607)
    return await self._reflect_streams()
[3607](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3608)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/extras/daemon/components.py", line 744, in announce_forever
[3608](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3609)
    await asyncio.sleep(sleep_seconds)
[3609](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3610)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/scribe/prefetcher.py", line 47, in main_loop
[3610](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3611)
    await asyncio.sleep(self.polling_delay)
[3611](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3612)
  File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/takeovers/lib/python3.7/site-packages/hub/herald/session.py", line 1033, in _receive_messages
[3612](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3613)
    message = await self.framer.receive_message()
[3613](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3614)
  File "/home/runner/work/lbry-sdk/lbry-sdk/tests/integration/takeovers/test_resolve_command.py", line 1067, in test_spec_example
[3614](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3615)
    await self.send_to_address_and_wait(address, 400.0)
[3615](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3616)
  File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py", line 293, in wait
[3616](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3617)
    await fut
[3617](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:3618)
ERROR

======================================================================
[5210](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5211)
ERROR: test_spec_example (integration.takeovers.test_resolve_command.ResolveClaimTakeovers)
[5211](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5212)
----------------------------------------------------------------------
[5212](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5213)
Traceback (most recent call last):
[5213](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5214)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 145, in run
[5214](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5215)
    self.loop.run_until_complete(maybe_coroutine)
[5215](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5216)
  File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
[5216](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5217)
    return future.result()
[5217](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5218)
concurrent.futures._base.CancelledError
[5218](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5219)

[5219](https://github.com/moodyjon/lbry-sdk/actions/runs/3062810249/jobs/4944196405#step:11:5220)
----------------------------------------------------------------------

Test code:

        name = 'test'
        await self.generate(494)
        address = (await self.account.receiving.get_addresses(True))[0]
        await self.send_to_address_and_wait(address, 400.0).     <<<<<<<<<<< HANG here
        await self.account.ledger.on_address.first
@moodyjon
Copy link
Contributor Author

Likely fixed by lbryio/hub#99

Will monitor for failures after hub version bump and then close.

@moodyjon
Copy link
Contributor Author

Seen again AFTER all the fixes in #3707:
https://github.com/moodyjon/lbry-sdk/actions/runs/3515119954/jobs/5889980919

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area: hub area: tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@moodyjon