fix(httphandler): req should return as soon as results is exhausted. #5462

youngsofun · 2022-05-19T08:35:43Z

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Summary about this PR

carry bool done aside block when pop()
notify when stop_push()
not return data and next_url when error.

Changelog

Bug Fix

Related Issues

Fixes #5456

vercel · 2022-05-19T08:35:47Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		May 19, 2022 at 3:49PM (UTC)

mergify · 2022-05-19T08:36:16Z

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

BohuTANG · 2022-05-19T10:42:58Z

failures:
servers::http::http_query_handlers::test_pagination always failed.

https://github.com/datafuselabs/databend/runs/6505375747?check_suite_focus=true#step:3:3320

youngsofun · 2022-05-19T11:43:23Z

@BohuTANG should have been fixed. took a while to debug.

youngsofun · 2022-05-19T12:43:06Z

@TCeason need your help

02_0000_kill_query:                                                     2022-05-19T12:23:26.361052Z ERROR databend_query::pipelines::processors::processor_merge: Merge processor cannot push data: channel closed
[ FAIL ] - return code 1
, result:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 178, in read
    data += self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.8/queue.py", line 178, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 152, in expect
    data = self.read(timeout=timeleft, raise_exception=True)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 186, in read
    raise TimeoutError(timeout)
uexpect.TimeoutError: Timeout 60.000s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/02_0000_kill_query.py", line 25, in <module>
    with NativeClient(name='client2>') as client2:
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/../../../helpers/native_client.py", line 38, in __init__
    self.client.expect('[#\$] ', timeout=60)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 159, in expect
    raise exception
uexpect.ExpectTimeoutError: Timeout 60.000s for '[#\\$] '

TCeason · 2022-05-19T12:50:21Z

@TCeason need your help

02_0000_kill_query:                                                     2022-05-19T12:23:26.361052Z ERROR databend_query::pipelines::processors::processor_merge: Merge processor cannot push data: channel closed
[ FAIL ] - return code 1
, result:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 178, in read
    data += self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.8/queue.py", line 178, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 152, in expect
    data = self.read(timeout=timeleft, raise_exception=True)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 186, in read
    raise TimeoutError(timeout)
uexpect.TimeoutError: Timeout 60.000s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/02_0000_kill_query.py", line 25, in <module>
    with NativeClient(name='client2>') as client2:
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/../../../helpers/native_client.py", line 38, in __init__
    self.client.expect('[#\$] ', timeout=60)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 159, in expect
    raise exception
uexpect.ExpectTimeoutError: Timeout 60.000s for '[#\\$] '

Yes that a flaky test, needs a rerun.

BohuTANG · 2022-05-19T12:59:16Z

@TCeason need your help

02_0000_kill_query:                                                     2022-05-19T12:23:26.361052Z ERROR databend_query::pipelines::processors::processor_merge: Merge processor cannot push data: channel closed
[ FAIL ] - return code 1
, result:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 178, in read
    data += self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.8/queue.py", line 178, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 152, in expect
    data = self.read(timeout=timeleft, raise_exception=True)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 186, in read
    raise TimeoutError(timeout)
uexpect.TimeoutError: Timeout 60.000s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/02_0000_kill_query.py", line 25, in <module>
    with NativeClient(name='client2>') as client2:
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/../../../helpers/native_client.py", line 38, in __init__
    self.client.expect('[#\$] ', timeout=60)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 159, in expect
    raise exception
uexpect.ExpectTimeoutError: Timeout 60.000s for '[#\\$] '

Yes that a flaky test, needs a rerun.

Re-run can't save it. It seems to fail every time, needs improvement, or skip this test.

youngsofun · 2022-05-19T13:05:44Z

a unit test can not stop, I am working on it.

TCeason · 2022-05-19T13:10:16Z

@TCeason need your help

02_0000_kill_query:                                                     2022-05-19T12:23:26.361052Z ERROR databend_query::pipelines::processors::processor_merge: Merge processor cannot push data: channel closed
[ FAIL ] - return code 1
, result:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 178, in read
    data += self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.8/queue.py", line 178, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 152, in expect
    data = self.read(timeout=timeleft, raise_exception=True)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 186, in read
    raise TimeoutError(timeout)
uexpect.TimeoutError: Timeout 60.000s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/02_0000_kill_query.py", line 25, in <module>
    with NativeClient(name='client2>') as client2:
  File "/home/runner/work/databend/databend/tests/suites/1_stateful/02_query/../../../helpers/native_client.py", line 38, in __init__
    self.client.expect('[#\$] ', timeout=60)
  File "/home/runner/work/databend/databend/tests/helpers/uexpect.py", line 159, in expect
    raise exception
uexpect.ExpectTimeoutError: Timeout 60.000s for '[#\\$] '

Yes that a flaky test, needs a rerun.

Re-run can't save it. It seems to fail every time, needs improvement, or skip this test.

Got and doing

TCeason · 2022-05-19T13:14:35Z

And I think you can skip this test tmp. @youngsofun

remove file : 02_0000_kill_query.result in your pr.

youngsofun · 2022-05-19T15:04:55Z

And I think you can skip this test tmp. @youngsofun

remove file : 02_0000_kill_query.result in your pr.

@TCeason
removed

youngsofun · 2022-05-19T15:07:28Z

@junnplus rebased to rm 02_0000_kill_query.result

youngsofun · 2022-05-19T16:34:08Z

servers::http::http_query_handlers::test_http_session seems like a flaky test, can not reproduce on my mac
. seen for the first time and passed after rerun, maybe we can merge for now. I will try to improve it later，others can skip it if see it again

youngsofun requested a review from BohuTANG as a code owner May 19, 2022 08:35

databend-bot added the need-review label May 19, 2022

mergify bot added the pr-bugfix this PR patches a bug in codebase label May 19, 2022

youngsofun requested a review from junnplus May 19, 2022 08:37

BohuTANG removed their request for review May 19, 2022 08:39

youngsofun force-pushed the http branch from 3e41d62 to 23b986a Compare May 19, 2022 08:40

junnplus approved these changes May 19, 2022

View reviewed changes

youngsofun force-pushed the http branch from 30f9915 to 0e0cf0a Compare May 19, 2022 15:01

youngsofun added 5 commits May 19, 2022 23:49

fix(httphandler): req should return as soon as results is exhausted.

1f25995

test(httphandler): req should return as soon as result is exhausted.

6226d0b

polish(httphanlder): not return data and next_url when error.

46f41f8

test(query): rm flaky test.

2e3a8ec

test(httphandler): test_download show response when assert fail.

4c2c871

youngsofun force-pushed the http branch from 314c685 to 4c2c871 Compare May 19, 2022 15:49

youngsofun mentioned this pull request May 19, 2022

test_http_session failed once, may be flaky #5477

Closed

BohuTANG merged commit 48bb67d into databendlabs:main May 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(httphandler): req should return as soon as results is exhausted. #5462

fix(httphandler): req should return as soon as results is exhausted. #5462

youngsofun commented May 19, 2022 •

edited

Loading

vercel bot commented May 19, 2022 •

edited

Loading

mergify bot commented May 19, 2022

BohuTANG commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022 •

edited

Loading

TCeason commented May 19, 2022

BohuTANG commented May 19, 2022

youngsofun commented May 19, 2022 •

edited

Loading

TCeason commented May 19, 2022

TCeason commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022 •

edited

Loading

fix(httphandler): req should return as soon as results is exhausted. #5462

fix(httphandler): req should return as soon as results is exhausted. #5462

Conversation

youngsofun commented May 19, 2022 • edited Loading

Summary

Changelog

Related Issues

vercel bot commented May 19, 2022 • edited Loading

mergify bot commented May 19, 2022

BohuTANG commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022 • edited Loading

TCeason commented May 19, 2022

BohuTANG commented May 19, 2022

youngsofun commented May 19, 2022 • edited Loading

TCeason commented May 19, 2022

TCeason commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022

youngsofun commented May 19, 2022 • edited Loading

youngsofun commented May 19, 2022 •

edited

Loading

vercel bot commented May 19, 2022 •

edited

Loading

youngsofun commented May 19, 2022 •

edited

Loading

youngsofun commented May 19, 2022 •

edited

Loading

youngsofun commented May 19, 2022 •

edited

Loading