Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI Failure (Added node 5 disk usage 1055041199 is to large,) in ScalingUpTest.test_fast_node_addition #20224

Closed
vbotbuildovich opened this issue Jun 26, 2024 · 12 comments · Fixed by #22317
Closed

CI Failure (Added node 5 disk usage 1055041199 is to large,) in ScalingUpTest.test_fast_node_addition #20224

vbotbuildovich opened this issue Jun 26, 2024 · 12 comments · Fixed by #22317
Assignees
@mmaslankaprv
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Jun 26, 2024 •
edited by jira bot
Loading

https://buildkite.com/redpanda/vtools/builds/15023

Module: rptest.tests.scaling_up_test
Class: ScalingUpTest
Method: test_fast_node_addition

test_id:    ScalingUpTest.test_fast_node_addition
status:     FAIL
run time:   64.723 seconds

AssertionError('Added node 5 disk usage 1055041199 is to large,')
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 105, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/scaling_up_test.py", line 484, in test_fast_node_addition
    verify_disk_usage(usage,
  File "/home/ubuntu/redpanda/tests/rptest/tests/scaling_up_test.py", line 477, in verify_disk_usage
    assert added_node_usage < percentage * avg_usage, \
AssertionError: Added node 5 disk usage 1055041199 is to large,

JIRA Link: CORE-4506
@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Jun 26, 2024
@michael-redpanda
Copy link
Contributor

Fixed by: #20310

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51554
*https://buildkite.com/redpanda/redpanda/builds/51562
*https://buildkite.com/redpanda/redpanda/builds/51567

@mmaslankaprv mmaslankaprv self-assigned this Jul 17, 2024
@mmaslankaprv mmaslankaprv changed the title CI Failure (key symptom) in ScalingUpTest.test_fast_node_addition CI Failure (Added node 5 disk usage 1055041199 is to large,) in ScalingUpTest.test_fast_node_addition Jul 17, 2024
@mmaslankaprv mmaslankaprv mentioned this issue Jul 17, 2024
Merged
7 tasks
@andrwng andrwng mentioned this issue Jul 17, 2024
Merged
7 tasks
@ztlpn ztlpn mentioned this issue Jul 17, 2024
Merged
7 tasks
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51692
*https://buildkite.com/redpanda/redpanda/builds/51702

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51771
*https://buildkite.com/redpanda/redpanda/builds/51769

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51811
*https://buildkite.com/redpanda/redpanda/builds/51811

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51830

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51861
*https://buildkite.com/redpanda/redpanda/builds/51865

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51905

@ztlpn ztlpn mentioned this issue Jul 24, 2024
Merged
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51948
*https://buildkite.com/redpanda/redpanda/builds/51969

@mmaslankaprv mmaslankaprv mentioned this issue Jul 25, 2024
Merged
7 tasks
vbotbuildovich pushed a commit to vbotbuildovich/redpanda that referenced this issue Jul 25, 2024
@mmaslankaprv @vbotbuildovich
tests: increased tolerance in fast addition tests
1b13973 
In order to make the test stable increased a tolerance on the size of
data on a node that joins the cluster. The slight increase doesn't
influence the test correctness but makes it less vulnerable to small
changes in size of partition metadata.

Fixes: redpanda-data#20224

Signed-off-by: Michał Maślanka <michal@redpanda.com>
(cherry picked from commit 7674dc0)
@vbotbuildovich vbotbuildovich mentioned this issue Jul 25, 2024
Closed
vbotbuildovich pushed a commit to vbotbuildovich/redpanda that referenced this issue Jul 25, 2024
@mmaslankaprv @vbotbuildovich
tests: increased tolerance in fast addition tests
deae122 
In order to make the test stable increased a tolerance on the size of
data on a node that joins the cluster. The slight increase doesn't
influence the test correctness but makes it less vulnerable to small
changes in size of partition metadata.

Fixes: redpanda-data#20224

Signed-off-by: Michał Maślanka <michal@redpanda.com>
(cherry picked from commit 7674dc0)
@vbotbuildovich vbotbuildovich mentioned this issue Jul 25, 2024
Closed
@pgellert pgellert mentioned this issue Jul 26, 2024
Merged
7 tasks
@pgellert
Copy link
Contributor

I saw this failure again on this PR:

@pgellert pgellert reopened this Jul 26, 2024
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/vtools/builds/16118

@piyushredpanda
Copy link
Contributor

Closing older-bot-filed CI issues as we transition to a more reliable system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmaslankaprv mmaslankaprv

Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Increased tolerance in fast addition tests mmaslankaprv/redpanda
5 participants
@pgellert @mmaslankaprv @vbotbuildovich @piyushredpanda @michael-redpanda