Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loadtest mocha-3 with random blobs #2197

Closed
evan-forbes opened this issue Jul 31, 2023 · 5 comments
Closed

Loadtest mocha-3 with random blobs #2197

evan-forbes opened this issue Jul 31, 2023 · 5 comments
Assignees
@staheri14
Labels
testing items that are strictly related to adding or extending test coverage

Comments

@evan-forbes
Copy link
Member

evan-forbes commented Jul 31, 2023 •
edited
Loading

We need to monitor mocha-3's performance with handling large amounts of blobs. This means that we should spam the mempool with blobs while monitoring various metrics.

  • We should expect to see no to minor increases in block times since it can take multiple seconds to gossip large blocks, but we should not expect to see that it takes multiple rounds to reach consensus. It would be nice if we could know exactly how much longer it is taking to gossip the block.
  • We should expect that bandwidth increases significantly, but it would be good to know exactly how much, and the exact distribution between the mempool and proposal gossiping.
  • We should expect that node's mempools will begin filtering out transactions with lower fees after hitting their local limit.
  • We should also look for memory leaks or other critical failures during this time.

edit: the two crossed issues will be tracked in a second set of issues dedicated to more formalized performance

it would be nice to combine this with celestiaorg/celestia-core#1054

as a bonus:

  • We should increase the block size to 8MB and perform the same tests
  • We should run the same tests with a large portion (if not all) of the validators running CAT
@evan-forbes evan-forbes added T:investigate testing items that are strictly related to adding or extending test coverage and removed needs:triage labels Jul 31, 2023
@evan-forbes evan-forbes added this to the Mainnet milestone Jul 31, 2023
@evan-forbes evan-forbes mentioned this issue Aug 7, 2023
Closed
This was referenced Aug 17, 2023
Merged
Merged
evan-forbes added a commit that referenced this issue Aug 17, 2023
@evan-forbes
feat: add the ability to specifiy the txsim master account (#2292)
2102189 
## Overview

This PR adds the ability to specify a specific account to use as master
for txsim. This is useful for when we don't want to always use the
account with the most funds in our wallet.

closes #2286
kinda sorta blocking #2197 

## Checklist

- [x] New and updated code has appropriate documentation
- [x] New and updated code has new and/or updated testing
- [x] Required CI checks are passing
- [x] Visual proof for any user facing features like CLI or
documentation updates
- [x] Linked issues closed with keywords
mergify bot pushed a commit that referenced this issue Aug 17, 2023
@evan-forbes @mergify
feat: add the ability to specifiy the txsim master account (#2292)
2d0b7d3 
## Overview

This PR adds the ability to specify a specific account to use as master
for txsim. This is useful for when we don't want to always use the
account with the most funds in our wallet.

closes #2286
kinda sorta blocking #2197

## Checklist

- [x] New and updated code has appropriate documentation
- [x] New and updated code has new and/or updated testing
- [x] Required CI checks are passing
- [x] Visual proof for any user facing features like CLI or
documentation updates
- [x] Linked issues closed with keywords

(cherry picked from commit 2102189)

# Conflicts:
#	test/e2e/simple_test.go
evan-forbes added a commit that referenced this issue Aug 17, 2023
@evan-forbes
feat!: add feegranter option to txsim (#2291)
33536b0 
## Overview

This PR adds feegrant back to txsim. Except this time instead of only
using feegrant, we now have feegrant as an option. This way we can test
without it enabled, which keeps our testing environments _slightly_ more
realistic while still being able to enable it when convient. This is
particularly useful when we don't want to send gobs of testnet tokens
around or have a very long running sequence.

closes #2289
blocking #2197
mergify bot pushed a commit that referenced this issue Aug 17, 2023
@evan-forbes @mergify
feat!: add feegranter option to txsim (#2291)
8c39588 
## Overview

This PR adds feegrant back to txsim. Except this time instead of only
using feegrant, we now have feegrant as an option. This way we can test
without it enabled, which keeps our testing environments _slightly_ more
realistic while still being able to enable it when convient. This is
particularly useful when we don't want to send gobs of testnet tokens
around or have a very long running sequence.

closes #2289
blocking #2197

(cherry picked from commit 33536b0)

# Conflicts:
#	test/e2e/simple_test.go
@staheri14
Copy link
Contributor

Created a Notion page exclusively aimed at compiling insights and discoveries pertaining to the performance evaluation of the network under a spamming attack. This encompasses the testing scenario, essential metrics, and performance indicators.

@evan-forbes evan-forbes changed the title Spam mocha-3 with random blobs Loadtest mocha-3 with random blobs Sep 5, 2023
@evan-forbes
Copy link
Member Author

Here is a preliminary analysis

time between cosecutive blocks and block size

@evan-forbes evan-forbes removed their assignment Sep 5, 2023
@staheri14 staheri14 mentioned this issue Sep 5, 2023
Closed
@staheri14
Copy link
Contributor

staheri14 commented Sep 13, 2023 •
edited
Loading

We should expect that bandwidth increases significantly, but it would be good to know exactly how much, and the exact distribution between the mempool and proposal gossiping.

Link to the notion page where the second item of this issue is covered (the Notion page is currently undergoing active updates with additional findings and results):

https://www.notion.so/celestiaorg/Analysis-of-Traffic-Distribution-0236546b09d247a99ec4e540da3bc012

@staheri14 staheri14 mentioned this issue Sep 14, 2023
Closed
4 tasks
@evan-forbes
Copy link
Member Author

I think we can close this as completed @staheri14 wdyt?

@staheri14
Copy link
Contributor

I think we can close this as completed @staheri14 wdyt?

Certainly, we can proceed with closing it, as we have already obtained a variety of insightful test results concerning the network's health and have addressed all the questions in the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@staheri14 staheri14

Labels
testing items that are strictly related to adding or extending test coverage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@staheri14 @evan-forbes