Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update readme #33

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 46 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,56 @@
# Performance Analysis of 0G, Evmos, Kava, Bera, and Sei

| Chain | Simple | ERC20 | Uniswap |
|---------|--------|-------|---------|
| 0G | 769 | 369 | 164 |
| Bera | 910 | 638 | 224 |
| Evmos | 790 | 859 | 689 |
| Kava | 637 | 84 | 36 |
| Sei | 784 | 784 | 392 |
| **Chain** | **Simple TPS** | **ERC20 TPS** | **Uniswap TPS** |
|-----------|----------------|---------------|-----------------|
| **0G** | 769 | 369 | 164 |
| **Bera** | 910 | 638 | 224 |
| **Evmos** | 790 | 859 | 689 |
| **Kava** | 637 | 84 | 36 |
| **Sei** | 784 | 784 | 392 |

### Key Clarifications

1. **TPS Measurements**
The numbers shown in the chart represent **transactions per second (TPS)** achieved during our benchmarking. These results are based on specific configurations we tested and optimized to ensure fairness. While we aimed to make the comparison competitive by adjusting configurations for each chain, we acknowledge that some configurations may not be fully optimal, as optimizing every chain requires significant time investment.

2. **Benchmarking Methodology**
This repository enables **anyone** to benchmark these chains using the same testing suite we used, ensuring **truthful and reproducible results**. For added convenience:

- We used **GitHub Actions** to automate the benchmarking process.
- **Anyone can fork the repository** and click **Run** in the **GitHub Actions** tab to execute the tests and view the results.
- Alternatively, you can directly view **historical results** in our GitHub Actions records, which include the complete testing steps and detailed logs.

This approach ensures transparency and provides a compelling demonstration for both technical and non-technical users.

---

## Key Observations and Insights

### 1. Cosmos+Ethermint vs. Cosmos+Beacon API+Geth/Reth
- **0G, Evmos, and Kava** use Cosmos+Ethermint, where each Ethereum transaction is wrapped into a Cosmos transaction for consensus processing. This introduces additional overhead compared to directly processing Ethereum transactions in an EVM.
- **Bera** adopts Cosmos+Beacon API+Geth/Reth, wrapping an entire Ethereum block payload into a single Cosmos transaction. This significantly reduces the transaction load on the consensus layer, resulting in better performance across all test categories.
### 1. Cosmos+Ethermint vs. Cosmos+Beacon API+Geth/Reth
- **0G, Evmos, and Kava** use Cosmos+Ethermint, where each Ethereum transaction is wrapped into a Cosmos transaction for consensus processing. This approach introduces additional consensus overhead, resulting in slightly lower TPS compared to Bera.
- **Bera** leverages a **Cosmos+Beacon API+Geth/Reth** design, which wraps entire Ethereum block payloads into single Cosmos transactions. This block-level payload processing significantly reduces consensus overhead, yielding consistently higher TPS across all categories.

### 2. Why Include Sei?
While Sei's extensive modifications to Cosmos, Tendermint, and Go-Ethereum make its architecture fundamentally different, we included Sei here for completeness. However, given Sei’s deviation from the standard Cosmos framework, direct comparisons may not fully reflect differences in architectural efficiency.

### 2. Sei's Unique Modifications
- **Sei** has extensively modified Cosmos, Tendermint, and Go-Ethereum. These deep changes make it fundamentally different from standard Cosmos chains like 0G, Evmos, and Kava. As such, Sei's performance cannot be directly compared to other chains in this analysis.
### 3. Evmos’ Higher TPS and Block Configuration Tradeoffs
Evmos achieves higher ERC20 and Uniswap TPS primarily due to its **larger block size** configuration, rather than inherent performance optimizations.
- **Tradeoffs of larger block size**:
- **Finality**: Larger blocks increase finality time, delaying confirmation of transactions.
- **Latency**: They can introduce higher network latency due to increased transmission times.
- **Network Congestion**: Larger blocks may cause higher peak loads, leading to temporary congestion under high transaction volume.
- **Gas Fees**: Larger blocks can reduce gas fees during normal conditions but may increase volatility during congestion.

### 3. Block Production in Cosmos+Ethermint
- Ethermint-based chains produce blocks based on Ethereum transactions' **gas limits** rather than **gas used**, as the Cosmos consensus layer cannot calculate gas used during block production. To prevent misuse of inflated gas limits, a **minimum gas usage ratio** (typically 50%, adjustable via consensus) is enforced.
- **Evmos’ higher TPS** in ERC20 and Uniswap tests is primarily due to its larger block size configuration, not inherent performance optimizations.
### 4. Performance Gap Between 0G and Kava
Despite similar block size configurations, **0G outperforms Kava** due to its enhanced `estimateGas` method. This improvement allows for more precise gas limit calculations, reducing unnecessary overhead and boosting overall transaction throughput.

### 4. Performance Gap Between 0G and Kava
- **0G and Kava** share similar block size configurations, but **0G achieves better TPS** due to an improved `estimateGas` method. This enhancement allows for more accurate gas limit calculations, improving transaction processing efficiency.
### 5. Bera’s Block-Level Payload Advantage
Bera’s architecture avoids the transaction-by-transaction consensus overhead of Ethermint chains by processing entire blocks as single payloads. This design gives Bera a clear performance edge in all test categories.

### 5. Bera's Design Advantage
- Bera's use of block-level payload processing avoids the overhead of transaction-by-transaction consensus inherent in Ethermint chains. This architectural decision gives Bera a significant performance edge in all test scenarios.
---

## Conclusion
The performance differences highlight the impact of architectural and implementation choices:
- **Bera** excels due to its block-level payload processing approach, which reduces consensus overhead.
- **0G** demonstrates the advantages of refining critical components like `estimateGas` to improve transaction throughput.
- **Evmos** achieves high ERC20 and Uniswap TPS through increased block size, while **Kava** lags due to older CometBFT and less efficient gas estimation.
- **Sei’s extensive customizations** set it apart from other chains, making direct comparisons to standard Cosmos-based architectures inappropriate.
## Next Steps: V2 Enhancements
In our upcoming **v2 release**, we are:
- **Adopting a block-level payload approach** similar to Bera’s design to reduce consensus overhead.
- **Optimizing block finality time** to ensure fast transaction confirmation without sacrificing high TPS.
- **Conducting further benchmarks** to refine performance tradeoffs between block size, latency, and gas fees.
Loading