Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: tree heights that last past 3 days #9760

Merged
merged 17 commits into from
Nov 8, 2024

Conversation

ludamad
Copy link
Collaborator

@ludamad ludamad commented Nov 5, 2024

The L1-L2 message tree height was a bottleneck running

post-mortem of 1-validator network (bot set to 0.05 TPS, 1 private / 2 public transfers per tx)
Lasted long, got to block 4091, last tried to propose block 4097
Hit issues and did not reorg past them
Root issue (guess):
2024-11-05 08:32:53.148	Error assembling block: 'Error: Failed to append leaves: Tree is full'

Also updated tree heights with constants proposed by @iAmMichaelConnor here (#9451) (thanks for the thoughtful analysis I could lazily steal!
Automated test is a bit awkward here or I'd write one. It'd either trivially pass or have to go through 3-days worth of transactions.

The L1-L2 message tree height was a bottleneck running 
```
post-mortem of 1-validator network (bot set to 0.05 TPS, 1 private / 2 public transfers per tx)
Lasted long, got to block 4091, last tried to propose block 4097
Hit issues and did not reorg past them
Root issue (guess):
2024-11-05 08:32:53.148	Error assembling block: 'Error: Failed to append leaves: Tree is full'
```
Automated test is a bit awkward here or I'd write one. It'd either trivially pass or have to go through 3-days worth of transactions.
@ludamad ludamad enabled auto-merge (squash) November 5, 2024 19:25
@ludamad ludamad added A-installation Area: issues relating to installing / building something. network-all Run this CI job. e2e-all CI: Enables this CI job. and removed A-installation Area: issues relating to installing / building something. labels Nov 5, 2024
Copy link
Contributor

github-actions bot commented Nov 5, 2024

Changes to circuit sizes

Generated at commit: e6266c16a270d4263007cdf43b99f073b09c10fa, compared to commit: 3a49cfb4d053f26f6f73f342c2c457bf18ae77d2

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_reset +10,767 ❌ +14.68% +155,407 ❌ +33.59%
private_kernel_reset_4_4_4_4_4_4_4_4_1 +667 ❌ +2.10% +9,667 ❌ +12.57%
rollup_base_private +15,774 ❌ +4.97% +221,618 ❌ +6.90%
rollup_base_public +15,774 ❌ +3.47% +221,618 ❌ +6.24%
rollup_block_root +432 ❌ +10.74% +5,928 ❌ +0.21%

Full diff report 👇
Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_reset 84,098 (+10,767) +14.68% 618,118 (+155,407) +33.59%
private_kernel_reset_4_4_4_4_4_4_4_4_1 32,456 (+667) +2.10% 86,602 (+9,667) +12.57%
rollup_base_private 332,869 (+15,774) +4.97% 3,432,522 (+221,618) +6.90%
rollup_base_public 470,052 (+15,774) +3.47% 3,770,650 (+221,618) +6.24%
rollup_block_root 4,453 (+432) +10.74% 2,863,216 (+5,928) +0.21%

@ludamad ludamad disabled auto-merge November 5, 2024 23:52
@ludamad ludamad requested a review from just-mitch November 6, 2024 23:58
@fcarreiro fcarreiro requested review from dbanks12 and removed request for fcarreiro, IlyasRidhuan and jeanmon November 7, 2024 10:00
Copy link
Collaborator

@dbanks12 dbanks12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resultant bumps in avm gas are fine with me! Those opcodes trigger merkle checks, so I made them scale with tree heights. The scaling factors are somewhat arbitrary at the moment.

I don't know if someone else should review who has a better understanding of exactly what our tree height requirements are, but I'm happy to approve

@just-mitch
Copy link
Collaborator

The kind smoke test failure I think was fixed by #9790

Maybe changing all the constants at once was too ambitious? Maybe just start with the l1-l2 tree? I'm just concerned about the time needed to track down the places where the other values are hardcoded/assumed.

@ludamad
Copy link
Collaborator Author

ludamad commented Nov 7, 2024

I systematically went through the hardcoded ones now that had a comment above them with the formula and hopefully that's enough (there was 1, alerted to by the e2e-authwit test). If not I can redo this - but note a redo is a surprising amount of faff unless I'm doing it wrong

@ludamad ludamad removed e2e-all CI: Enables this CI job. network-all Run this CI job. labels Nov 8, 2024
Copy link
Collaborator

@just-mitch just-mitch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@ludamad ludamad merged commit 23c122d into master Nov 8, 2024
72 checks passed
@ludamad ludamad deleted the ad/fix-tree-heights-crash-3-days branch November 8, 2024 23:07
ludamad added a commit that referenced this pull request Nov 11, 2024
The L1-L2 message tree height was a bottleneck running
```
post-mortem of 1-validator network (bot set to 0.05 TPS, 1 private / 2 public transfers per tx)
Lasted long, got to block 4091, last tried to propose block 4097
Hit issues and did not reorg past them
Root issue (guess):
2024-11-05 08:32:53.148	Error assembling block: 'Error: Failed to append leaves: Tree is full'
```
Also updated tree heights with constants proposed by @iAmMichaelConnor
here (#9451)
(thanks for the thoughtful analysis I could lazily steal!
Automated test is a bit awkward here or I'd write one. It'd either
trivially pass or have to go through 3-days worth of transactions.
TomAFrench added a commit that referenced this pull request Nov 11, 2024
* master: (182 commits)
  feat(avm): mem specific range check (#9828)
  refactor: remove public kernel inner (#9865)
  chore: Revert "chore: Validate RPC inputs" (#9875)
  Revert "fix: deploy l2 contracts fails on 48 validator" (#9871)
  fix: deploy l2 contracts fails on 48 validator (#9860)
  chore: Validate RPC inputs (#9672)
  fix: fixing devcontainers to use the sandbox docker-compose file (#9782)
  fix: Revert changes to ci.yml (#9863)
  chore: Move epoch and slot durations to config (#9861)
  git subrepo push --branch=master noir-projects/aztec-nr
  git_subrepo.sh: Fix parent in .gitrepo file. [skip ci]
  chore: replace relative paths to noir-protocol-circuits
  git subrepo push --branch=master barretenberg
  fix: tree heights that last past 3 days (#9760)
  fix(build): l1-contracts .rebuild_patterns did not cover test files (#9862)
  fix: bench prover test (#9856)
  fix: Fix mac build by calling `count` on durations (#9855)
  feat: zk shplemini (#9830)
  feat: domain separate block proposals and attestations (#9842)
  chore: bump runner cache disk size (#9849)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants