delta team monoissue

[ludamad]

Catch-all delta team issues not spun out into an epic, but doesn't make much sense to granulize until that happens

---

Epics

• EPIC: deployments for S&P testnet #9630

---

Current

Tasks

Preview Give feedback

1. add 'stern' output in background to current KIND network tests so we see logs on failure
2. Look into race condition when builders are created without setup-runner step, see https://github.com/AztecProtocol/aztec-packages/actions/runs/11501047193
3. Do pass with pull-request-target trigger exclusively for external PRs (secrets dont currently work even if approved)
4. Summarize state of building the repo in an all-hands (ideally with granularity fix for bootstrap_cache.sh) i.e. mainly encourage people to use ./bootstrap.sh fast
5. Consider alerts for testnet
6. Use https://github.com/tj/commander.js?tab=readme-ov-file#typescript and type magic to type current commander setup
7. Implement Palla's logging wishlist https://hackmd.io/@aztec-network/rkebFFEhA?type=view (possibly with pino
8. Refactor helm chart value passing to use themed values files, e.g. load-balancers-on.yml metrics-on.yml bots-5.yml etc
9. Remove old devnet deployment stuff - Spyros
10. Test ARM images on GCP
11. Test multiple boot nodes
12. Add check if master PR can be cherry-picked onto release/testnet-1

Needs breakdown

Tasks

Preview Give feedback

1. Improve logging from the point of view of a node operator #9015
   +0
   
2. Run provernet-like deployment after getting docker compose setup, fire many tx bots, see stability, create node scenario where node tries to sync periodically #9222
   +0
   
3. Once stable, run Aztec Scenario Tests #9285
   +0
   

Recently closed

Tasks

Preview Give feedback

1. Work towards our nightly scenario tests #9305
   +0
   
2. Look into race condition in ensure-builder with cloning git repo + initializing run folder #9240
   +0
   
3. Discuss: Rename fee juice to TST token or other #9415
   +0
   
4. Ensure that bb.js publishes with new release flow #9283
   +0
   
5. Version S3 Public aztec-up scripts #9427
   +0
6. Add a native network test (non-k8s) that tests a node going down and resync'ing often #9282
   +0
   

Lower priority

Tasks

Preview Give feedback

1. Delete yarn-project/end-to-end/Earthfile and replace with shell script that provides same data (and runner kind) #9221
   +0
   
2. Separate boot node from contract deployment for improved scalability
3. Fix aztecprotocol/aztec:master on merges to master #9220
   +0
   
4. helm charts: pxe may need bigger timeout, restarted 4 times
5. Connect spartan deployments to existing grafana instance #9224
   +0
   
6. nightly/masterly kind test: assert that nothing has restarted
7. persistent view of jaeger (or elasticsearch? just grafana for tracing? cloudwatch?) and grafana etc otel metrics in nightly tests (ideally just grafana and cloudwatch)
8. fixed jaeger (or elasticsearch? just grafana for tracing? just cloudwatch?) installation in aws tunnelled to with SSH?
9. provide way to change debug level of node when running live e.g. through a signal handler or watching a config file for live config changes
10. Introduce more registry images in place of current apt installs (cache-proof). Update them with build-images pattern
11. Push intermediate images in publish-images.yml to aztecprotocol/cache #9230
    +0
12. Replace critical build path with a docker build that uses/populates S3 cache from minio, rather than earthly
13. Replace earthly pattern slowly with docker + pull from minio
14. Finish guide PR #9232
    +0
    
15. Run e2e tests against a remote #9598
    +0
16. bb and yarn project formatting should simply update the PR for productivity
17. Earthly writes 34gb of cache on a fresh build - get that down while simplifying earthly

Admin

Tasks that fall outside team mandate (here for convenience)

Tasks

Preview Give feedback

1. Adam: try again to strip bb wasm / reduce size of bb.js artifacts
2. Work with Alex (?) to lower 11 minute fully cached build times