ci: testnet (#4459)

* feat: allow horizontal scalability of lighthouse prover (#4347)

* feat: install amqplib

* build: fix block-scoped variable

* feat: make pub/sub

* feat: add cache

* fix: cache and config

* feat: mvp of mq consumer

* feat: lambda subscriber (#4372)

* ci: lambda subscriber

* ci: lambda subscriber

* chore: bump chain abstraction

* fix: update tf config for sequncer cname

* fix: sequencer http queue config update

* fix: add auto scaling to seq publisher

* fix: typo and seq health port config change

* ci: prover

---------

Co-authored-by: Rahul Sethuram <rahul.eth@icloud.com>
Co-authored-by: just-a-node <eye1717@gmail.com>
Co-authored-by: preethamr <preethamr@users.noreply.github.com>

* chore: yarn

* ci: fix tf and build docker

* ci: always build lh

* ci: try adding package

* ci: secrets

* ci: fix lh config

* ci: fix mq

* test: missing functions and branches

* chore: Yarn lock

* chore: eslint

* ci: check the latest commit

* fix: index

* ci: ops

* feat: create a queue and binding on publisher side

* ci: bump memory

* feat: get event from process.argv

* fix: type

* chore: logging arg1 and arg2

* chore: move logging location

* chore: console

* chore: optional

* feat: decode base64 string

* fix: pass event param

* ci: change secret

* fix: decode properly

* ci: secret

* ci: secret

* ci: ops for lh prover

* ci: remove lighthouse lambda subscriber

* ci: var

* feat: implement message cache

* feat: update message cache per pair

* feat: store processed messages

* choe: print logs

* chore: remove console

* fix: use lighthouse directory

* fix: disable lh prover caching temporarily

* fix: update tf config for lh prover sub

* fix: make auto scaling more responsive to load

* fix: add lh redis config to main and staging tf

* Revert "fix: disable lh prover caching temporarily"

This reverts commit 8204f7a.

* fix: update config to connect lambda to redis

* lh prover in vpc

* fix: lh prover lambda network config

* fix: attach private subnets only

* ci: fix

* ci: ops

---------

Co-authored-by: Rahul Sethuram <rahul.eth@icloud.com>
Co-authored-by: Carlo Mazzaferro <carlo.mazzaferro@gmail.com>
Co-authored-by: just-a-node <eye1717@gmail.com>
Co-authored-by: preethamr <preethamr@users.noreply.github.com>

* feat: batch size (#4458)

---------

Co-authored-by: wanglonghong <wanglonghong4467@gmail.com>
Co-authored-by: Carlo Mazzaferro <carlo.mazzaferro@gmail.com>
Co-authored-by: just-a-node <eye1717@gmail.com>
Co-authored-by: preethamr <preethamr@users.noreply.github.com>

ops/testnet/prod/core/config.tf

@@ -315,7 +315,24 @@ locals {
     }
     hubDomain = "1735353714"
     proverBatchSize = {
-      "1668247156" = 10
+      "1668247156" = 10,
+      "9991" = 10,
+      "1735353714" = 10,
+      "2053862260" = 10,
+      "1734439522" = 10,
+      "1735356532" = 10
+    }
+    messageQueue = {
+      connection = {
+        uri = "amqps://${var.rmq_mgt_user}:${var.rmq_mgt_password}@${module.centralised_message_queue.aws_mq_amqp_endpoint}"
+      }
+      exchange = {
+        name           = "proverX"
+        type           = "direct"
+        publishTimeout = 1000
+        persistent     = true
+        durable        = true
+      }
     }
     messageQueue = {
       connection = {