Test celestia-node with celestia-app v2

[rootulp]

Problem

The main branch in this repo diverges significantly from the release that celestia-node uses here. We don't expect any of the consensus breaking changes in celestia-app v2 to impact celestia-node but we should verify that celestia-node remains compatible with main.

Proposal

We should gather information on the compatability between celestia-app v2 and celestia-node. Two proposed tests:

1. Bump the celestia-node dependency on celestia-app to current main and test on a devnet, Robusta, or Knuu
2. Don't bump the celestia-node dependency on celestia-app and point it at a celestia-app main node (via core endpoint) on a devnet, Robusta, or Knuu

[rootulp]

Triggered a Robusta (a.k.a testnet) creation here for celestia-app v2.0.0-rc0 and celestia-node v0.13.5. Waiting to hear back from DevOps on where to get endpoints for the testnet b/c Notion just links to a Grafana dashboard that doesn't contain info from my Robusta.

[rootulp]

Robusta cluster is failing because validator can't start due to

panic: failed to execute DeliverTx for '{"body":{"messages":[{"@type":"/cosmos.staking.v1beta1.MsgCreateValidator","description":{"moniker":"consensus-validator-1-0","identity":"","website":"","security_contact":"","details":""},"commission":{"rate":"0.100000000000000000","max_rate":"0.200000000000000000","max_change_rate":"0.010000000000000000"},"min_self_delegation":"1","delegator_address":"celestia17l99xdwqeczwyn6ntleyflhhw6t8nj3eg0luzn","validator_address":"celestiavaloper17l99xdwqeczwyn6ntleyflhhw6t8nj3edsa954","pubkey":{"@type":"/cosmos.crypto.ed25519.PubKey","key":"kEdBxt24XEPUVEkh6e+WTP14Fk8rPffpkM4CJAX/C64="},"value":{"denom":"utia","amount":"500000000000000"}}],"memo":"ee06dc70c64e9a52fd3f3cafc2a6c467d23711b6@100.65.124.48:26656","timeout_height":"0","extension_options":[],"non_critical_extension_options":[]},"auth_info":{"signer_infos":[{"public_key":{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"Aj23FpNdoi5HtPe2QjQ1aOm1f/EKfp3NQkS5Qw1C4CH/"},"mode_info":{"single":{"mode":"SIGN_MODE_DIRECT"}},"sequence":"0"}],"fee":{"amount":[],"gas_limit":"210000","payer":"","granter":""},"tip":null},"signatures":["ekgoNuQwwBQyIGzmdXFISZq+otYFXiBp8jD7FADO4sccsQLTGHRNOxluJK9Ggboqz3P0fg0V16iqxlUDd8UxjQ=="]}': insufficient gas price for the network; got: 0 required at least: 1.000000000000000000: insufficient fee

We already have an issue: #3271 but we can workaround it by including a minfee in the transactions that create validators.

It looks like the create validator already provides a fee so perhaps we just need to provide a fee for the gentx command? I can't find the script in auto-devops that actually invokes gentx.

[rootulp]

Yay this PR fixed auto devops so now we can deploy a Robusta with v2.0.0-rc0.

[rootulp]

The DA nodes fail to start because

Error: node: failed to start: app version mismatch, expected: 1, got 2

[rootulp]

1. We need celestia-node to bump it's dependency of celestia-app to v2.0.0-rc1 and also adjust these lines. See chore(deps): upgrade to celestia-app v2.0.0 celestia-node#3453
2. We should make Robusta modify the genesis file to use app version v1 and then supply a --v2-upgrade-height flag. Created https://github.com/celestiaorg/celestia-stack-internal/issues/435
3. We should test existing celestia-node with celestia-app v2.0.0-rc1 on a chain that is still using app version v1. Tested using these scripts

celestia-app version	celestia-node version	chain app version	works	logs
v2.0.0-rc1	v0.13.6	v1	✅
v2.0.0-rc1	v0.13.6	v2	❌	2024-05-31T14:57:14.809-0400 ERROR header/p2p p2p/subscriber.go:123 invalid header {"from": "<peer.ID 12*USyMPb>", "err": "app version mismatch, expected: 1, got 2"}
v2.0.0-rc1	rp/support-v2	v1	✅
v2.0.0-rc1	rp/support-v2	v2	✅
v1.9.0	rp/support-v2	v1	✅