Usability/DevUX Meta Issue

[liamsi]

This issue collects smaller quirks with the current API/docs/CLI in one place. I will list them here and break them out into their own issues where it makes sense:

• the comment here is wrong:

  celestia-node/blob/service.go

  Line 113 in f2b664f

  // Get retrieves all the blobs for given namespaces at the given height by commitment.

  (it only returns one blob; GetAll is what returns all blobs)
• GetAll is confusing and edge-cases are under-specified: bug: blob.getAll returns ERROR when no blob is found, but is not error #3292 (comment)
• some examples in API docs are not representative; e.g. blob.Get returns a different commitment than requested ([Feature Request]: improve example api values as they are not realistic #3317 )
• CLI: when looking at the for usage returned by --help encoding related guidance is often missing, eg. celestia blob get [height] [namespace] [commitment] [flags] should state which format the inputs are expected in (hex? base64?). The CLI help should be self-contained; currently one needs to e.g. look here to know how to use this
• CLI: namespaces are submitted as hex but returned base64 encoded (which is very confusing); example:

celestia blob submit 0x42690c204d39600fddd3 0x676d                                                                                                                  
{
  "result": {
    "height": 1671475,
    "commitments": [
      "0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY="
    ]
  }
}
⋊> ~/G/celestia-node on main ⨯ celestia blob get-all 1671475  0x42690c204d39600fddd3                                                                                                               12:06:50
{
  "result": [
    {
      "namespace": "AAAAAAAAAAAAAAAAAAAAAAAAAEJpDCBNOWAP3dM=",
      "data": "0x676d",
      "share_version": 0,
      "commitment": "0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=",
      "index": 531
    }
  ]
}

• CLI/API: the node daemon gets started with only one account and one can only submit PFBs from that single account; this does not reflect how people interact with chains using different wallets for different purposes (applies to e.g. celestia blob submit and a bunch of state related commands / rpcs) [Feature Request]: Multiple celestia accounts can submit blobs via 1 DA Node #3295
• CLI/API: very unclear how to use state submit-tx #3345
• consumers of the API do not have sufficient control for retries on blob submission
• also related to the above: [Feature Request]: allow user to set account sequence #3164
• Blobstream API should be moved to node #3361
• API: Clarify State.SubmitPayForBlob vs Blob.Submit: both methods are too similar and it's unclear when to use which (related: feat!(state/core_access): add TxConfig #3349 (comment))

[Bidon15]

Should be populated in node somehow

Blobstream APIs in node

celestiaorg/celestia-core#1306

Many Execution Environments to 1 DA Node

celestiaorg/celestia-app#3259
#3295
#3022

Separation of endpoints for DA Node (RPC/GRPC)

#2931

JWT Secret Rotation

#3293

[tuxcanfly]

I would like to add a couple of improvement ideas for reliable blob submission which is a blocker for rollup liveness:

• "rpc error: code = Unknown desc = timed out waiting for tx to be included in a block" error is returned frequently possibly due to 1) mempool congestion 2) big blobs 3) too small timeout_broadcast_tx_commit
• ": tx already in mempool" always happens immediately after the above error and requires waiting out until the tx drops from the mempool
• ": incorrect account sequence" occasionally happens in place of the above error but also requires waiting out until the tx drops from the mempool
• rpc error: code = Unknown desc = error on broadcastTxCommit: tx size is too big: 1962442, max: 1962441 happens when submitting big blobs due to padding shares even though the supposed limit is 1974272 so this breaks the contract with node.
• rpc error: code = Unknown desc = error on broadcastTxCommit: tx too large occasionally big blobs return this in place of the above. I see that there's also Tx too large. Max size is %d, but got %d in the code.

Since there is no unique error code for these errors, we're currently grepping the string for these error strings which is not ideal.

It would be good to have a standardised error list and unique error codes and if possible, a error severity, and hint to allow clients to retry gracefully.

[MSevey]

Related: rollkit/go-da#65

[vgonkivs]

#3537

[vgonkivs]

also related to the above: #3164

After the fix on the app the issue "invalid account sequence" gone.

[vgonkivs]

the comment here is wrong

#3530

GetAll is confusing and edge-cases are under-specified

#3223

some examples in API docs are not representative; e.g. blob.Get returns a different commitment than requested

#3479

CLI: when looking at the for usage returned by --help encoding related guidance is often missing, eg. celestia blob get [height] [namespace] [commitment] [flags] should state which format the inputs are expected in
CLI: namespaces are submitted as hex but returned base64 encoded (which is very confusing); example:

#3553
#3537

CLI/API: the node daemon gets started with only one account and one can only submit PFBs from that single account; this does not reflect how people interact with chains using different wallets for different purposes (applies to e.g. celestia blob submit and a bunch of state related commands / rpcs

#3349

#3345

#3391

API: Clarify State.SubmitPayForBlob vs Blob.Submit: both methods are too similar and it's unclear when to use which

#3349 (comment)

[liamsi]

As all issues here have been tackled I think it is good to close this one for now. If there is a list of new issues regarding the API, we should open a new meta issue. Massive shoutout to @vgonkivs for tackling all these 🙏🏼 ❤️ 🎉