psm.test.js behaves flakey in z:acceptance

[anilhelvaci]

Problem Definition

In CI we observed some flakiness with psm.test.js.

https://github.com/Agoric/agoric-sdk/actions/runs/12363738833/job/34505666038?pr=10692#step:10:8349
https://github.com/Agoric/agoric-sdk/actions/runs/12363738833/job/34505666038?pr=10692#step:10:7692

The error message being;

  Error: psmSwap tx send failed: {"exitCode":2}
      at psmSwap (file:///usr/src/proposals/z:acceptance/test-lib/psm-lib.js:552:11)
      at async file:///usr/src/proposals/z:acceptance/psm.test.js:195:3

7:00PM ERR Error on broadcastTxCommit err="timed out waiting for tx to be included in a block" module=rpc
psmSwap tx send failed {
  command: 'agoric wallet --keyring-backend=test send --offer /tmp/agops.OlQ --from agoric1zlesuj60a3f59slushj65xx5asmzd8s7mn59qm --verbose',
  durationMs: 11091.834717,
  shortMessage: "Command failed with exit code 2: agoric wallet '--keyring-backend=test' send --offer /tmp/agops.OlQ --from agoric1zlesuj60a3f59slushj65xx5asmzd8s7mn59qm --verbose",
  cause: undefined,
  exitCode: 2,
…
    'Error: error in json rpc client, with http response metadata: (Status: 200 OK, Protocol HTTP/1.1). RPC error -32603 - Internal error: timed out waiting for tx to be included in a block\n' +

Cause

This issue seems to be related to -bblock flag that is used, used in;

• agoric-sdk/a3p-integration/proposals/z:acceptance/test-lib/psm-lib.js

  Line 447 in 7b7ceb9

  export const sendOfferAgoric = async (address, offerPromise) => {

• agoric-sdk/a3p-integration/proposals/z:acceptance/test-lib/psm-lib.js

  Line 470 in 7b7ceb9

  export const sendOfferAgd = async (address, offerPromise) => {

Looking at cosmos-sdk repo, we can see that -bblock is deprecated as of v0.47. Here are some issues talking about this;

• Deprecate and phase out broadcast mode block. cosmos/cosmos-sdk#12167
• when "ERROR: broadcast_tx_commit: Response error: RPC error -32603 - Internal error: Timed out waiting for tx to be included in a block" cosmos/cosmos-sdk#3104
• Failed tx stuck in cache cosmos/cosmos-sdk#1043
• Sending a proto tx message without the public key causes error in RPC and tx gets stuck in cache cosmos/cosmos-sdk#7585
• CLI times out too quickly cosmos/cosmos-sdk#3875
• Broadcast timeout should still display the TxHash cosmos/cosmos-sdk#12144

Here's the PR that deprecated -bblock: cosmos/cosmos-sdk#12659

This issue describes a possible reproducing scenario: cosmos/cosmos-sdk#8510

Why do we only see this error in psm.test.js?

I went over all tests under z:acceptance and observed that all tests other than psm.test.js use executeOffer which uses agops.perf that doesn't specify a broadcast option (sync, async, block).

The reason why we didn't use executeOffer in psm.test.js is; in order to test auto provisioning we needed to agoric wallet send as agops.perf needs a provisioned account to function.

Solution

Start using sendOfferAgd and remove -bblock option from it.