Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: testnet prod sync #4358

Merged
merged 47 commits into from
May 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
e9b90ab
feat: separate router and sequencer admin keys (#4316)
May 23, 2023
bb4f0e6
feat: add nonce to stableswap staging events
liu-zhipeng May 23, 2023
4a5d41a
feat: add nonce to v0 stableswap events
liu-zhipeng May 23, 2023
04550e9
fix: event id
liu-zhipeng May 23, 2023
2993cee
feat: add nonce field to types
liu-zhipeng May 23, 2023
f386c5c
feat: add nonce to stableswap db migration
liu-zhipeng May 23, 2023
33dbede
fix: database adapter for stableswap nonce
liu-zhipeng May 23, 2023
c8b74bf
chore: add commitlint local setup
rhlsthrm May 23, 2023
e1456b8
ci: update ci workflow for commitlint
rhlsthrm May 23, 2023
3c32ef6
feat: update subgraph & poller logic with nonce
liu-zhipeng May 23, 2023
29d401e
feat: rebuild graph-client
liu-zhipeng May 23, 2023
37e999c
fix: query with bigInt params
liu-zhipeng May 23, 2023
4e8abb4
fix: carto poller unit tests
liu-zhipeng May 23, 2023
28eb9ec
fix: remove unused checkpoint for asset updates
liu-zhipeng May 24, 2023
c222238
fix(test): unit tests for poller
liu-zhipeng May 24, 2023
efa30c4
Merge pull request #4321 from connext/4313-stableswap-lp-poller-500-tx
liu-zhipeng May 24, 2023
b91e4cb
chore: add code owners (#4318)
May 24, 2023
f754122
Merge pull request #4320 from connext/4319-add-commitlint-verificatio…
sanchaymittal May 24, 2023
8a94afd
ci: fix syntax
rhlsthrm May 24, 2023
f852100
ci: try different install
rhlsthrm May 24, 2023
d6b61e5
ci: fix commitlint
rhlsthrm May 24, 2023
043509d
ci: validate commit
rhlsthrm May 24, 2023
c204fb4
feat: graph-client build
liu-zhipeng May 24, 2023
ed4e269
Merge pull request #4332 from connext/rebuild-graph-client
liu-zhipeng May 24, 2023
57768c4
Merge branch 'testnet-prod'
rhlsthrm May 24, 2023
9817a68
Don't commitlint testnet-prod and prod
rhlsthrm May 24, 2023
af4c3ef
Hot Fix: Update Linea goerli RPC
preethamr May 24, 2023
7ca74b5
Merge branch 'testnet-prod' into main
preethamr May 24, 2023
1c24c81
Fix: Disable commitlint in testnet-prod
preethamr May 25, 2023
3626c78
feat: getSupportedAssets and tests added (#4324)
prathmeshkhandelwal1 May 25, 2023
998d0db
Fix aggregate root no exist (#4335)
wanglonghong May 25, 2023
6bc1259
feat: should ack and reject (#4336)
wanglonghong May 25, 2023
37cbd57
Merge branch 'testnet-prod'
rhlsthrm May 25, 2023
c2e21f3
chore: change comparison (#4330)
wanglonghong May 25, 2023
ca03018
Improve logging and refactor (#4337)
wanglonghong May 25, 2023
0f16b83
Paginate reading pending txs from the redis cache (#4342)
wanglonghong May 26, 2023
fe3b06c
fix: commitlint
rhlsthrm May 26, 2023
3c0f154
Merge branch 'testnet-prod'
rhlsthrm May 26, 2023
c502b36
Merge branch 'testnet-prod'
rhlsthrm May 26, 2023
4730aee
chore: remove console logs
rhlsthrm May 26, 2023
7991381
ci: fix batch size
rhlsthrm May 26, 2023
87fe039
feat: refactor sequencer subscriber (#4352)
wanglonghong May 29, 2023
d223e76
feat: dont scale with load (#4354)
wanglonghong May 29, 2023
1a138a2
fix: resolve testnet prod sync conflicts (#4356)
wanglonghong May 29, 2023
c7ba300
Merge branch 'testnet-prod'
rhlsthrm May 29, 2023
a213cda
ci: commitlint
rhlsthrm May 29, 2023
badfaac
fix: expire data check (#4357)
wanglonghong May 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@

## Connext Architecture

The Connext architecture can be seen as a layered system, as follows:
The Connext architecture can be seen as a layered system, as follows.

| Layer | Protocol/Stakeholders |
| -------------------------------- | -------------------------------------- |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export const bindSubscriber = async (queueName: string, channel: Broker.Channel)
if (consumeMessage) {
const { transferId, type: messageType } = JSON.parse(consumeMessage.content.toString()) as Message;
const { requestContext, methodContext } = createLoggingContext("Subscriber.consume", undefined, transferId);
logger.info("Executing the transfer", requestContext, methodContext, { message: consumeMessage });
logger.info("Executing the transfer", requestContext, methodContext, { transferId, messageType });

if (messageType === MessageType.ExecuteFast) {
await cache.auctions.setExecStatus(transferId, ExecStatus.Dequeued);
Expand Down Expand Up @@ -80,7 +80,7 @@ export const bindSubscriber = async (queueName: string, channel: Broker.Channel)
}
}

channel.reject(consumeMessage);
channel.reject(consumeMessage, false);
}
}
},
Expand Down
8 changes: 3 additions & 5 deletions packages/agents/sequencer/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,9 @@ export const getEnvConfig = (
configJson.auctionWaitTime ||
configFile.auctionWaitTime ||
DEFAULT_AUCTION_WAIT_TIME,
executionWaitTime:
process.env.SEQ_EXECUTION_WAIT_TIME ||
configJson.executionWaitTime ||
configFile.executionWaitTime ||
DEFAULT_EXECUTION_WAIT_TIME,
executionWaitTime: process.env.SEQ_EXECUTION_WAIT_TIME
? +process.env.SEQ_EXECUTION_WAIT_TIME
: undefined || configJson.executionWaitTime || configFile.executionWaitTime || DEFAULT_EXECUTION_WAIT_TIME,
mode: {
cleanup: process.env.SEQ_CLEANUP_MODE || configJson.mode?.cleanup || configFile.mode?.cleanup || false,
},
Expand Down
1 change: 1 addition & 0 deletions packages/agents/sequencer/src/lib/entities/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ export const TMessageQueueConfig = Type.Object({
queues: Type.Array(TMQQueueConfig),
bindings: Type.Array(TMQBindingConfig),
executerTimeout: Type.Integer(),
prefetch: Type.Optional(Type.Integer()),
publisher: Type.Optional(Type.String()),
subscriber: Type.Optional(Type.String()),
});
Expand Down
4 changes: 2 additions & 2 deletions packages/agents/sequencer/src/lib/errors/executor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { ExecStatus, NxtpError } from "@connext/nxtp-utils";

export class ExecutorDataExpired extends NxtpError {
constructor(status: ExecStatus, context: any = {}) {
super("This lighthouse data has already expired.", { status, ...context }, ExecutorDataExpired.name);
super("This executor data has already expired.", { status, ...context }, ExecutorDataExpired.name);
}
}

Expand All @@ -26,7 +26,7 @@ export class MissingTransfer extends NxtpError {

export class MissingExecutorData extends NxtpError {
constructor(context: any = {}) {
super("No lighthouse data found in the cache", context, MissingExecutorData.name);
super("No executor data found in the cache", context, MissingExecutorData.name);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ export const executeSlowPathData = async (

// Ensure that the executor data for this transfer hasn't expired.
const status = await cache.executors.getExecStatus(transferId);
if (status !== ExecStatus.Enqueued) {
if (status !== ExecStatus.Dequeued) {
throw new ExecutorDataExpired(status, {
transferId,
executorData,
Expand Down
5 changes: 4 additions & 1 deletion packages/agents/sequencer/src/sequencer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const context: AppContext = {} as any;
export const getContext = () => context;
export const msgContentType = "application/json";
export const SlippageErrorPatterns = ["dy < minDy", "Reverted 0x6479203c206d696e4479", "more than pool balance"]; // 0x6479203c206d696e4479 -- encoded hex string of "dy < minDy"

export const DEFAULT_PREFETCH_SIZE = 100;
/// MARK - Make Agents
/**
* Sets up and runs the sequencer publisher unit. Receives bids from router network and assigns to transfer by ID,
Expand Down Expand Up @@ -86,6 +86,9 @@ export const makeSubscriber = async () => {
try {
const mqClient = context.adapters.mqClient;
const channel = await mqClient.createChannel();
const prefetchSize = context.config.messageQueue.prefetch ?? DEFAULT_PREFETCH_SIZE;
channel.prefetch(prefetchSize);

await channel.assertExchange(
context.config.messageQueue.exchanges[0].name,
context.config.messageQueue.exchanges[0].type,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ describe("Operations:Execute:SlowPath", () => {

getTransferStub.resolves(mockTransfer);
getExecutorDataStub.resolves(mockExecutorData);
getExecStatusStub.resolves(ExecStatus.Enqueued);
getExecStatusStub.resolves(ExecStatus.Dequeued);
getBackupDataStub.resolves([mockExecutorBackupData1, mockExecutorBackupData2]);
sendExecuteSlowToRelayerStub.onCall(0).throws("Failed to send to the gelato");
sendExecuteSlowToRelayerStub.onCall(1).resolves({ taskId: undefined, relayer: undefined });
Expand All @@ -208,7 +208,7 @@ describe("Operations:Execute:SlowPath", () => {

getTransferStub.resolves(mockTransfer);
getExecutorDataStub.resolves(mockExecutorData);
getExecStatusStub.resolves(ExecStatus.Enqueued);
getExecStatusStub.resolves(ExecStatus.Dequeued);
getBackupDataStub.resolves([mockExecutorBackupData1, mockExecutorBackupData2]);
sendExecuteSlowToRelayerStub.onCall(0).throws("Failed to send to the gelato");
sendExecuteSlowToRelayerStub.onCall(1).resolves({ taskId: undefined, relayer: undefined });
Expand Down