Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth_getLogs - add support for multiple addresses #719

Merged
merged 17 commits into from
Dec 7, 2022
Merged

eth_getLogs - add support for multiple addresses #719

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c2df64c
feat: support multiple address search in eth_getLogs
Ivo-Yankov Nov 28, 2022
08cd041
feat: add acceptance test
Ivo-Yankov Nov 28, 2022
2a3f625
test: add unit test
Ivo-Yankov Nov 28, 2022
960bb2f
chore: pass addresses as array
Ivo-Yankov Nov 29, 2022
cec817f
refactor: reduce getLogs complexity
Ivo-Yankov Nov 29, 2022
f5406d1
refactor: further reduce complexity
Ivo-Yankov Nov 29, 2022
a9e4df1
Merge branch 'main' into 675-get-logs-multiple-addresses
Ivo-Yankov Dec 1, 2022
6194b16
fix: handle 404 getLogs by address
Ivo-Yankov Dec 1, 2022
9b811dd
nit: rename ambiguous functions
Ivo-Yankov Dec 1, 2022
7b80d61
nit: code simplification
Ivo-Yankov Dec 1, 2022
edf71d6
Merge branch 'main' into 675-get-logs-multiple-addresses
Ivo-Yankov Dec 2, 2022
67f3c72
chore: resolve conflicts
Ivo-Yankov Dec 2, 2022
5788aeb
feat: throw error for toBlock with no fromBlock param
Ivo-Yankov Dec 2, 2022
799274d
fix: failing acceptance test
Ivo-Yankov Dec 2, 2022
d5a9810
feat: behaviour parity with other relays
Ivo-Yankov Dec 2, 2022
27f1277
chore: reduce complexity
Ivo-Yankov Dec 5, 2022
c896281
Merge branch 'main' into 675-get-logs-multiple-addresses
Ivo-Yankov Dec 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 81 additions & 45 deletions packages/relay/src/lib/eth.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1281,81 +1281,117 @@ export class EthImpl implements Eth {
});
}

async getLogs(blockHash: string | null, fromBlock: string | null, toBlock: string | null, address: string | null, topics: any[] | null, requestId?: string): Promise<Log[]> {
const params: any = {};
if (blockHash) {
try {
const block = await this.mirrorNodeClient.getBlock(blockHash, requestId);
if (block) {
params.timestamp = [
`gte:${block.timestamp.from}`,
`lte:${block.timestamp.to}`
];
}else {
return [];
}
private async applyBlockHashParams(params: any, blockHash: string, requestId?: string) {
try {
const block = await this.mirrorNodeClient.getBlock(blockHash, requestId);
if (block) {
params.timestamp = [
`gte:${block.timestamp.from}`,
`lte:${block.timestamp.to}`
];
} else {
return false;
}
catch(e: any) {
if (e instanceof MirrorNodeClientError && e.isNotFound()) {
return [];
}

throw e;
}
catch(e: any) {
if (e instanceof MirrorNodeClientError && e.isNotFound()) {
return false;
}

throw e;
}

return true;
}

private async applyBlockRangeParams(params: any, fromBlock: string | null, toBlock: string | null, requestId?: string) {
Nana-EC marked this conversation as resolved.
Show resolved Hide resolved
const blockRangeLimit = Number(process.env.ETH_GET_LOGS_BLOCK_RANGE_LIMIT) || constants.DEFAULT_ETH_GET_LOGS_BLOCK_RANGE_LIMIT;
if (!fromBlock && !toBlock) {
const blockResponse = await this.getHistoricalBlockResponse("latest", true, requestId);
params.timestamp = [`gte:${blockResponse.timestamp.from}`, `lte:${blockResponse.timestamp.to}`];
} else {
const blockRangeLimit = Number(process.env.ETH_GET_LOGS_BLOCK_RANGE_LIMIT) || constants.DEFAULT_ETH_GET_LOGS_BLOCK_RANGE_LIMIT;
let fromBlockNum = 0;
let toBlockNum;
params.timestamp = [];

if (!fromBlock && !toBlock) {
const blockResponse = await this.getHistoricalBlockResponse("latest", true, requestId);
fromBlockNum = parseInt(blockResponse.number);
toBlockNum = parseInt(blockResponse.number);
params.timestamp = [`gte:${blockResponse.timestamp.from}`, `lte:${blockResponse.timestamp.to}`];
} else {
params.timestamp = [];
// Use the `toBlock` if it is the only passed tag, if not utilize the `fromBlock`
Copy link
Collaborator

@dimitrovmaksim dimitrovmaksim Nov 30, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't quite get what this means. However I think if toBlock is passed without fromBlock and the block number/tag is not latest/pending, we can either return an error (Alchemy) or an empty response (Infura). In the case of Infura I think it returns an empty response, because if not passed the fromBlock defaults to latest and it becomes fromBlock > toBlock, which results to logs not being found.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's return an error like Alchemy but make sure the error notes that it's due to a missing fromBlock when a toBlock was provided

Copy link
Collaborator

@dimitrovmaksim dimitrovmaksim Dec 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context:
When toBlock param value is the tag "latest" or "pending" -> returns logs for latest block
When toBlock param is hex number >= "latest" block number -> returns logs for latest block
When toBlock param is hex number < "latest" block number -> {"jsonrpc":"2.0","id":1,"error":{"code":-32000,"message":"One of the blocks specified in filter (fromBlock, toBlock or blockHash) cannot be found."}}

Unfortunately the 2 seconds per block creation time in hedera may sometimes be a problem here.

const blockTag = toBlock && !fromBlock ? toBlock : fromBlock || "latest";

// Use the `toBlock` if it is the only passed tag, if not utilize the `fromBlock` or default to "latest"
const blockTag = toBlock && !fromBlock ? toBlock : fromBlock || "latest";
const fromBlockResponse = await this.getHistoricalBlockResponse(blockTag, true, requestId);
if (!fromBlockResponse) {
return false;
}

const fromBlockResponse = await this.getHistoricalBlockResponse(blockTag, true, requestId);
if (fromBlockResponse != null) {
params.timestamp.push(`gte:${fromBlockResponse.timestamp.from}`);
fromBlockNum = parseInt(fromBlockResponse.number);
} else {
return [];
}
params.timestamp.push(`gte:${fromBlockResponse.timestamp.from}`);
fromBlockNum = parseInt(fromBlockResponse.number);

const toBlockResponse = await this.getHistoricalBlockResponse(toBlock || "latest", true, requestId);
if (toBlockResponse != null) {
params.timestamp.push(`lte:${toBlockResponse.timestamp.to}`);
toBlockNum = parseInt(toBlockResponse.number);
}
const toBlockResponse = await this.getHistoricalBlockResponse(toBlock || "latest", true, requestId);
if (toBlockResponse != null) {
params.timestamp.push(`lte:${toBlockResponse.timestamp.to}`);
toBlockNum = parseInt(toBlockResponse.number);
}

if (fromBlockNum > toBlockNum) {
return [];
} else if ((toBlockNum - fromBlockNum) > blockRangeLimit) {
return false;
} else if (toBlockNum - fromBlockNum > blockRangeLimit) {
throw predefined.RANGE_TOO_LARGE(blockRangeLimit);
}
}

return true;
}

private applyTopicParams(params: any, topics: any[] | null) {
if (topics) {
for (let i = 0; i < topics.length; i++) {
params[`topic${i}`] = topics[i];
}
}
}

private async getLogsByAddress(address: string | [string], params: any, requestId) {
const addresses = Array.isArray(address) ? address : [address];
let result = {
logs: []
};
const logPromises = addresses.map(addr => this.mirrorNodeClient.getContractResultsLogsByAddress(addr, params, undefined, requestId));

const logResults = await Promise.all(logPromises);
logResults.forEach(res => {
result.logs = result.logs.concat(res.logs);
})

result.logs.sort((a: any, b: any) => {
return a.timestamp >= b.timestamp ? 1 : -1;
})

return result;
}

async getLogs(blockHash: string | null, fromBlock: string | null, toBlock: string | null, address: string | [string] | null, topics: any[] | null, requestId?: string): Promise<Log[]> {
const EMPTY_RESPONSE = [];
const params: any = {};

if (blockHash) {
if ( !(await this.applyBlockHashParams(params, blockHash, requestId)) ) {
return EMPTY_RESPONSE;
}
} else if ( !(await this.applyBlockRangeParams(params, fromBlock, toBlock, requestId)) ) {
return EMPTY_RESPONSE;
}

this.applyTopicParams(params, topics);

let result;
if (address) {
result = await this.mirrorNodeClient.getContractResultsLogsByAddress(address, params, undefined, requestId);
result = await this.getLogsByAddress(address, params, requestId);
}
else {
result = await this.mirrorNodeClient.getContractResultsLogs(params, undefined, requestId);
}

if (!result || !result.logs) {
return [];
return EMPTY_RESPONSE;
}

const unproccesedLogs = await this.mirrorNodeClient.pageAllResults(result, requestId);
Expand Down
2 changes: 1 addition & 1 deletion packages/relay/tests/helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ export const blockTransactionCount = 77;
export const gasUsed1 = 200000;
export const gasUsed2 = 800000;
export const maxGasLimit = 250000;
export const firstTransactionTimestampSeconds = '1653077547';
export const firstTransactionTimestampSeconds = '1653077541';
export const contractAddress1 = '0x000000000000000000000000000000000000055f';
export const contractTimestamp1 = `${firstTransactionTimestampSeconds}.983983199`;
export const contractHash1 = '0x4a563af33c4871b51a8b108aa2fe1dd5280a30dfb7236170ae5e5e7957eb6392';
Expand Down
34 changes: 33 additions & 1 deletion packages/relay/tests/lib/eth.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ describe('Eth calls using MirrorNode', async function () {
const contractCallData = "0xef641f44";
const blockTimestamp = '1651560386';
const blockTimestampHex = EthImpl.numberTo0x(Number(blockTimestamp));
const firstTransactionTimestampSeconds = '1653077547';
const firstTransactionTimestampSeconds = '1653077541';
const contractAddress1 = '0x000000000000000000000000000000000000055f';
const htsTokenAddress = '0x0000000000000000000000000000000002dca431';
const contractTimestamp1 = `${firstTransactionTimestampSeconds}.983983199`;
Expand Down Expand Up @@ -431,6 +431,12 @@ describe('Eth calls using MirrorNode', async function () {
"runtime_bytecode": mirrorNodeDeployedBytecode
};

const defaultContract2 = {
...defaultContract,
"address": contractAddress2,
"contract_id": contractId2,
}

const defaultHTSToken =
{
"admin_key": null,
Expand Down Expand Up @@ -1647,6 +1653,32 @@ describe('Eth calls using MirrorNode', async function () {
expectLogData3(result[2]);
});

it('multiple addresses filter', async function () {
const filteredLogsAddress1 = {
logs: [defaultLogs.logs[0], defaultLogs.logs[1], defaultLogs.logs[2]]
};
const filteredLogsAddress2 = {
logs: defaultLogs3
};
mock.onGet("blocks?limit=1&order=desc").reply(200, { blocks: [defaultBlock] });
mock.onGet(`contracts/${contractAddress1}/results/logs?timestamp=gte:${defaultBlock.timestamp.from}&timestamp=lte:${defaultBlock.timestamp.to}&limit=100&order=asc`).reply(200, filteredLogsAddress1);
mock.onGet(`contracts/${contractAddress2}/results/logs?timestamp=gte:${defaultBlock.timestamp.from}&timestamp=lte:${defaultBlock.timestamp.to}&limit=100&order=asc`).reply(200, filteredLogsAddress2);
for (const log of filteredLogsAddress1.logs) {
mock.onGet(`contracts/${log.address}`).reply(200, defaultContract);
}
mock.onGet(`contracts/${contractAddress2}`).reply(200, defaultContract2);

const result = await ethImpl.getLogs(null, null, null, [contractAddress1, contractAddress2], null);

expect(result).to.exist;

expect(result.length).to.eq(4);
expectLogData1(result[0]);
expectLogData2(result[1]);
expectLogData3(result[2]);
expectLogData4(result[3]);
});

it('blockHash filter', async function () {
const filteredLogs = {
logs: [defaultLogs.logs[0], defaultLogs.logs[1]]
Expand Down
8 changes: 4 additions & 4 deletions packages/server/tests/acceptance/index.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ describe('RPC Server Acceptance Tests', function () {
logger.info(`OPERATOR_ID_MAIN: ${process.env.OPERATOR_ID_MAIN}`);
logger.info(`MIRROR_NODE_URL: ${process.env.MIRROR_NODE_URL}`);
logger.info(`E2E_RELAY_HOST: ${process.env.E2E_RELAY_HOST}`);

if (USE_LOCAL_NODE === 'true') {
runLocalHederaNetwork();
}
Expand Down Expand Up @@ -128,12 +128,12 @@ describe('RPC Server Acceptance Tests', function () {
process.env['NETWORK_NODE_IMAGE_TAG'] = '0.32.0-alpha.1';
process.env['HAVEGED_IMAGE_TAG'] = '0.32.0-alpha.1';
process.env['MIRROR_IMAGE_TAG'] = '0.67.3';

console.log(`Docker container versions, services: ${process.env['NETWORK_NODE_IMAGE_TAG']}, mirror: ${process.env['MIRROR_IMAGE_TAG']}`);

console.log('Installing local node...');
shell.exec(`npm install @hashgraph/hedera-local -g`);

console.log('Starting local node...');
shell.exec(`hedera start -d`);
console.log('Hedera Hashgraph local node env started');
Expand Down
42 changes: 34 additions & 8 deletions packages/server/tests/acceptance/rpc.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,17 +116,24 @@ describe('@api RPC Server Acceptance Tests', function () {

describe('eth_getLogs', () => {

let log0Block, log4Block, contractAddress;
let log0Block, log4Block, contractAddress, contractAddress2, latestBlock, tenBlocksBehindLatest, log0, log4, log5;

it('@release should deploy a contract', async () => {
before(async () => {
const logsContract = await servicesNode.deployContract(logsContractJson);

const mirrorNodeResp = await mirrorNode.get(`/contracts/${logsContract.contractId}`, requestId);
expect(mirrorNodeResp).to.have.property('evm_address');
expect(mirrorNodeResp.env_address).to.not.be.null;
contractAddress = mirrorNodeResp.evm_address;

const logsContract2 = await servicesNode.deployContract(logsContractJson);
const mirrorNodeResp2 = await mirrorNode.get(`/contracts/${logsContract2.contractId}`, requestId);
expect(mirrorNodeResp2).to.have.property('evm_address');
expect(mirrorNodeResp2.env_address).to.not.be.null;
contractAddress2 = mirrorNodeResp2.evm_address;

const params = new ContractFunctionParameters().addUint256(1);
const log0 = await accounts[1].client.executeContractCall(logsContract.contractId, 'log0', params, 75000, requestId);
log0 = await accounts[1].client.executeContractCall(logsContract.contractId, 'log0', params, 75000, requestId);
await accounts[1].client.executeContractCall(logsContract.contractId, 'log1', params, 75000, requestId);

params.addUint256(1);
Expand All @@ -136,10 +143,16 @@ describe('@api RPC Server Acceptance Tests', function () {
await accounts[1].client.executeContractCall(logsContract.contractId, 'log3', params, 75000, requestId);

params.addUint256(1);
const log4 = await accounts[1].client.executeContractCall(logsContract.contractId, 'log4', params, 75000, requestId);
log4 = await accounts[1].client.executeContractCall(logsContract.contractId, 'log4', params, 75000, requestId);
log5 = await accounts[1].client.executeContractCall(logsContract2.contractId, 'log4', params, 75000, requestId);

await new Promise(r => setTimeout(r, 5000));
const tenBlocksBehindLatest = Number(await relay.call('eth_blockNumber', [], requestId)) - 10;
latestBlock = Number(await relay.call('eth_blockNumber', [], requestId));
tenBlocksBehindLatest = latestBlock - 10;
});

it('@release should deploy a contract', async () => {

//empty params for get logs defaults to latest block, which doesn't have required logs, that's why we fetch the last 10
const logs = await relay.call('eth_getLogs', [{
fromBlock: tenBlocksBehindLatest
Expand Down Expand Up @@ -209,9 +222,6 @@ describe('@api RPC Server Acceptance Tests', function () {

it('should be able to use `address` param', async () => {
//when we pass only address, it defaults to the latest block
//we fetch last 10 blocks
const tenBlocksBehindLatest = Number(await relay.call('eth_blockNumber', [], requestId)) - 10;

const logs = await relay.call('eth_getLogs', [{
'fromBlock': tenBlocksBehindLatest,
'address': contractAddress
Expand All @@ -223,6 +233,22 @@ describe('@api RPC Server Acceptance Tests', function () {
}
});

it('should be able to use `address` param with multiple addresses', async () => {
const logs = await relay.call('eth_getLogs', [{
'fromBlock': tenBlocksBehindLatest,
'address': [contractAddress, contractAddress2, NON_EXISTING_ADDRESS]
}], requestId);
expect(logs.length).to.be.greaterThan(0);
expect(logs.length).to.be.eq(6);

for (let i = 0; i < 5; i++) {
expect(logs[i].address).to.equal(contractAddress);
}

expect(logs[5].address).to.equal(contractAddress2);
});


it('should be able to use `blockHash` param', async () => {
const logs = await relay.call('eth_getLogs', [{
'blockHash': log0Block.blockHash
Expand Down