Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Add disable for batch request limit #682

Merged
merged 5 commits into from
Aug 17, 2022

Conversation

ZeljkoBenovic
Copy link
Contributor

Description

This PR adds the ability to disable json-rpc batch request limits, introduced in #638.

Some tools, like Blockscout are using batch requests in order to efficiently gather data from the chain.
With current default limits, Blockscout in production, is not working as it is hitting this limits.
Also, this protection is not needed for nodes that are not interacting with general public, for example a dedicated non-validator node that will serve only Blockscout.

To disable this limits just set the values to 0:
--json-rpc-block-range-limit 0 and
--json-rpc-batch-request-limit 0

Also, this PR fixes the issue that these values could not be fed trough config file. Right now, they can be set only with flags.

Changes include

  • Bugfix (non-breaking change that solves an issue)
  • Hotfix (change that solves an urgent issue, and requires immediate attention)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (change that is not backwards-compatible and/or changes current functionality)

Checklist

  • I have assigned this PR to myself
  • I have added at least 1 reviewer
  • I have added the relevant labels
  • I have updated the official documentation
  • I have added sufficient documentation in code

Testing

  • I have tested this code with the official test suite
  • I have tested this code manually

Manual tests

Set the flags --json-rpc-block-range-limit 0 , --json-rpc-batch-request-limit 0 and you should not see Batch request length too long error

@codecov
Copy link

codecov bot commented Aug 11, 2022

Codecov Report

Merging #682 (2278acb) into develop (fb219ba) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop     #682   +/-   ##
========================================
  Coverage    48.60%   48.60%           
========================================
  Files          112      112           
  Lines        15381    15382    +1     
========================================
+ Hits          7476     7477    +1     
  Misses        7271     7271           
  Partials       634      634           
Impacted Files Coverage Δ
jsonrpc/dispatcher.go 53.82% <100.00%> (+0.15%) ⬆️
jsonrpc/filter_manager.go 76.96% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@0xAleksaOpacic 0xAleksaOpacic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an issue with this PR, i will take some time today to debug it:

The scenario that does not work:

  1. change hardcoded DefaultJSONRPCBlockRangeLimit to 50 (just so you don't need to wait for 1k blocks)
  2. start 4 node network wait til 50+ blocks
  3. trigger getLogs jsonrpc with (from: 0, to: latest), i am expecting ErrBlockRangeTooHigh but call passes

This issue was introduced with adding initJSONRPCBulkLimits.

if p.rawConfig.JSONRPCBatchRequestLimit != config.DefaultJSONRPCBatchRequestLimit {
		p.jsonRPCBatchLengthLimit = p.rawConfig.JSONRPCBatchRequestLimit
	}

if p.rawConfig.JSONRPCBlockRangeLimit != config.DefaultJSONRPCBlockRangeLimit {
		p.jsonRPCBlockRangeLimit = p.rawConfig.JSONRPCBlockRangeLimit
	}

this method is called after we setFlags. p.rawConfig.JSONRPCBatchRequestLimit will always be 0 because the value is defined in p.JSONRPCBatchRequestLimit. Because 0 is != config.DefaultJSONRPCBlockRangeLimit we are going to override p.jsonRPCBlockRangeLimit with 0. And with the functionality, you've added 0 means disabled.

Copy link
Contributor

@zivkovicmilos zivkovicmilos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me 💯

Please see @Aleksao998's comment on removing that method, and we're good to go.

@0xAleksaOpacic
Copy link
Contributor

0xAleksaOpacic commented Aug 15, 2022

Looks ok to me 💯

Please see @Aleksao998's comment on removing that method, and we're good to go.

Quick question, why we have to have jsonRPCBatchLengthLimit/jsonRPCBlockRangeLimit in 2 places.

type serverParams struct {

           rawConfig{
                JSONRPCBatchRequestLimit
                JSONRPCBlockRangeLimit
           }
           jsonRPCBatchLengthLimit
           jsonRPCBlockRangeLimit

}

@ZeljkoBenovic
Copy link
Contributor Author

You were right @Aleksao998 ,

I've removed the method that doesn't do anything and I've removed redundant variables from the PR that introduced this functionality.

I've also added a small test, to verify that when set to 0, there are no request limits.

Copy link
Contributor

@0xAleksaOpacic 0xAleksaOpacic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for fixing all comments. Double checked everything it works great now!

@ZeljkoBenovic ZeljkoBenovic merged commit de62840 into develop Aug 17, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Aug 17, 2022
@ZeljkoBenovic ZeljkoBenovic deleted the fix/add-disable-batch-request-limit branch August 22, 2022 16:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug fix Functionality that fixes a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants