Lock the epoch length in a devchain using mainnet data

[phroi]

Feature Request

Background

Currently I'm working on rescuing dCKB deposits by writing a new web interface. Given the lack of documentation, it seems impossible to re-deploy correctly those contracts on a brand new devnet.

Is your feature request related to a problem? Please describe.

Following @doitian instructions, I'm running a devchain using existing mainnet data.

Now I want to speed up this devnet to test dCKB and NervosDAO interactions, so I followed the steps previously suggested, except for changing the genesis_epoch_length as indicated.

The issue is the epoch length doesn't seem much affected by them, as currently one epoch is more than 16 minutes.

To fully reproduce this issue follow the steps indicated on the Readme.

Describe the solution you'd like

@doitian suggested:

It seems we need a feature to lock the epoch length since a starting epoch for dev environment.

Keep up the great work 💪
Phroi

[eval-exec]

as currently one epoch is more than 16 minutes.

Hello, have you tried speeding up the ckb miner?

You can achieve this by setting lower values for miner.client.poll_interval and miner.workers.value in ckb-miner.toml file , ensuring that miner.client.poll_interval is smaller than miner.workers.value:

[miner.client]
rpc_url = "http://127.0.0.1:8114/"
block_on_submit = true

# block template polling interval in milliseconds
poll_interval = 50

# enable listen notify mode
# listen = "127.0.0.1:8888"

[[miner.workers]]
worker_type = "Dummy"
delay_type = "Constant"
value = 100

[phroi]

@eval-exec the relevant part of ckb-miner.toml was like this:

[miner.client]
rpc_url = "http://127.0.0.1:8114/"
block_on_submit = true

# block template polling interval in milliseconds
poll_interval = 1000

# enable listen notify mode
# listen = "127.0.0.1:8888"

[[miner.workers]]
worker_type = "Dummy"
delay_type = "Constant"
value = 200 # instead of 5000

[phroi]

Now using @eval-exec ckb-miner.toml parameters one epoch is down to 4 minutes 🥳

Still very different from the parameter set in dev.toml: epoch_duration_target = 2`

[eval-exec]

@phroi I read the v1-interface's README.md, learned how you start a CKB node.

The ckb node is initialized with data from the mainnet, and in this context, setting epoch_duration_target = 2 does not affect epoch duration when permanent_difficulty_in_dummy is enabled.

[phroi]

I just tried modifying permanent_difficulty_in_dummy = false and running the following command for a few epochs

(trap 'kill -INT 0' SIGINT; cd ~/ckb_dev/; ./ckb run --skip-spec-check --overwrite-spec --indexer & sleep 1 && ./ckb miner)

The epoch length keeps being of constant length 1800 🤔

[phroi]

Let me know if you want to try something else 🤗

[eval-exec]

@phroi Before you run the command ckb init -c dev --import-spec /path/to/downloaded/mainnet.toml --force, what is the tip header number or hash of your previous mainnet node?

You can get this from the tail of data/logs/run.log file.

[phroi]

I copied the data folder from my Neuron wallet and probably I haven't open it since then 🤔

This likely is the old tip header information:

{
  compactTarget: '0x1908d8ab',
  parentHash: '0xe85526864dd125841e4a416f842e63bbb879c7b2f453ab237cabb69178a568a5',
  transactionsRoot: '0xeff28f2c3ae8997b0c75f4ed1af1445c7895116b3577b9e3edd6369c43b38b06',
  proposalsHash: '0x6853de88d71a10a555e1f7569c738c075c0a8d888db9d7bb37fd528510ebcdeb',
  extraHash: '0x0000000000000000000000000000000000000000000000000000000000000000',
  dao: '0x17f4c23e34d11b4b4ef83f2d25e127002b403a0b9d5577050075de52a0c93307',
  epoch: '0x45f01e5001faa',
  hash: '0xe933e9b52b2c2c56dff489e45b537e7933a43baafae47ca2e6000d91049441cc',
  nonce: '0xdaa23c3d733da1bb00000001ed200209',
  number: '0xa1c649',
  timestamp: '0x189b685192f',
  version: '0x0'
}

Parsing the epoch field:
length = 1119
index = 485
number = 8106

The issue is that epoch length now is 1800, but it was 1119 🤔 🤔 🤔

Side note: when importing the data into the devchain, I was asked to update the database to the new version.

[eval-exec]

The issue is that epoch length now is 1800, but it was 1119 🤔 🤔 🤔

Thank you, I'm trying to reproduce this.

[eval-exec]

I've discovered that the maximum value for epoch length is 1800. I am currently investigating the reasons behind this.

ckb/spec/src/consensus.rs

Line 74 in 0a3f9b1

const MAX_EPOCH_LENGTH: u64 = DEFAULT_EPOCH_DURATION_TARGET / MIN_BLOCK_INTERVAL; // 1800

[phroi]

If it helps solving the issue, I can sync Neuron and we can redo the full procedure together, so we are 100% sure that we are both seeing the same data 🤗

[eval-exec]

If it helps solving the issue, I can sync Neuron

Sure. I'm syncing my node to latest tip too.

[eval-exec]

I just tried modifying permanent_difficulty_in_dummy = false and running the following command for a few epochs

(trap 'kill -INT 0' SIGINT; cd ~/ckb_dev/; ./ckb run --skip-spec-check --overwrite-spec --indexer & sleep 1 && ./ckb miner)

The epoch length keeps being of constant length 1800 thinking

This behavior aligns with our expectations. When permanent_difficulty_in_dummy is set to false, CKB's consensus mechanism will dynamically adjust the epoch length and the mining difficulty based on the orphan rate:

ckb/spec/src/consensus.rs

Lines 847 to 852 in 0a3f9b1

	let (next_epoch_length, bound) = if epoch_uncles_count == 0 {
	(
	cmp::min(self.max_epoch_length(), epoch.length() * TAU),
	true,
	)
	} else {

[phroi]

If it helps solving the issue, I can sync Neuron

Sure. I'm syncing my node to latest tip too.

It'll take hours for me:

1. I have a slow connection right now
2. I first have to update to the latest Neuron wallet 110.2 (this avoids migrating the database later on)
3. I'll update the blockchain data
4. When I have the previous steps ready, likely tomorrow, I'll let you know

If you have any other question, feel free to ask 🤗

[phroi]

@eval-exec whenever you want I'm ready, also I have updated the node to ckb v0.110.0 (Portable)

[eval-exec]

I just tried modifying permanent_difficulty_in_dummy = false ...
The epoch length keeps being of constant length 1800

@phroi I have reproduced successfully, and my previous mainnet node's tip header is 10668735. After ckb miner running for a while, epoch length keeps being of constant 1800, this is in line with expectations due to CKB's consensus mechanism.

[phroi]

#4097 seems exactly what I had in mind 🥳

[eval-exec]

#4097 seems exactly what I had in mind partying_face

If you'd like to give it a try, you can compile this branch of the CKB node yourself by make prod, then you'll get ./target/prod/ckb.

[eval-exec]

@phroi

I compiled ckb by #4097, and specify epoch_duration_target = 80 in specs/dev.toml,

In v1-interface: npm run start completes in 73 seconds:

This is my specs/dev.toml :

[params]
genesis_epoch_length = 1743
# Other parameters...
initial_primary_epoch_reward = 1_917_808_21917808
secondary_epoch_reward = 613_698_63013698
max_block_cycles = 10_000_000_000
cellbase_maturity = 0
primary_epoch_reward_halving_interval = 8760
epoch_duration_target = 80 # instead of 14400
permanent_difficulty_in_dummy = true

[pow]
func = "Dummy"

and ckb-miner.toml:

[miner.client]
rpc_url = "http://127.0.0.1:8114/"
block_on_submit = true

# block template polling interval in milliseconds
poll_interval = 10

# enable listen notify mode
# listen = "127.0.0.1:8888"

[[miner.workers]]
worker_type = "Dummy"
delay_type = "Constant"
value = 10

[phroi]

I tried to reproduce all steps from zero, but ckb node kept crashing. So I standardized and repeated all steps again and again until I remembered that on this old laptop I have to use ckb Portable, so make prod_portable... Understood that, it was smooth ride!! 💪

I confirm that #4097 solves this issue, feel free to close this issue when the PR gets merged!

Thanks for your support @eval-exec 🤗
Phroi

[doitian]

A new RPC method will be added to ease generating an epoch: #4128