From 66b006755f840df1c37b224f45095a03c1e90d63 Mon Sep 17 00:00:00 2001 From: Travis Person Date: Tue, 2 Mar 2021 17:27:11 +0000 Subject: [PATCH 1/8] Draft doc on how to setup a network --- NETWORK_SETUP.md | 443 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 443 insertions(+) create mode 100644 NETWORK_SETUP.md diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md new file mode 100644 index 000000000..78dc6e4e2 --- /dev/null +++ b/NETWORK_SETUP.md @@ -0,0 +1,443 @@ +Note: This entire document needs to be updated to support the new network build flags for lotus + +# Setting up a Filecoin network + +Starting a Filecoin network using Lotus has a few steps which I will outline below, these steps will be the rather +verbose way to setup a network which tries to cover everything. Certain steps may not be required, but providing them all +gives the most flexibility and outlines the way we setup networks. + +## Introduction + +Starting a Filecoin network requires some special setup. To produce blocks on the Filecoin network, and therefore process +messages, a miner on the network must exist with power. For the start of the network this power is given to a miner during +the creation of the genesis. The power given to the miner isn't just a number, the miner is still required to +submit Window PoSt proofs as well as Winning PoSts for block production, there is no special casing for these miners. (The sectors +can be special in one way though, because we can freely construct the genesis there is no pre-commit process). +This requires that sectors exist. The process of creating these sectors before the network is created is called pre-sealing, +and is done through the lotus-seed tool. More than one miner can be created during this process. + +## Basic steps + +0. Preseal Sectors +0. Generate Genesis +0. Initialize Genesis Miner +0. Start Genesis Miner +0. Initialize all other miners +0. Start all other miners + +Binaries required: +- jq +- lotus-shed +- lotus-seed +- lotus +- lotus-miner + +## Lotus Shed + +The `lotus-shed` binary is where lotus keeps all of our tool programs. + +### Generating bls keys for pre-sealing sectors + +Creates a file called `bls-
.keyinfo` which will contain a keyinfo json object. + +``` +lotus-shed keyinfo new bls +``` + +### Generating libp2p ed25519 peer key + +This will create a file called `libp2p-host-
.keyinfo` which will contain a keyinfo json object. + +``` +lotus-shed keyinfo new libp2p +``` + +### Importing keys to lotus repository + +Note: Be sure to set the `LOTUS_PATH` before running the command if using a non standard location. + +``` +lotus-shed keyinfo import .keyinfo +``` + +### Downloading proof params + +The lotus binary can also be used to download these, but to reduce the number of binaries that need be moved around +the command has also been added to `lotus-shed`. + +``` +lotus-shed fetch-params --proving-params 0 +lotus-shed fetch-params --proving-params +``` + +## Lotus Seed + +The `lotus-seed` binary is used to generate pre-sealed sectors for network bootstrap, as well as +constructing the network json configuration use to generate the genesis. + +Note: As long as the replication construct of proofs do not change, keys stay secret, and the pre-sealed sectors +meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reused between networks. + +### Pre-sealing sectors + +For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed. + +``` +lotus-seed pre-seal --miner-addr t01000 --key ./pre-seal-t01000.key --sector-size --num-sectors 4 --sector-offset 0 +``` + +The pre-sealing process can be run in parallel by suppling proper `--sector-offset` values. In a difference process the +output of this process can be merged together to create a single metadata file for the miner. + +``` +lotus-seed --sector-dir /storage//pre-seal-0 pre-seal ... --num-sectors 4 --sector-offset 0 +lotus-seed --sector-dir /storage//pre-seal-1 pre-seal ... --num-sectors 4 --sector-offset 4 +lotus-seed --sector-dir /storage//pre-seal-2 pre-seal ... --num-sectors 4 --sector-offset 8 +``` + +Note: The + +Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing +using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However, +the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by +editing the `pre-seal-.json` files, or better yet, waiting till after running the `aggregate-manifests` command which +will produce a single file making it easier to edit the owner and worker keys. + +``` +cat pre-seal-t01000.json +{ + "t01000": { + "ID": "t01000", + "Owner": "t3qbp2kex6ljigeezzceeq4qoqyznshoekvfswpje4sqy36t6xbbb3fyzxlneftmy57bnj7kezc3ceqjm5mk5a", + "Worker": "t3qbp2kex6ljigeezzceeq4qoqyznshoekvfswpje4sqy36t6xbbb3fyzxlneftmy57bnj7kezc3ceqjm5mk5a", + "PeerId": "12D3KooWSsGT5Ky7EgAPcpKmGkKWq2fWSfR2CznSHrHLihiCQ1UR", + "MarketBalance": "0", + "PowerBalance": "0", + "SectorSize": 2048, + "Sectors": [...] +} +``` + +TODO: There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config? + +NOTE: pre-seal does not generate t_aux files which are required to exist along side every p_aux file. You will have to add these later +TODO: THere should be a command to do this or pre-seal should do it automatically. + +### Merging pre-seal metadata of parallel pre-seals + +If run a parallel pre-seal, sector metadata will need to be merged (`pre-seal-.json`) together into a single +file. This file will then be used to construct the network configuration file. + +``` +lotus-seed aggregate-manifests \ + /storage//pre-seal-0/pre-seal-.json \ + /storage//pre-seal-1/pre-seal-.json \ + /storage//pre-seal-3/pre-seal-.json > /storage//pre-seal-.json +``` + +NOTE: Each of these lines updates some value, the first 3 lines (starting with the `$MinerId`) are probably not needed +``` +/usr/local/bin/lotus-seed aggregate-manifests {{ lotus_miner_presealed_sectors | product([metadatafile]) | map('join', '/') | join(' ') }} \ +| jq --arg Addr "{{ lotus_miner_wallet_address }}" --arg MinerId "{{ lotus_miner_addr }}" --arg VerifiedDeal "{{ lotus_miner_verified_deals | lower }}" ' + .[$MinerId].Owner = $Addr + | .[$MinerId].Worker = $Addr + | .[$MinerId].ID = $MinerId + | .[$MinerId].Sectors[].Deal.Client = $Addr + | .[$MinerId].Sectors[].Deal.VerifiedDeal = ($VerifiedDeal == "true") + | .[$MinerId].Sectors[] |= (.Deal.Label = .CommR."/") +' > "{{ lotus_miner_presealed_metadata }}" +``` + +### Generate network configuration + +Every network requires a genesis file to initialize from. This section will cover the generation of the configuration +which will later be used to generate the `genesis.car` file itself. + +``` +lotus-seed genesis new --network-name /storage//genesis.json +``` + +#### Setting a network start time + +If a genesis timestamp is not specified, the current time when the genesis is created will be used. A timestamp should +be defined to reduce the network mining race effect. + +The `Timestamp` in the genesis configuration is a unix epoch. + +TODO: This should be a command in lotus-seed + +``` +# GENESISDELAY is a time in seconds added to the current time to delay the network start by some amount of time +GENESISDELAY=3600 + +GENESISTMP=$(mktemp) +GENESISTIMESTAMP=$(date --utc +%FT%H:%M:00Z) +TIMESTAMP=$(echo $(date -d ${GENESISTIMESTAMP} +%s) + ${GENESISDELAY} | bc) + +jq --arg Timestamp ${TIMESTAMP} \ + ' . + { Timestamp: $Timestamp|tonumber } ' \ + < "/storage//genesis.json" > ${GENESISTMP} +mv ${GENESISTMP} "/storage//genesis.json" +``` + +#### Adding miners + +For each miner with pre-sealed sectors that should be included in the network will need to have their metadata file +added to the genesis. + +``` +lotus-seed genesis add-miner /storage//genesis.json /storage//pre-seal-.json +``` + +#### Adding additional accounts + +Any additional accounts which should have preallocated funds in the network can be added by updating the `Accounts` +field. + +You will want to keep this amount of funds preallocated through additional accounts to a minimum as all funds in these +accounts will count towards the circulating supply. Having a high ciruclating supply will result in high sector pledge +fees. For this reason, it's better to allocated all of the funds to a single actors, and set that actor to the account ID +of 90, which will not be counted towards the ciruclating supply. For public test networks, this account would be called the +faucet. + +TODO: This should be a command in lotus-seed to add accounts +TODO: There needs to a way to set the actor ID 90 account. + +``` +# OWNER is a wallet address, use lotus-shed keyinfo --format '{{ .Address }}' .keyinfo +OWNER="" + +# BALANCE is the amount of AttoFIL (10^-18 FIL) which will be preallocated to the account (wallet) +# 1 FIL == 1000000000000000000 AttoFIL +BALANCE= + +GENESISTMP=$(mktemp) + +jq --arg Owner ${OWNER} --arg Balance ${BALANCE} \ + '.Accounts |= . + [{Type: "account", Balance: $Balance, Meta: {Owner: $Owner}}]' \ + < "/storage//genesis.json" > ${GENESISTMP} +mv ${GENESISTMP} "/storage//genesis.json" +``` + +### Generating the genesis.car + +To generate the genesis car file, starting a lotus daemon is required. The `genesis.car` file will be created along side +the `genesis.json` file at `/storage//genesis.car` (this is a detail of this script). + +``` +GENESIS_JSON="/storage//genesis.json" +GENPATH=$(mktemp --suffix=genesis -d) + +cp "${GENESIS_JSON}" "${GENPATH}/genesis.json" + +lotus-seed genesis car \ + --out="${GENPATH}/devnet.car" \ + "${GENPATH}/genesis.json" + +cp ${GENPATH}/devnet.car $(dirname $GENESIS_JSON)/genesis.car +rm -rf ${GENPATH} +``` + +## Running bootstrap nodes + +It's a good idea to run dedicated bootstrap nodes that are not the pre-sealed miners themselves. This is less important +on smaller networks, but on larger ones the bootstrap peers received so many connections that they cycle through peer +lists rather quickly which may affect block / message propagation to and from the miners. + +Bootstrap nodes are just lotus daemons, ideally with a pre-generated libp2p host key so that redeployment is possible +without having to distribute new multiaddrs. Bootstrap nodes must all be publicly dial-able and have a static port, as +lotus by default uses a random port assigned by the kernel. + +To compile bootstrap multiaddrs into the `lotus` binary, place the multiaddrs in `./build/bootstrap/bootstrappers.pi`. + +Bootstrap nodes should also run the bootstrap profile by specifying it on the daemon commands. + +``` +lotus daemon --profile=bootstrapper +``` + +Optically you may want to increase the `ConnMgrLow` and `ConnMgrHigh` values to increase. + +The `config.toml` should be located under `$LOTUS_PATH/config.toml`. + +``` +[Libp2p] +ListenAddresses = ["/ip4/0.0.0.0/tcp/1347"] +# ConnMgrLow = +# ConnMgrHigh = +``` + +Be sure to open any firewall rules. Here is an example ufw profile: + +``` +[Lotus Daemon] +title=Lotus Daemon +description=Lotus daemon firewall rules +ports=1347/tcp +``` + +### Tipset Thresholds + +Before lotus will start syncing a tipset, lotus requires that is has seen the tipset from a certain number of peers. +This value is defined by the build constant `BootstrapPeerThreshold`. When this value is set to any number greather +than one (1), starting a network runs into issues. This is because at the beggining of the network, there is only a +single peer that actually is producing blocks and sharing the tipset. Therefore no peers will try to sync the tipset. + +To resolve this issue, the environment var `LOTUS_SYNC_BOOTSTRAP_PEERS` needs to be set to `1` on at least a number of peers +equal to the value of `BootstrapPeerThreshold`. + +See https://github.com/filecoin-project/lotus/issues/5474 for more information. + +## Initializing pre-sealed miners + +Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start +a network, but all pre-sealed miners for the network should be initialized together. It's not strictly required though. + +There are no special flags we pass to the lotus daemon for these miners, besides the `--genesis` flag. + +One miner must be marked the genesis miner.The genesis miner is the first miner to run and will mine the initial blocks +required for initialization of all other. Technically the other miners can be initialized after the genesis miner is +actually running the chain the chain is progressing. However, we always initialize them together and allow for a full +network initialization before actually running the miner. + +For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using three +directories. + +Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a +`lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap +nodes and compile their addresses into the lotus daemon using the `build/bootstrap/bootstrappers.pi`. + +After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`. + +### Genesis miner + +Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner + +``` +lotus-storage-miner init --actor t01000 \ + --sector-size \ + --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \ + --pre-sealed-sectors /storage/t01000/pre-seal-0 \ + --pre-sealed-sectors /storage/t01000/pre-seal-1 \ + --pre-sealed-sectors /storage/t01000/pre-seal-2 \ + --nosync --genesis-miner +``` + +### Other miners + +Other miners have a similar initialization process, they just should not specify the `--genesis-miner` flag. This will +stop them from forking the network immediately by trying to mine their own chain. + +``` +lotus-storage-miner init --actor \ + --sector-size \ + --pre-sealed-metadata /storage//pre-seal-.json \ + --pre-sealed-sectors /storage//pre-seal-0 \ + --pre-sealed-sectors /storage//pre-seal-1 \ + --pre-sealed-sectors /storage//pre-seal-2 \ + --nosync +``` + +## Other network configuration + +Not all network configuration is defined in the genesis itself. Some of these values are defined in varies policy files +of the specs-actors code itself. + +Most notability are the supported sector sizes, minimum miner power, and block time. Currently these should be changed +by editing `build/params_testnet.go` of the lotus source code. + +------------------------------ + +### Verified Registery + +The Verified Registery can be interactived through the lotus-shed tool using signed messages. The address is set during +genesis creation. To be able to use the verified registery on a network, generate a key and replace the `Signers` array with +the address. + +``` +{ + "Accounts": [], + "Miners": [], + "NetworkName": "localnet-e415afe6-f7ed-402f-a6b9-480f44064561", + "VerifregRootKey": { + "Type": "multisig", + "Balance": "0", + "Meta": { + "Signers": [ + "t1ceb34gnsc6qk5dt6n7xg6ycwzasjhbxm3iylkiy" + ], + "Threshold": 1, + "VestingDuration": 0, + "VestingStart": 0 + } + } +} +``` + +### Limits on number of sectors + +There is an upper bound to the number of sectors a miner can have. This doesn't really affect 32/64 sector miners, but +the smaller test sectors have a limit of 2 sectors per partition and because lotus submits all parittions as a single +message this puts an upper bound of 5 partitions due to the proof size being 192 bytes and the maximum param size of +1024 bytes + +https://github.com/filecoin-project/specs-actors/blob/v0.9.13/actors/builtin/sector.go#L83-L100 +https://github.com/filecoin-project/specs-actors/blob/v3.0.0/actors/builtin/miner/policy.go#L96-L97 + +## Generating easily storable and reusable sectors + +The following is a script which generates bucketed sectors which can easily be shared publicly and resused many times. +Sectors used in this way require that miner owner and worker keys are changed! + +This script also truncates the sectors and requires that they are fixed after downloading. Bring the sectors back to their +correct size can be done using `truncate -s `. + +``` +#!/usr/bin/bash + +set -xe + +MINERS=(t01000 t01001 t01002 t01003 t01004 t01005) +PROOF_VERSION="v28" +SECTOR_SIZE="512MiB" +NUM_SECTORS=32 +SECTOR_OFFSET=0 +PATH_PREFIX="/storage/" +ROUNDS=$(seq 0 31) + +export IPFS_GATEWAY="http://localhost:8080/ipfs/" +export FIL_PROOFS_USE_GPU_TREE_BUILDER=1 +export FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1 +export RUST_LOG=info +export GOLOG_LOG_LEVEL=warn +export RUST_BACKTRACE=1 + +for MINER_ADDR in ${MINERS[@]}; do + rm -rf "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}" + mkdir -p "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}" + + for i in ${ROUNDS}; do + SECTOR_OFFSET=$((i * NUM_SECTORS)) + SECTOR_OFFSET_VANITY=$(printf "%03d\n" ${SECTOR_OFFSET}) + + mkdir -p "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}" + + lotus-seed --sector-dir "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}" \ + pre-seal \ + --fake-sectors \ + --miner-addr ${MINER_ADDR} \ + --sector-size ${SECTOR_SIZE} \ + --num-sectors ${NUM_SECTORS} \ + --sector-offset ${SECTOR_OFFSET} \ + 2>&1 | tee -a ${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}/pre-seal.log + + for i in $(ls "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}/sealed/"); do + truncate -s 0 "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}/sealed/${i}" + truncate -s 0 "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}/cache/${i}/t_aux" + done + + tree -Fifs "${PATH_PREFIX}/${PROOF_VERSION}/${SECTOR_SIZE}/${MINER_ADDR}/${SECTOR_OFFSET_VANITY}" + + done +done +``` From dc07e82d8a1bae3b46141238af5ca41b5a24e1a0 Mon Sep 17 00:00:00 2001 From: ognots Date: Fri, 16 Apr 2021 10:06:28 -0400 Subject: [PATCH 2/8] re-arrange document in to basic and advanaced usage basic example targets starting the Nerpa network with the minimum pieces required for a public devnet all other information is organized under an advanaced usage section --- NETWORK_SETUP.md | 224 +++++++++++++++++++++++++---------------------- 1 file changed, 119 insertions(+), 105 deletions(-) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index 78dc6e4e2..962acc2d5 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -11,10 +11,7 @@ gives the most flexibility and outlines the way we setup networks. Starting a Filecoin network requires some special setup. To produce blocks on the Filecoin network, and therefore process messages, a miner on the network must exist with power. For the start of the network this power is given to a miner during the creation of the genesis. The power given to the miner isn't just a number, the miner is still required to -submit Window PoSt proofs as well as Winning PoSts for block production, there is no special casing for these miners. (The sectors -can be special in one way though, because we can freely construct the genesis there is no pre-commit process). -This requires that sectors exist. The process of creating these sectors before the network is created is called pre-sealing, -and is done through the lotus-seed tool. More than one miner can be created during this process. +submit Window PoSt proofs as well as Winning PoSts for block production, there is no special casing for these miners. ## Basic steps @@ -32,6 +29,16 @@ Binaries required: - lotus - lotus-miner +# Simple Example + +This example documents how to set up the Nerpa Network with a simple network configuration. + +A minimally viable Lotus network is comprised of: +* Lotus "miner" with presealed sectors +* Lotus "bootstrap" node +* Genesis block +* Faucet + ## Lotus Shed The `lotus-shed` binary is where lotus keeps all of our tool programs. @@ -72,6 +79,10 @@ lotus-shed fetch-params --proving-params ## Lotus Seed +(The sectors can be special in one way though, because we can freely construct the genesis there is no pre-commit process). +This requires that sectors exist. The process of creating these sectors before the network is created is called pre-sealing, +and is done through the `lotus-seed` tool. + The `lotus-seed` binary is used to generate pre-sealed sectors for network bootstrap, as well as constructing the network json configuration use to generate the genesis. @@ -86,16 +97,6 @@ For each miner wanted during the initial setup of a new network, sectors need to lotus-seed pre-seal --miner-addr t01000 --key ./pre-seal-t01000.key --sector-size --num-sectors 4 --sector-offset 0 ``` -The pre-sealing process can be run in parallel by suppling proper `--sector-offset` values. In a difference process the -output of this process can be merged together to create a single metadata file for the miner. - -``` -lotus-seed --sector-dir /storage//pre-seal-0 pre-seal ... --num-sectors 4 --sector-offset 0 -lotus-seed --sector-dir /storage//pre-seal-1 pre-seal ... --num-sectors 4 --sector-offset 4 -lotus-seed --sector-dir /storage//pre-seal-2 pre-seal ... --num-sectors 4 --sector-offset 8 -``` - -Note: The Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However, @@ -118,35 +119,11 @@ cat pre-seal-t01000.json } ``` -TODO: There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config? +[TODO]: <> (There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config?) NOTE: pre-seal does not generate t_aux files which are required to exist along side every p_aux file. You will have to add these later -TODO: THere should be a command to do this or pre-seal should do it automatically. -### Merging pre-seal metadata of parallel pre-seals - -If run a parallel pre-seal, sector metadata will need to be merged (`pre-seal-.json`) together into a single -file. This file will then be used to construct the network configuration file. - -``` -lotus-seed aggregate-manifests \ - /storage//pre-seal-0/pre-seal-.json \ - /storage//pre-seal-1/pre-seal-.json \ - /storage//pre-seal-3/pre-seal-.json > /storage//pre-seal-.json -``` - -NOTE: Each of these lines updates some value, the first 3 lines (starting with the `$MinerId`) are probably not needed -``` -/usr/local/bin/lotus-seed aggregate-manifests {{ lotus_miner_presealed_sectors | product([metadatafile]) | map('join', '/') | join(' ') }} \ -| jq --arg Addr "{{ lotus_miner_wallet_address }}" --arg MinerId "{{ lotus_miner_addr }}" --arg VerifiedDeal "{{ lotus_miner_verified_deals | lower }}" ' - .[$MinerId].Owner = $Addr - | .[$MinerId].Worker = $Addr - | .[$MinerId].ID = $MinerId - | .[$MinerId].Sectors[].Deal.Client = $Addr - | .[$MinerId].Sectors[].Deal.VerifiedDeal = ($VerifiedDeal == "true") - | .[$MinerId].Sectors[] |= (.Deal.Label = .CommR."/") -' > "{{ lotus_miner_presealed_metadata }}" -``` +[TODO]: <> (There should be a command to do this or pre-seal should do it automatically.) ### Generate network configuration @@ -164,7 +141,7 @@ be defined to reduce the network mining race effect. The `Timestamp` in the genesis configuration is a unix epoch. -TODO: This should be a command in lotus-seed +[TODO]: <> (This should be a command in lotus-seed) ``` # GENESISDELAY is a time in seconds added to the current time to delay the network start by some amount of time @@ -189,36 +166,6 @@ added to the genesis. lotus-seed genesis add-miner /storage//genesis.json /storage//pre-seal-.json ``` -#### Adding additional accounts - -Any additional accounts which should have preallocated funds in the network can be added by updating the `Accounts` -field. - -You will want to keep this amount of funds preallocated through additional accounts to a minimum as all funds in these -accounts will count towards the circulating supply. Having a high ciruclating supply will result in high sector pledge -fees. For this reason, it's better to allocated all of the funds to a single actors, and set that actor to the account ID -of 90, which will not be counted towards the ciruclating supply. For public test networks, this account would be called the -faucet. - -TODO: This should be a command in lotus-seed to add accounts -TODO: There needs to a way to set the actor ID 90 account. - -``` -# OWNER is a wallet address, use lotus-shed keyinfo --format '{{ .Address }}' .keyinfo -OWNER="" - -# BALANCE is the amount of AttoFIL (10^-18 FIL) which will be preallocated to the account (wallet) -# 1 FIL == 1000000000000000000 AttoFIL -BALANCE= - -GENESISTMP=$(mktemp) - -jq --arg Owner ${OWNER} --arg Balance ${BALANCE} \ - '.Accounts |= . + [{Type: "account", Balance: $Balance, Meta: {Owner: $Owner}}]' \ - < "/storage//genesis.json" > ${GENESISTMP} -mv ${GENESISTMP} "/storage//genesis.json" -``` - ### Generating the genesis.car To generate the genesis car file, starting a lotus daemon is required. The `genesis.car` file will be created along side @@ -256,7 +203,7 @@ Bootstrap nodes should also run the bootstrap profile by specifying it on the da lotus daemon --profile=bootstrapper ``` -Optically you may want to increase the `ConnMgrLow` and `ConnMgrHigh` values to increase. +Optimally you may want to increase the `ConnMgrLow` and `ConnMgrHigh` values. The `config.toml` should be located under `$LOTUS_PATH/config.toml`. @@ -276,18 +223,6 @@ description=Lotus daemon firewall rules ports=1347/tcp ``` -### Tipset Thresholds - -Before lotus will start syncing a tipset, lotus requires that is has seen the tipset from a certain number of peers. -This value is defined by the build constant `BootstrapPeerThreshold`. When this value is set to any number greather -than one (1), starting a network runs into issues. This is because at the beggining of the network, there is only a -single peer that actually is producing blocks and sharing the tipset. Therefore no peers will try to sync the tipset. - -To resolve this issue, the environment var `LOTUS_SYNC_BOOTSTRAP_PEERS` needs to be set to `1` on at least a number of peers -equal to the value of `BootstrapPeerThreshold`. - -See https://github.com/filecoin-project/lotus/issues/5474 for more information. - ## Initializing pre-sealed miners Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start @@ -300,8 +235,8 @@ required for initialization of all other. Technically the other miners can be in actually running the chain the chain is progressing. However, we always initialize them together and allow for a full network initialization before actually running the miner. -For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using three -directories. +For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using one +directory. Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a `lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap @@ -318,25 +253,10 @@ lotus-storage-miner init --actor t01000 --sector-size \ --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \ --pre-sealed-sectors /storage/t01000/pre-seal-0 \ - --pre-sealed-sectors /storage/t01000/pre-seal-1 \ - --pre-sealed-sectors /storage/t01000/pre-seal-2 \ --nosync --genesis-miner ``` -### Other miners - -Other miners have a similar initialization process, they just should not specify the `--genesis-miner` flag. This will -stop them from forking the network immediately by trying to mine their own chain. - -``` -lotus-storage-miner init --actor \ - --sector-size \ - --pre-sealed-metadata /storage//pre-seal-.json \ - --pre-sealed-sectors /storage//pre-seal-0 \ - --pre-sealed-sectors /storage//pre-seal-1 \ - --pre-sealed-sectors /storage//pre-seal-2 \ - --nosync -``` +# Advanced Usage ## Other network configuration @@ -346,9 +266,8 @@ of the specs-actors code itself. Most notability are the supported sector sizes, minimum miner power, and block time. Currently these should be changed by editing `build/params_testnet.go` of the lotus source code. ------------------------------- -### Verified Registery +## Verified Registery The Verified Registery can be interactived through the lotus-shed tool using signed messages. The address is set during genesis creation. To be able to use the verified registery on a network, generate a key and replace the `Signers` array with @@ -374,7 +293,90 @@ the address. } ``` -### Limits on number of sectors +## Adding additional accounts + +Any additional accounts which should have preallocated funds in the network can be added by updating the `Accounts` +field. + +You will want to keep this amount of funds preallocated through additional accounts to a minimum as all funds in these +accounts will count towards the circulating supply. Having a high ciruclating supply will result in high sector pledge +fees. For this reason, it's better to allocated all of the funds to a single actors, and set that actor to the account ID +of 90, which will not be counted towards the ciruclating supply. For public test networks, this account would be called the +faucet. + +[TODO]: <> (This should be a command in lotus-seed to add accounts) +[TODO]: <> (There needs to a way to set the actor ID 90 account.) + +``` +# OWNER is a wallet address, use lotus-shed keyinfo --format '{{ .Address }}' .keyinfo +OWNER="" + +# BALANCE is the amount of AttoFIL (10^-18 FIL) which will be preallocated to the account (wallet) +# 1 FIL == 1000000000000000000 AttoFIL +BALANCE= + +GENESISTMP=$(mktemp) + +jq --arg Owner ${OWNER} --arg Balance ${BALANCE} \ + '.Accounts |= . + [{Type: "account", Balance: $Balance, Meta: {Owner: $Owner}}]' \ + < "/storage//genesis.json" > ${GENESISTMP} +mv ${GENESISTMP} "/storage//genesis.json" +``` + + +## Other miners + +Other miners have a similar initialization process, they just should not specify the `--genesis-miner` flag. This will +stop them from forking the network immediately by trying to mine their own chain. + +``` +lotus-storage-miner init --actor \ + --sector-size \ + --pre-sealed-metadata /storage//pre-seal-.json \ + --pre-sealed-sectors /storage//pre-seal-0 \ + --pre-sealed-sectors /storage//pre-seal-1 \ + --pre-sealed-sectors /storage//pre-seal-2 \ + --nosync +``` + +## Pre-sealing Sectors in Parallel + +The pre-sealing process can be run in parallel by suppling proper `--sector-offset` values. In a difference process the +output of this process can be merged together to create a single metadata file for the miner. + +``` +lotus-seed --sector-dir /storage//pre-seal-0 pre-seal ... --num-sectors 4 --sector-offset 0 +lotus-seed --sector-dir /storage//pre-seal-1 pre-seal ... --num-sectors 4 --sector-offset 4 +lotus-seed --sector-dir /storage//pre-seal-2 pre-seal ... --num-sectors 4 --sector-offset 8 +``` + +### Merging pre-seal metadata of parallel pre-seals + +If run a parallel pre-seal, sector metadata will need to be merged (`pre-seal-.json`) together into a single +file. This file will then be used to construct the network configuration file. + +``` +lotus-seed aggregate-manifests \ + /storage//pre-seal-0/pre-seal-.json \ + /storage//pre-seal-1/pre-seal-.json \ + /storage//pre-seal-3/pre-seal-.json > /storage//pre-seal-.json +``` + +NOTE: Each of these lines updates some value, the first 3 lines (starting with the `$MinerId`) are probably not needed +``` +/usr/local/bin/lotus-seed aggregate-manifests {{ lotus_miner_presealed_sectors | product([metadatafile]) | map('join', '/') | join(' ') }} \ +| jq --arg Addr "{{ lotus_miner_wallet_address }}" --arg MinerId "{{ lotus_miner_addr }}" --arg VerifiedDeal "{{ lotus_miner_verified_deals | lower }}" ' + .[$MinerId].Owner = $Addr + | .[$MinerId].Worker = $Addr + | .[$MinerId].ID = $MinerId + | .[$MinerId].Sectors[].Deal.Client = $Addr + | .[$MinerId].Sectors[].Deal.VerifiedDeal = ($VerifiedDeal == "true") + | .[$MinerId].Sectors[] |= (.Deal.Label = .CommR."/") +' > "{{ lotus_miner_presealed_metadata }}" +``` + + +## Limits on number of sectors There is an upper bound to the number of sectors a miner can have. This doesn't really affect 32/64 sector miners, but the smaller test sectors have a limit of 2 sectors per partition and because lotus submits all parittions as a single @@ -441,3 +443,15 @@ for MINER_ADDR in ${MINERS[@]}; do done done ``` + +## Tipset Thresholds + +Before lotus will start syncing a tipset, lotus requires that is has seen the tipset from a certain number of peers. +This value is defined by the build constant `BootstrapPeerThreshold`. When this value is set to any number greather +than one (1), starting a network runs into issues. This is because at the beggining of the network, there is only a +single peer that actually is producing blocks and sharing the tipset. Therefore no peers will try to sync the tipset. + +To resolve this issue, the environment var `LOTUS_SYNC_BOOTSTRAP_PEERS` needs to be set to `1` on at least a number of peers +equal to the value of `BootstrapPeerThreshold`. + +See https://github.com/filecoin-project/lotus/issues/5474 for more information. From d1519eb5a92c2a68fe67601112df85d93752201d Mon Sep 17 00:00:00 2001 From: ognots Date: Fri, 16 Apr 2021 18:24:53 -0400 Subject: [PATCH 3/8] add todo for build instructions --- NETWORK_SETUP.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index 962acc2d5..7c9646959 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -29,6 +29,9 @@ Binaries required: - lotus - lotus-miner +# Building Binaries +@TODO add instructions relevant to editing Lotus source, build configuration and make commands for building proper binaries + # Simple Example This example documents how to set up the Nerpa Network with a simple network configuration. From e9078488a4bb4504fcbd1f308b1a1c9681bd6f50 Mon Sep 17 00:00:00 2001 From: Travis Person Date: Mon, 19 Apr 2021 05:12:05 +0000 Subject: [PATCH 4/8] Add lotus source code configuration --- NETWORK_SETUP.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index 7c9646959..d388289e3 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -32,6 +32,53 @@ Binaries required: # Building Binaries @TODO add instructions relevant to editing Lotus source, build configuration and make commands for building proper binaries +## Creating a new Network + +The network configuration for lotus are defined at build. This means to add a new network, some source code configuration is required. +Network parameters are stored in the `./build` folder of lotus, with the convention of `params_.go`. The default configration +for mainnet is in `params_mainnet.go`. + +Adding a new network requires naming the network, and creating a new network configuration files. It's best to use another network configuration as +a template as there is no complete documenation of all the configuration that is required to be set. + +Some of the things can are configured for networks include +- drand network configuration +- network upgrade epochs +- supported sector sizes +- minimum miner size +- block production delay + +Note: The network parameters is also where to define the `BootstrapPeerThreshold`, which is covered with more detail in another section. + +[TODO]: <> (mainnet and testnet params should be fully documented param files) + + +- `./build/params_.go` +- `./build/bootstrap/.pi` + +In the network parameters file (`params_.go`), there are two important constants that must be set, `BootstrappersFile`, and `GenesisFile`, these +describe the names of the files that will be built into the the binary. The `BootstrappersFile` points to a file of multiaddrs that will be used for bootstrap configuration, and the `GenesisFile` points to the genesis file that will be built in to the binary if present. + +_params_<network>.go +``` +// +build +package build + +const BootstrappersFile = ".pi" +const GenesisFile = ".car" +``` + +Important! For each network created a build configuration needs to be added to mainnet configuration to disable it when building a different network. You must had a `// +build !` to `params_mainnet.go`. + +## Adding make targets + +To make it easier to build the binaries for the new network, make targets can be added to the `Makefile`. + +``` +: GOFLAGS+=-tags= +: build-devnets +``` + # Simple Example This example documents how to set up the Nerpa Network with a simple network configuration. From f0aacbde66512b83a64229d027fc2154dc690e75 Mon Sep 17 00:00:00 2001 From: ognots Date: Tue, 17 Aug 2021 16:28:00 -0400 Subject: [PATCH 5/8] minor edits --- NETWORK_SETUP.md | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index d388289e3..77c83b068 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -10,8 +10,7 @@ gives the most flexibility and outlines the way we setup networks. Starting a Filecoin network requires some special setup. To produce blocks on the Filecoin network, and therefore process messages, a miner on the network must exist with power. For the start of the network this power is given to a miner during -the creation of the genesis. The power given to the miner isn't just a number, the miner is still required to -submit Window PoSt proofs as well as Winning PoSts for block production, there is no special casing for these miners. +the creation of the genesis. ## Basic steps @@ -29,8 +28,6 @@ Binaries required: - lotus - lotus-miner -# Building Binaries -@TODO add instructions relevant to editing Lotus source, build configuration and make commands for building proper binaries ## Creating a new Network @@ -91,7 +88,11 @@ A minimally viable Lotus network is comprised of: ## Lotus Shed -The `lotus-shed` binary is where lotus keeps all of our tool programs. +The `lotus-shed` binary is where lotus keeps all of our tool programs. It can be compiled from lotus source code by running the following: +``` +make clean deps lotus-shed + +``` ### Generating bls keys for pre-sealing sectors @@ -129,13 +130,19 @@ lotus-shed fetch-params --proving-params ## Lotus Seed -(The sectors can be special in one way though, because we can freely construct the genesis there is no pre-commit process). -This requires that sectors exist. The process of creating these sectors before the network is created is called pre-sealing, +Creating a genesis file to start a new network requires that sectors exist for genesis miners. +The process of creating these sectors before the network is created is called pre-sealing, and is done through the `lotus-seed` tool. The `lotus-seed` binary is used to generate pre-sealed sectors for network bootstrap, as well as constructing the network json configuration use to generate the genesis. +It can be compiled from lotus source code by running the following: +``` +make clean deps lotus-seed + +``` + Note: As long as the replication construct of proofs do not change, keys stay secret, and the pre-sealed sectors meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reused between networks. @@ -144,15 +151,10 @@ meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reus For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed. ``` -lotus-seed pre-seal --miner-addr t01000 --key ./pre-seal-t01000.key --sector-size --num-sectors 4 --sector-offset 0 +lotus-seed pre-seal --miner-addr t01000 --sector-size --num-sectors 4 --sector-offset 0 ``` - -Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing -using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However, -the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by -editing the `pre-seal-.json` files, or better yet, waiting till after running the `aggregate-manifests` command which -will produce a single file making it easier to edit the owner and worker keys. +This will produce a file that looks something like: ``` cat pre-seal-t01000.json @@ -169,6 +171,13 @@ cat pre-seal-t01000.json } ``` +Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing +using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However, +the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by +editing the `pre-seal-.json` files, or better yet, waiting till after running the `aggregate-manifests` command which +will produce a single file making it easier to edit the owner and worker keys. + + [TODO]: <> (There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config?) NOTE: pre-seal does not generate t_aux files which are required to exist along side every p_aux file. You will have to add these later From 108a60d74493dcc34738224e2b6228fed17d8ad8 Mon Sep 17 00:00:00 2001 From: ognots Date: Wed, 1 Sep 2021 20:31:40 -0400 Subject: [PATCH 6/8] partly audited and added build instructions --- NETWORK_SETUP.md | 197 +++++++++++++++++++++++++---------------------- 1 file changed, 105 insertions(+), 92 deletions(-) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index 77c83b068..83e69bf2f 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -10,7 +10,7 @@ gives the most flexibility and outlines the way we setup networks. Starting a Filecoin network requires some special setup. To produce blocks on the Filecoin network, and therefore process messages, a miner on the network must exist with power. For the start of the network this power is given to a miner during -the creation of the genesis. +the creation of the genesis. ## Basic steps @@ -29,70 +29,78 @@ Binaries required: - lotus-miner -## Creating a new Network +# Simple Example -The network configuration for lotus are defined at build. This means to add a new network, some source code configuration is required. -Network parameters are stored in the `./build` folder of lotus, with the convention of `params_.go`. The default configration -for mainnet is in `params_mainnet.go`. +This example documents how to set up a Filecoin Network called `demonet` with a simple network configuration. -Adding a new network requires naming the network, and creating a new network configuration files. It's best to use another network configuration as -a template as there is no complete documenation of all the configuration that is required to be set. +A minimally viable Lotus network is comprised of: +* Lotus "miner" with presealed sectors +* Lotus "bootstrap" node(s) +* Genesis block +* Faucet -Some of the things can are configured for networks include -- drand network configuration -- network upgrade epochs -- supported sector sizes -- minimum miner size -- block production delay +## Compiling Binaries -Note: The network parameters is also where to define the `BootstrapPeerThreshold`, which is covered with more detail in another section. +At this time, there are several [lotus](https://github.com/filecoin-project/lotus) files that need edited to create a new network. -[TODO]: <> (mainnet and testnet params should be fully documented param files) +For this example we'll start by copying and editing network parameters from a pre-defined network - `butterflynet`. +``` +cp buid/params_butterfly.go build/params_demo.go +``` -- `./build/params_.go` -- `./build/bootstrap/.pi` +We'll need to replace the top two lines with our new network name and some others basic parameters by editing `build/params_demo.go` +``` +//go:build demonet +// +build demonet +... +const BootstrappersFile = "demonet.pi" +const GenesisFile = "demonet.car" +... +BuildType = BuildDemonet +... +const BootstrapPeerThreshold = 1 -In the network parameters file (`params_.go`), there are two important constants that must be set, `BootstrappersFile`, and `GenesisFile`, these -describe the names of the files that will be built into the the binary. The `BootstrappersFile` points to a file of multiaddrs that will be used for bootstrap configuration, and the `GenesisFile` points to the genesis file that will be built in to the binary if present. +``` -_params_<network>.go +Configuration and explanation of all parameters is outside the scope of this document. + +We'll also need to add a new constant and `BuildType` case in `build/version.go` ``` -// +build -package build +// +build demonet -const BootstrappersFile = ".pi" -const GenesisFile = ".car" +const( + ... + BuildDemonet = 0x8 +) +... + case BuildDemonet: + return "+demonet" ``` -Important! For each network created a build configuration needs to be added to mainnet configuration to disable it when building a different network. You must had a `// +build !` to `params_mainnet.go`. +Also we need to make sure mainnet params aren't used for demonet. In `build/params_mainnet` add -## Adding make targets +``` +// +build !demonet +``` -To make it easier to build the binaries for the new network, make targets can be added to the `Makefile`. +Finally we should add a `make` target in `Makefile` ``` -: GOFLAGS+=-tags= -: build-devnets +demonet: GOFLAGS+=-tags=demonet +demonet: build-devnets ``` -# Simple Example +Once we've done all this, we can finally compile our binaries +``` +make clean deps demonet install +``` -This example documents how to set up the Nerpa Network with a simple network configuration. - -A minimally viable Lotus network is comprised of: -* Lotus "miner" with presealed sectors -* Lotus "bootstrap" node -* Genesis block -* Faucet +Optionally move `lotus-seed` and `lotus-shed` to your executable path. ## Lotus Shed -The `lotus-shed` binary is where lotus keeps all of our tool programs. It can be compiled from lotus source code by running the following: -``` -make clean deps lotus-shed - -``` +The `lotus-shed` binary is where lotus keeps all of our tool programs. ### Generating bls keys for pre-sealing sectors @@ -107,7 +115,7 @@ lotus-shed keyinfo new bls This will create a file called `libp2p-host-
.keyinfo` which will contain a keyinfo json object. ``` -lotus-shed keyinfo new libp2p +lotus-shed keyinfo new libp2p-host ``` ### Importing keys to lotus repository @@ -115,7 +123,7 @@ lotus-shed keyinfo new libp2p Note: Be sure to set the `LOTUS_PATH` before running the command if using a non standard location. ``` -lotus-shed keyinfo import .keyinfo +lotus-shed keyinfo import bls-.keyinfo ``` ### Downloading proof params @@ -130,19 +138,13 @@ lotus-shed fetch-params --proving-params ## Lotus Seed -Creating a genesis file to start a new network requires that sectors exist for genesis miners. +Creating a genesis file to start a new network requires that sectors exist for genesis miners. The process of creating these sectors before the network is created is called pre-sealing, and is done through the `lotus-seed` tool. The `lotus-seed` binary is used to generate pre-sealed sectors for network bootstrap, as well as constructing the network json configuration use to generate the genesis. -It can be compiled from lotus source code by running the following: -``` -make clean deps lotus-seed - -``` - Note: As long as the replication construct of proofs do not change, keys stay secret, and the pre-sealed sectors meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reused between networks. @@ -151,7 +153,7 @@ meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reus For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed. ``` -lotus-seed pre-seal --miner-addr t01000 --sector-size --num-sectors 4 --sector-offset 0 +lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 ``` This will produce a file that looks something like: @@ -174,7 +176,7 @@ cat pre-seal-t01000.json Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However, the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by -editing the `pre-seal-.json` files, or better yet, waiting till after running the `aggregate-manifests` command which +editing the `pre-seal-.json` files. If performing concurrent sector pre-sealing, you can wail until after running the `aggregate-manifests` command which will produce a single file making it easier to edit the owner and worker keys. @@ -190,7 +192,7 @@ Every network requires a genesis file to initialize from. This section will cove which will later be used to generate the `genesis.car` file itself. ``` -lotus-seed genesis new --network-name /storage//genesis.json +lotus-seed genesis new --network-name demonet genesis.json ``` #### Setting a network start time @@ -220,28 +222,53 @@ mv ${GENESISTMP} "/storage//genesis.json" For each miner with pre-sealed sectors that should be included in the network will need to have their metadata file added to the genesis. +In our case we'll just include a single miner. ``` -lotus-seed genesis add-miner /storage//genesis.json /storage//pre-seal-.json +lotus-seed genesis add-miner genesis.json ~/.genesis-sectors/pre-seal-t01000.json ``` ### Generating the genesis.car To generate the genesis car file, starting a lotus daemon is required. The `genesis.car` file will be created along side -the `genesis.json` file at `/storage//genesis.car` (this is a detail of this script). +the `genesis.json` file at `genesis.car` (this is a detail of this script). ``` -GENESIS_JSON="/storage//genesis.json" -GENPATH=$(mktemp --suffix=genesis -d) +lotus-seed genesis car --out demonet.car genesis.json +``` -cp "${GENESIS_JSON}" "${GENPATH}/genesis.json" +## Initializing pre-sealed miners -lotus-seed genesis car \ - --out="${GENPATH}/devnet.car" \ - "${GENPATH}/genesis.json" +Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start +a network, but all pre-sealed miners for the network should be initialized together. It's not strictly required though. + +There are no special flags we pass to the lotus daemon for these miners, besides the `--genesis` flag. + +One miner must be marked the genesis miner.The genesis miner is the first miner to run and will mine the initial blocks +required for initialization of all other. Technically the other miners can be initialized after the genesis miner is +actually running the chain the chain is progressing. However, we always initialize them together and allow for a full +network initialization before actually running the miner. + +For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using one +directory. + +Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a +`lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap +nodes and compile their addresses into the lotus daemon using the `build/bootstrap/bootstrappers.pi`. + +After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`. + +### Genesis miner + +Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner -cp ${GENPATH}/devnet.car $(dirname $GENESIS_JSON)/genesis.car -rm -rf ${GENPATH} +``` +lotus daemon --genesis demonet.car & +lotus-miner init --actor t01000 \ + --sector-size \ + --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \ + --pre-sealed-sectors /storage/t01000/pre-seal-0 \ + --nosync --genesis-miner ``` ## Running bootstrap nodes @@ -254,9 +281,9 @@ Bootstrap nodes are just lotus daemons, ideally with a pre-generated libp2p host without having to distribute new multiaddrs. Bootstrap nodes must all be publicly dial-able and have a static port, as lotus by default uses a random port assigned by the kernel. -To compile bootstrap multiaddrs into the `lotus` binary, place the multiaddrs in `./build/bootstrap/bootstrappers.pi`. +### Start bootstrap daemon -Bootstrap nodes should also run the bootstrap profile by specifying it on the daemon commands. +Start the bootstrap daemon with a profile. ``` lotus daemon --profile=bootstrapper @@ -282,38 +309,24 @@ description=Lotus daemon firewall rules ports=1347/tcp ``` -## Initializing pre-sealed miners - -Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start -a network, but all pre-sealed miners for the network should be initialized together. It's not strictly required though. - -There are no special flags we pass to the lotus daemon for these miners, besides the `--genesis` flag. - -One miner must be marked the genesis miner.The genesis miner is the first miner to run and will mine the initial blocks -required for initialization of all other. Technically the other miners can be initialized after the genesis miner is -actually running the chain the chain is progressing. However, we always initialize them together and allow for a full -network initialization before actually running the miner. +### Update bootstrap list -For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using one -directory. +Get the bootstrap node's id -Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a -`lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap -nodes and compile their addresses into the lotus daemon using the `build/bootstrap/bootstrappers.pi`. +``` +lotus net id +``` -After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`. +Let's create a bootstrap node list at `build/bootstrap.demonet.pi` +with a single `localhost` multiaddr using the libp2p key we created +``` +/dns4/localhost/tcp/1347/p2p/ +``` -### Genesis miner +Once you've done this, you need to rebuild lotus again and distribute as needed. -Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner +As mentioned, using pre-generated keys is recommended outside of this demo. -``` -lotus-storage-miner init --actor t01000 \ - --sector-size \ - --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \ - --pre-sealed-sectors /storage/t01000/pre-seal-0 \ - --nosync --genesis-miner -``` # Advanced Usage From edf9552df8914b4a32c1e40742b3733a49f06413 Mon Sep 17 00:00:00 2001 From: ognots Date: Wed, 8 Sep 2021 13:13:18 -0400 Subject: [PATCH 7/8] some more tweaks --- NETWORK_SETUP.md | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index 83e69bf2f..ca5c5d0e2 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -39,6 +39,12 @@ A minimally viable Lotus network is comprised of: * Genesis block * Faucet +## Getting Ready + +To follow this demo, you should be on a Linux or Mac computer with at least 16GB memory. + +We'll need + ## Compiling Binaries At this time, there are several [lotus](https://github.com/filecoin-project/lotus) files that need edited to create a new network. @@ -123,7 +129,7 @@ lotus-shed keyinfo new libp2p-host Note: Be sure to set the `LOTUS_PATH` before running the command if using a non standard location. ``` -lotus-shed keyinfo import bls-.keyinfo +lotus-shed keyinfo import .keyinfo ``` ### Downloading proof params @@ -153,7 +159,7 @@ meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reus For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed. ``` -lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 +lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 --key bls-.keyinfo --network-version 5 ``` This will produce a file that looks something like: @@ -228,10 +234,7 @@ In our case we'll just include a single miner. lotus-seed genesis add-miner genesis.json ~/.genesis-sectors/pre-seal-t01000.json ``` -### Generating the genesis.car - -To generate the genesis car file, starting a lotus daemon is required. The `genesis.car` file will be created along side -the `genesis.json` file at `genesis.car` (this is a detail of this script). +### Generating the genesis car file ``` lotus-seed genesis car --out demonet.car genesis.json @@ -254,7 +257,7 @@ directory. Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a `lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap -nodes and compile their addresses into the lotus daemon using the `build/bootstrap/bootstrappers.pi`. +nodes and compile their addresses into the lotus daemon using the `build/bootstrap/demonet.pi`. After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`. @@ -263,11 +266,11 @@ After initialization the only thing requires is to start the storage miners them Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner ``` -lotus daemon --genesis demonet.car & -lotus-miner init --actor t01000 \ - --sector-size \ - --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \ - --pre-sealed-sectors /storage/t01000/pre-seal-0 \ +lotus daemon --genesis=demonet.car --bootstrap=false & +lotus-miner init --actor t01000 \ + --sector-size=512MiB \ + --pre-sealed-metadata=~/.genesis-sectors/pre-seal-t01000.json \ + --pre-sealed-sectors =~/.genesis-sectors \ --nosync --genesis-miner ``` @@ -317,7 +320,7 @@ Get the bootstrap node's id lotus net id ``` -Let's create a bootstrap node list at `build/bootstrap.demonet.pi` +Let's create a bootstrap node list at `build/bootstrap/demonet.pi` with a single `localhost` multiaddr using the libp2p key we created ``` /dns4/localhost/tcp/1347/p2p/ From 5f2e8e145bd12e8c4a3ebc87b7737380f33a8d07 Mon Sep 17 00:00:00 2001 From: ognots Date: Fri, 10 Sep 2021 15:24:01 -0400 Subject: [PATCH 8/8] final touches to make runthrough work --- NETWORK_SETUP.md | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md index ca5c5d0e2..557d21293 100644 --- a/NETWORK_SETUP.md +++ b/NETWORK_SETUP.md @@ -56,12 +56,33 @@ cp buid/params_butterfly.go build/params_demo.go ``` We'll need to replace the top two lines with our new network name and some others basic parameters by editing `build/params_demo.go` +Network upgrade epochs get set to negative integers which will signal all upgrades to be skipped and the network will start at the latest network version. ``` -//go:build demonet // +build demonet ... const BootstrappersFile = "demonet.pi" const GenesisFile = "demonet.car" + +const UpgradeBreezeHeight = -1 +const BreezeGasTampingDuration = -2 +const UpgradeSmokeHeight = -2 +const UpgradeIgnitionHeight = -3 +const UpgradeRefuelHeight = -4 + +var UpgradeAssemblyHeight = abi.ChainEpoch(-5) + +const UpgradeTapeHeight = -6 +const UpgradeLiftoffHeight = -7 +const UpgradeKumquatHeight = -8 +const UpgradeCalicoHeight = -9 +const UpgradePersianHeight = -10 +const UpgradeClausHeight = -11 +const UpgradeOrangeHeight = -12 +const UpgradeTrustHeight = -13 +const UpgradeNorwegianHeight = -14 +const UpgradeTurboHeight = -15 +const UpgradeHyperdriveHeight = -16 + ... BuildType = BuildDemonet ... @@ -159,7 +180,7 @@ meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reus For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed. ``` -lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 --key bls-.keyinfo --network-version 5 +lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 --key bls-.keyinfo ``` This will produce a file that looks something like: @@ -240,6 +261,8 @@ lotus-seed genesis add-miner genesis.json ~/.genesis-sectors/pre-seal-t01000.jso lotus-seed genesis car --out demonet.car genesis.json ``` +Optionally copy `demonet.car` to `build/genesis/` and rebuild lotus, lotus-miner and lotus-worker to distribute to others. + ## Initializing pre-sealed miners Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start @@ -270,7 +293,7 @@ lotus daemon --genesis=demonet.car --bootstrap=false & lotus-miner init --actor t01000 \ --sector-size=512MiB \ --pre-sealed-metadata=~/.genesis-sectors/pre-seal-t01000.json \ - --pre-sealed-sectors =~/.genesis-sectors \ + --pre-sealed-sectors=~/.genesis-sectors \ --nosync --genesis-miner ```