diff --git a/NETWORK_SETUP.md b/NETWORK_SETUP.md
index 78dc6e4e2..557d21293 100644
--- a/NETWORK_SETUP.md
+++ b/NETWORK_SETUP.md
@@ -10,11 +10,7 @@ gives the most flexibility and outlines the way we setup networks.
Starting a Filecoin network requires some special setup. To produce blocks on the Filecoin network, and therefore process
messages, a miner on the network must exist with power. For the start of the network this power is given to a miner during
-the creation of the genesis. The power given to the miner isn't just a number, the miner is still required to
-submit Window PoSt proofs as well as Winning PoSts for block production, there is no special casing for these miners. (The sectors
-can be special in one way though, because we can freely construct the genesis there is no pre-commit process).
-This requires that sectors exist. The process of creating these sectors before the network is created is called pre-sealing,
-and is done through the lotus-seed tool. More than one miner can be created during this process.
+the creation of the genesis.
## Basic steps
@@ -32,6 +28,103 @@ Binaries required:
- lotus
- lotus-miner
+
+# Simple Example
+
+This example documents how to set up a Filecoin Network called `demonet` with a simple network configuration.
+
+A minimally viable Lotus network is comprised of:
+* Lotus "miner" with presealed sectors
+* Lotus "bootstrap" node(s)
+* Genesis block
+* Faucet
+
+## Getting Ready
+
+To follow this demo, you should be on a Linux or Mac computer with at least 16GB memory.
+
+We'll need
+
+## Compiling Binaries
+
+At this time, there are several [lotus](https://github.com/filecoin-project/lotus) files that need edited to create a new network.
+
+For this example we'll start by copying and editing network parameters from a pre-defined network - `butterflynet`.
+
+```
+cp buid/params_butterfly.go build/params_demo.go
+```
+
+We'll need to replace the top two lines with our new network name and some others basic parameters by editing `build/params_demo.go`
+Network upgrade epochs get set to negative integers which will signal all upgrades to be skipped and the network will start at the latest network version.
+```
+// +build demonet
+...
+const BootstrappersFile = "demonet.pi"
+const GenesisFile = "demonet.car"
+
+const UpgradeBreezeHeight = -1
+const BreezeGasTampingDuration = -2
+const UpgradeSmokeHeight = -2
+const UpgradeIgnitionHeight = -3
+const UpgradeRefuelHeight = -4
+
+var UpgradeAssemblyHeight = abi.ChainEpoch(-5)
+
+const UpgradeTapeHeight = -6
+const UpgradeLiftoffHeight = -7
+const UpgradeKumquatHeight = -8
+const UpgradeCalicoHeight = -9
+const UpgradePersianHeight = -10
+const UpgradeClausHeight = -11
+const UpgradeOrangeHeight = -12
+const UpgradeTrustHeight = -13
+const UpgradeNorwegianHeight = -14
+const UpgradeTurboHeight = -15
+const UpgradeHyperdriveHeight = -16
+
+...
+BuildType = BuildDemonet
+...
+const BootstrapPeerThreshold = 1
+
+```
+
+Configuration and explanation of all parameters is outside the scope of this document.
+
+We'll also need to add a new constant and `BuildType` case in `build/version.go`
+```
+// +build demonet
+
+const(
+ ...
+ BuildDemonet = 0x8
+)
+...
+ case BuildDemonet:
+ return "+demonet"
+```
+
+Also we need to make sure mainnet params aren't used for demonet. In `build/params_mainnet` add
+
+```
+// +build !demonet
+```
+
+Finally we should add a `make` target in `Makefile`
+
+```
+demonet: GOFLAGS+=-tags=demonet
+demonet: build-devnets
+```
+
+Once we've done all this, we can finally compile our binaries
+```
+make clean deps demonet install
+```
+
+Optionally move `lotus-seed` and `lotus-shed` to your executable path.
+
## Lotus Shed
The `lotus-shed` binary is where lotus keeps all of our tool programs.
@@ -49,7 +142,7 @@ lotus-shed keyinfo new bls
This will create a file called `libp2p-host-
.keyinfo` which will contain a keyinfo json object.
```
-lotus-shed keyinfo new libp2p
+lotus-shed keyinfo new libp2p-host
```
### Importing keys to lotus repository
@@ -72,6 +165,10 @@ lotus-shed fetch-params --proving-params
## Lotus Seed
+Creating a genesis file to start a new network requires that sectors exist for genesis miners.
+The process of creating these sectors before the network is created is called pre-sealing,
+and is done through the `lotus-seed` tool.
+
The `lotus-seed` binary is used to generate pre-sealed sectors for network bootstrap, as well as
constructing the network json configuration use to generate the genesis.
@@ -83,25 +180,10 @@ meet the minimum `ConsensusMinerMinMiners` value, pre-sealed sectors can be reus
For each miner wanted during the initial setup of a new network, sectors need to be pre-sealed.
```
-lotus-seed pre-seal --miner-addr t01000 --key ./pre-seal-t01000.key --sector-size --num-sectors 4 --sector-offset 0
-```
-
-The pre-sealing process can be run in parallel by suppling proper `--sector-offset` values. In a difference process the
-output of this process can be merged together to create a single metadata file for the miner.
-
-```
-lotus-seed --sector-dir /storage//pre-seal-0 pre-seal ... --num-sectors 4 --sector-offset 0
-lotus-seed --sector-dir /storage//pre-seal-1 pre-seal ... --num-sectors 4 --sector-offset 4
-lotus-seed --sector-dir /storage//pre-seal-2 pre-seal ... --num-sectors 4 --sector-offset 8
+lotus-seed pre-seal --miner-addr t01000 --sector-size 512MiB --num-sectors 4 --sector-offset 0 --key bls-.keyinfo
```
-Note: The
-
-Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing
-using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However,
-the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by
-editing the `pre-seal-.json` files, or better yet, waiting till after running the `aggregate-manifests` command which
-will produce a single file making it easier to edit the owner and worker keys.
+This will produce a file that looks something like:
```
cat pre-seal-t01000.json
@@ -118,35 +200,18 @@ cat pre-seal-t01000.json
}
```
-TODO: There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config?
-
-NOTE: pre-seal does not generate t_aux files which are required to exist along side every p_aux file. You will have to add these later
-TODO: THere should be a command to do this or pre-seal should do it automatically.
+Note: Specifying a key is not required during lotus-seed (a key will be generated). When running parallel pre-sealing
+using the `--sector-offset` flag keys should be generated beforehand and pass it into the `lotus-seed` process. However,
+the sectors are not tied to any particilar key till the genesis is created. The key can be modified at anytime prior by
+editing the `pre-seal-.json` files. If performing concurrent sector pre-sealing, you can wail until after running the `aggregate-manifests` command which
+will produce a single file making it easier to edit the owner and worker keys.
-### Merging pre-seal metadata of parallel pre-seals
-If run a parallel pre-seal, sector metadata will need to be merged (`pre-seal-.json`) together into a single
-file. This file will then be used to construct the network configuration file.
+[TODO]: <> (There should be a command to change owner and work addresses at some point in the process, probably on the genesis network config?)
-```
-lotus-seed aggregate-manifests \
- /storage//pre-seal-0/pre-seal-.json \
- /storage//pre-seal-1/pre-seal-.json \
- /storage//pre-seal-3/pre-seal-.json > /storage//pre-seal-.json
-```
+NOTE: pre-seal does not generate t_aux files which are required to exist along side every p_aux file. You will have to add these later
-NOTE: Each of these lines updates some value, the first 3 lines (starting with the `$MinerId`) are probably not needed
-```
-/usr/local/bin/lotus-seed aggregate-manifests {{ lotus_miner_presealed_sectors | product([metadatafile]) | map('join', '/') | join(' ') }} \
-| jq --arg Addr "{{ lotus_miner_wallet_address }}" --arg MinerId "{{ lotus_miner_addr }}" --arg VerifiedDeal "{{ lotus_miner_verified_deals | lower }}" '
- .[$MinerId].Owner = $Addr
- | .[$MinerId].Worker = $Addr
- | .[$MinerId].ID = $MinerId
- | .[$MinerId].Sectors[].Deal.Client = $Addr
- | .[$MinerId].Sectors[].Deal.VerifiedDeal = ($VerifiedDeal == "true")
- | .[$MinerId].Sectors[] |= (.Deal.Label = .CommR."/")
-' > "{{ lotus_miner_presealed_metadata }}"
-```
+[TODO]: <> (There should be a command to do this or pre-seal should do it automatically.)
### Generate network configuration
@@ -154,7 +219,7 @@ Every network requires a genesis file to initialize from. This section will cove
which will later be used to generate the `genesis.car` file itself.
```
-lotus-seed genesis new --network-name /storage//genesis.json
+lotus-seed genesis new --network-name demonet genesis.json
```
#### Setting a network start time
@@ -164,7 +229,7 @@ be defined to reduce the network mining race effect.
The `Timestamp` in the genesis configuration is a unix epoch.
-TODO: This should be a command in lotus-seed
+[TODO]: <> (This should be a command in lotus-seed)
```
# GENESISDELAY is a time in seconds added to the current time to delay the network start by some amount of time
@@ -184,58 +249,52 @@ mv ${GENESISTMP} "/storage//genesis.json"
For each miner with pre-sealed sectors that should be included in the network will need to have their metadata file
added to the genesis.
+In our case we'll just include a single miner.
```
-lotus-seed genesis add-miner /storage//genesis.json /storage//pre-seal-.json
+lotus-seed genesis add-miner genesis.json ~/.genesis-sectors/pre-seal-t01000.json
```
-#### Adding additional accounts
-
-Any additional accounts which should have preallocated funds in the network can be added by updating the `Accounts`
-field.
+### Generating the genesis car file
-You will want to keep this amount of funds preallocated through additional accounts to a minimum as all funds in these
-accounts will count towards the circulating supply. Having a high ciruclating supply will result in high sector pledge
-fees. For this reason, it's better to allocated all of the funds to a single actors, and set that actor to the account ID
-of 90, which will not be counted towards the ciruclating supply. For public test networks, this account would be called the
-faucet.
+```
+lotus-seed genesis car --out demonet.car genesis.json
+```
-TODO: This should be a command in lotus-seed to add accounts
-TODO: There needs to a way to set the actor ID 90 account.
+Optionally copy `demonet.car` to `build/genesis/` and rebuild lotus, lotus-miner and lotus-worker to distribute to others.
-```
-# OWNER is a wallet address, use lotus-shed keyinfo --format '{{ .Address }}' .keyinfo
-OWNER=""
+## Initializing pre-sealed miners
-# BALANCE is the amount of AttoFIL (10^-18 FIL) which will be preallocated to the account (wallet)
-# 1 FIL == 1000000000000000000 AttoFIL
-BALANCE=
+Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start
+a network, but all pre-sealed miners for the network should be initialized together. It's not strictly required though.
-GENESISTMP=$(mktemp)
+There are no special flags we pass to the lotus daemon for these miners, besides the `--genesis` flag.
-jq --arg Owner ${OWNER} --arg Balance ${BALANCE} \
- '.Accounts |= . + [{Type: "account", Balance: $Balance, Meta: {Owner: $Owner}}]' \
- < "/storage//genesis.json" > ${GENESISTMP}
-mv ${GENESISTMP} "/storage//genesis.json"
-```
+One miner must be marked the genesis miner.The genesis miner is the first miner to run and will mine the initial blocks
+required for initialization of all other. Technically the other miners can be initialized after the genesis miner is
+actually running the chain the chain is progressing. However, we always initialize them together and allow for a full
+network initialization before actually running the miner.
-### Generating the genesis.car
+For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using one
+directory.
-To generate the genesis car file, starting a lotus daemon is required. The `genesis.car` file will be created along side
-the `genesis.json` file at `/storage//genesis.car` (this is a detail of this script).
+Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a
+`lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap
+nodes and compile their addresses into the lotus daemon using the `build/bootstrap/demonet.pi`.
-```
-GENESIS_JSON="/storage//genesis.json"
-GENPATH=$(mktemp --suffix=genesis -d)
+After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`.
-cp "${GENESIS_JSON}" "${GENPATH}/genesis.json"
+### Genesis miner
-lotus-seed genesis car \
- --out="${GENPATH}/devnet.car" \
- "${GENPATH}/genesis.json"
+Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner
-cp ${GENPATH}/devnet.car $(dirname $GENESIS_JSON)/genesis.car
-rm -rf ${GENPATH}
+```
+lotus daemon --genesis=demonet.car --bootstrap=false &
+lotus-miner init --actor t01000 \
+ --sector-size=512MiB \
+ --pre-sealed-metadata=~/.genesis-sectors/pre-seal-t01000.json \
+ --pre-sealed-sectors=~/.genesis-sectors \
+ --nosync --genesis-miner
```
## Running bootstrap nodes
@@ -248,15 +307,15 @@ Bootstrap nodes are just lotus daemons, ideally with a pre-generated libp2p host
without having to distribute new multiaddrs. Bootstrap nodes must all be publicly dial-able and have a static port, as
lotus by default uses a random port assigned by the kernel.
-To compile bootstrap multiaddrs into the `lotus` binary, place the multiaddrs in `./build/bootstrap/bootstrappers.pi`.
+### Start bootstrap daemon
-Bootstrap nodes should also run the bootstrap profile by specifying it on the daemon commands.
+Start the bootstrap daemon with a profile.
```
lotus daemon --profile=bootstrapper
```
-Optically you may want to increase the `ConnMgrLow` and `ConnMgrHigh` values to increase.
+Optimally you may want to increase the `ConnMgrLow` and `ConnMgrHigh` values.
The `config.toml` should be located under `$LOTUS_PATH/config.toml`.
@@ -276,67 +335,26 @@ description=Lotus daemon firewall rules
ports=1347/tcp
```
-### Tipset Thresholds
+### Update bootstrap list
-Before lotus will start syncing a tipset, lotus requires that is has seen the tipset from a certain number of peers.
-This value is defined by the build constant `BootstrapPeerThreshold`. When this value is set to any number greather
-than one (1), starting a network runs into issues. This is because at the beggining of the network, there is only a
-single peer that actually is producing blocks and sharing the tipset. Therefore no peers will try to sync the tipset.
+Get the bootstrap node's id
-To resolve this issue, the environment var `LOTUS_SYNC_BOOTSTRAP_PEERS` needs to be set to `1` on at least a number of peers
-equal to the value of `BootstrapPeerThreshold`.
-
-See https://github.com/filecoin-project/lotus/issues/5474 for more information.
-
-## Initializing pre-sealed miners
-
-Initializing requires running both the lotus daemon and a storage miner. Only a single miner is required to start
-a network, but all pre-sealed miners for the network should be initialized together. It's not strictly required though.
-
-There are no special flags we pass to the lotus daemon for these miners, besides the `--genesis` flag.
-
-One miner must be marked the genesis miner.The genesis miner is the first miner to run and will mine the initial blocks
-required for initialization of all other. Technically the other miners can be initialized after the genesis miner is
-actually running the chain the chain is progressing. However, we always initialize them together and allow for a full
-network initialization before actually running the miner.
-
-For each pre-sealed set of sectors, a `--pre-sealed-sectors` flag must be passed. In this example we are only using three
-directories.
-
-Note: All pre-sealed miner daemons must be connected, this can be done by manually connecting them by running a
-`lotus net connect` from all miners to the genesis miner's daemon. However, we recommend that you setup proper bootstrap
-nodes and compile their addresses into the lotus daemon using the `build/bootstrap/bootstrappers.pi`.
-
-After initialization the only thing requires is to start the storage miners themselves by running `lotus-storage-miner run`.
-
-### Genesis miner
-
-Note: There is nothing special about the `t01000` address here. It has just become practice to make it the genesis miner
+```
+lotus net id
+```
+Let's create a bootstrap node list at `build/bootstrap/demonet.pi`
+with a single `localhost` multiaddr using the libp2p key we created
```
-lotus-storage-miner init --actor t01000 \
- --sector-size \
- --pre-sealed-metadata /storage/t01000/pre-seal-t01000.json \
- --pre-sealed-sectors /storage/t01000/pre-seal-0 \
- --pre-sealed-sectors /storage/t01000/pre-seal-1 \
- --pre-sealed-sectors /storage/t01000/pre-seal-2 \
- --nosync --genesis-miner
+/dns4/localhost/tcp/1347/p2p/
```
-### Other miners
+Once you've done this, you need to rebuild lotus again and distribute as needed.
-Other miners have a similar initialization process, they just should not specify the `--genesis-miner` flag. This will
-stop them from forking the network immediately by trying to mine their own chain.
+As mentioned, using pre-generated keys is recommended outside of this demo.
-```
-lotus-storage-miner init --actor \
- --sector-size \
- --pre-sealed-metadata /storage//pre-seal-.json \
- --pre-sealed-sectors /storage//pre-seal-0 \
- --pre-sealed-sectors /storage//pre-seal-1 \
- --pre-sealed-sectors /storage//pre-seal-2 \
- --nosync
-```
+
+# Advanced Usage
## Other network configuration
@@ -346,9 +364,8 @@ of the specs-actors code itself.
Most notability are the supported sector sizes, minimum miner power, and block time. Currently these should be changed
by editing `build/params_testnet.go` of the lotus source code.
-------------------------------
-### Verified Registery
+## Verified Registery
The Verified Registery can be interactived through the lotus-shed tool using signed messages. The address is set during
genesis creation. To be able to use the verified registery on a network, generate a key and replace the `Signers` array with
@@ -374,7 +391,90 @@ the address.
}
```
-### Limits on number of sectors
+## Adding additional accounts
+
+Any additional accounts which should have preallocated funds in the network can be added by updating the `Accounts`
+field.
+
+You will want to keep this amount of funds preallocated through additional accounts to a minimum as all funds in these
+accounts will count towards the circulating supply. Having a high ciruclating supply will result in high sector pledge
+fees. For this reason, it's better to allocated all of the funds to a single actors, and set that actor to the account ID
+of 90, which will not be counted towards the ciruclating supply. For public test networks, this account would be called the
+faucet.
+
+[TODO]: <> (This should be a command in lotus-seed to add accounts)
+[TODO]: <> (There needs to a way to set the actor ID 90 account.)
+
+```
+# OWNER is a wallet address, use lotus-shed keyinfo --format '{{ .Address }}' .keyinfo
+OWNER=""
+
+# BALANCE is the amount of AttoFIL (10^-18 FIL) which will be preallocated to the account (wallet)
+# 1 FIL == 1000000000000000000 AttoFIL
+BALANCE=
+
+GENESISTMP=$(mktemp)
+
+jq --arg Owner ${OWNER} --arg Balance ${BALANCE} \
+ '.Accounts |= . + [{Type: "account", Balance: $Balance, Meta: {Owner: $Owner}}]' \
+ < "/storage//genesis.json" > ${GENESISTMP}
+mv ${GENESISTMP} "/storage//genesis.json"
+```
+
+
+## Other miners
+
+Other miners have a similar initialization process, they just should not specify the `--genesis-miner` flag. This will
+stop them from forking the network immediately by trying to mine their own chain.
+
+```
+lotus-storage-miner init --actor \
+ --sector-size \
+ --pre-sealed-metadata /storage//pre-seal-.json \
+ --pre-sealed-sectors /storage//pre-seal-0 \
+ --pre-sealed-sectors /storage//pre-seal-1 \
+ --pre-sealed-sectors /storage//pre-seal-2 \
+ --nosync
+```
+
+## Pre-sealing Sectors in Parallel
+
+The pre-sealing process can be run in parallel by suppling proper `--sector-offset` values. In a difference process the
+output of this process can be merged together to create a single metadata file for the miner.
+
+```
+lotus-seed --sector-dir /storage//pre-seal-0 pre-seal ... --num-sectors 4 --sector-offset 0
+lotus-seed --sector-dir /storage//pre-seal-1 pre-seal ... --num-sectors 4 --sector-offset 4
+lotus-seed --sector-dir /storage//pre-seal-2 pre-seal ... --num-sectors 4 --sector-offset 8
+```
+
+### Merging pre-seal metadata of parallel pre-seals
+
+If run a parallel pre-seal, sector metadata will need to be merged (`pre-seal-.json`) together into a single
+file. This file will then be used to construct the network configuration file.
+
+```
+lotus-seed aggregate-manifests \
+ /storage//pre-seal-0/pre-seal-.json \
+ /storage//pre-seal-1/pre-seal-.json \
+ /storage//pre-seal-3/pre-seal-.json > /storage//pre-seal-.json
+```
+
+NOTE: Each of these lines updates some value, the first 3 lines (starting with the `$MinerId`) are probably not needed
+```
+/usr/local/bin/lotus-seed aggregate-manifests {{ lotus_miner_presealed_sectors | product([metadatafile]) | map('join', '/') | join(' ') }} \
+| jq --arg Addr "{{ lotus_miner_wallet_address }}" --arg MinerId "{{ lotus_miner_addr }}" --arg VerifiedDeal "{{ lotus_miner_verified_deals | lower }}" '
+ .[$MinerId].Owner = $Addr
+ | .[$MinerId].Worker = $Addr
+ | .[$MinerId].ID = $MinerId
+ | .[$MinerId].Sectors[].Deal.Client = $Addr
+ | .[$MinerId].Sectors[].Deal.VerifiedDeal = ($VerifiedDeal == "true")
+ | .[$MinerId].Sectors[] |= (.Deal.Label = .CommR."/")
+' > "{{ lotus_miner_presealed_metadata }}"
+```
+
+
+## Limits on number of sectors
There is an upper bound to the number of sectors a miner can have. This doesn't really affect 32/64 sector miners, but
the smaller test sectors have a limit of 2 sectors per partition and because lotus submits all parittions as a single
@@ -441,3 +541,15 @@ for MINER_ADDR in ${MINERS[@]}; do
done
done
```
+
+## Tipset Thresholds
+
+Before lotus will start syncing a tipset, lotus requires that is has seen the tipset from a certain number of peers.
+This value is defined by the build constant `BootstrapPeerThreshold`. When this value is set to any number greather
+than one (1), starting a network runs into issues. This is because at the beggining of the network, there is only a
+single peer that actually is producing blocks and sharing the tipset. Therefore no peers will try to sync the tipset.
+
+To resolve this issue, the environment var `LOTUS_SYNC_BOOTSTRAP_PEERS` needs to be set to `1` on at least a number of peers
+equal to the value of `BootstrapPeerThreshold`.
+
+See https://github.com/filecoin-project/lotus/issues/5474 for more information.