fix(content): update sectors section (#1154)

This PR adds a new, updated description of the sectors subsection within the Storage Mining Subsystem. It also changes the structure of the subsections.

content/systems/filecoin_mining/sector/_index.md

@@ -3,30 +3,37 @@ title: Sector
 weight: 2
 bookCollapseSection: true
 dashboardWeight: 2
-dashboardState: reliable
+dashboardState: stable
 dashboardAudit: n/a
 dashboardTests: 0
 ---
 
 # Sector
+---
+
+Sectors are the basic units of storage on Filecoin. They have standard sizes, as well as well-defined time-increments for commitments. The size of a sector balances security concerns against usability. A sectorʼs lifetime is determined in the storage market, and sets the promised duration of the sector.
+
+In the first iteration of the protocol, 32GiB and 64GiB sectors are supported. Maximum sector lifetime is determined by the proof algorithm. Maximum sector lifetime is initially 18 months. A sector naturally expires when it reaches the end of its lifetime.  Additionally, the miner can extend the lifetime of their sectors. Rewards are earned and collaterals recovered when the miner fulfils their commitment.
+
+Individual deals are formed when a storage miner and client are matched on Filecoinʼs storage market. The protocol does not distinguish miners matching with real clients from miners generating self-deals. However, **committed capacity** is a construction that is introduced to make self-dealing unnecessary and economically irrational. In earlier designs of the network, only sectors filled with deals increased the minerʼs likelihood of winning the block reward. This led to the expectation that miners would attack and exploit the network by playing the role of both storage provider and client, creating a malicious self-deal.
 
-The `Sector` is a fundamental "storage container" abstraction used in Filecoin Storage Mining. It is the basic unit of storage,
-and serves to make storage conform to a set of expectations.
+If a sector is only partially full of deals, the network considers the remainder to be _committed capacity_. Similarly, sectors with no deals are called committed capacity sectors; miners are rewarded for proving to the network that they are pledging storage capacity and are encouraged to find clients who need storage. When a miner finds storage demand, they can upgrade their committed capacity sectors to earn additional revenue in the form of a deal fee from paying clients. More details on how to add storage and upgrade sectors in [Adding Storage](adding_storage).
 
-New sectors are empty upon creation. As the miner receives client data, they fill or "pack" the piece(s) into an unsealed sector.
+Committed capacity sectors improve minersʼ incentives to store client data, but they donʼt solve the problem entirely. Storing real client files adds some operational overhead for storage miners. In certain circumstances – for example, if a miner values block rewards far more than deal fees – miners might still choose to ignore client data entirely and simply store committed capacity to increase their storage power as rapidly as possible in pursuit of block rewards. This would make Filecoin less useful and limit clientsʼ ability to store data on the network. Filecoin addresses this issue by introducing the concept of verified clients. Verified clients are certified by a decentralized network of verifiers. Once verified, they can post a predetermined amount of verified client deal data to the storage market, set by the size of their DataCap. Sectors with verified client deals are awarded more storage power – and therefore more block rewards – than sectors without. This provides storage miners with an additional incentive to store client data.
 
-Once a sector is full, the unsealed sector is combined by a proving tree into a single root `UnsealedSectorCID`. The sealing process then encodes (using CBOR) an unsealed sector into a sealed sector, with the root `SealedSectorCID`.
+Verification is not intended to be scarce – it will be very easy to acquire for anyone with real data to store on Filecoin. Even though verifiers may allocate verified client DataCaps liberally (yet responsibly and transparently) to make onboarding easier, the overall effect should be a dramatic increase in the proportion of useful data stored on Filecoin.
+
+Once a sector is full (either with client data or as committed capacity), the unsealed sector is combined by a proving tree into a single root `UnsealedSectorCID`. The sealing process then encodes (using CBOR) an unsealed sector into a sealed sector, with the root `SealedSectorCID`.
 
 This diagram shows the composition of an unsealed sector and a sealed sector.
 
 ![Unsealed Sectors and Sealed Sectors](sectors.png)
 
-{{<embed src="sector.id" lang="go" >}}
 
+**Sector Storage & Window PoSt**
+
+The Lotus implementation of the Window PoSt scheduler can be found [here](https://github.com/filecoin-project/lotus/blob/master/storage/wdpost_sched.go) and the actual execution of Window PoSt on a sector can be found [here](https://github.com/filecoin-project/lotus/blob/master/storage/wdpost_run.go).
+
+The Lotus block store implementation for sectors can be found [here](https://github.com/filecoin-project/lotus/blob/master/storage/sectorblocks/blocks.go).
 
-{{<hint warning >}}
-TODO:
 
-- describe sizing ranges of sectors
-- describe "storage/shipping container" analogy
-{{</hint >}}

content/systems/filecoin_mining/sector/adding_storage.md

@@ -0,0 +1,42 @@
+---
+title: Adding Storage
+weight: 7
+dashboardWeight: 2
+dashboardState: stable
+dashboardAudit: n/a
+dashboardTests: 0
+---
+
+# Adding Storage
+
+A Miner adds more storage in the form of Sectors. Adding more storage is a two-step process:
+1. **PreCommitting a Sector**: A Miner publishes a Sector's SealedCID and makes a deposit. The Sector is now registered to the Miner, and the Miner must ProveCommit the Sector or lose their deposit.
+2. **ProveCommitting a Sector**: The Miner provides a Proof of Replication (PoRep) for the Sector. This proof must be submitted AFTER a delay (the InteractiveEpoch), and BEFORE PreCommit expiration.
+
+This two-step process provides assurance that the Miner's PoRep _actually proves_ that the Miner has replicated the Sector data and is generating proofs from it:
+* ProveCommitments must happen AFTER the InteractiveEpoch (150 blocks after Sector PreCommit), as the randomness included at that epoch is used in the PoRep.
+* ProveCommitments must happen BEFORE the PreCommit expiration, which is a boundary established to make sure Miners don't have enough time to "fake" PoRep generation.
+
+For each Sector successfully ProveCommitted, the Miner becomes responsible for continuously proving the existence of their Sectors' data. In return, the Miner is awarded storage power.
+
+# Upgrading Sectors
+
+Miners are granted storage power in exchange for the storage space they dedicate to Filecoin. Ideally, this storage space is used to store data on behalf of Clients, but there may not always be enough Clients to utilize all the space a Miner has to offer.
+
+In order for a Miner to maximize storage power (and profit), they should take advantage of all available storage space immediately, _even before they find enough Clients to use this space_.
+
+To facilitate this, there are _two types_ of Sectors that may be sealed and ProveCommitted:
+* **Regular Sector**: A Sector that contains Client data
+* **Committed Capacity (CC) Sector**: A Sector with no data (all zeroes)
+
+In practice, Miners will add _as much storage as they can_ in the form of CC Sectors, because these can be added without needing to wait for Clients. CC Sectors empower Miners to immediately make use of existing disk space: earning storage power and a higher chance at producing a block.
+
+To incentivize Miners to hoard storage space and dedicate it to Filecoin, CC Sectors have a unique capability: **they can be "upgraded" to Regular Sectors** (also called "replacing a CC Sector").
+
+Miners upgrade their ProveCommitted CC Sectors by PreCommitting a Regular Sector, and specifying that it should replace an existing CC Sector. Once the Regular Sector is successfully ProveCommitted, it will replace the existing CC Sector. If the newly ProveCommitted Regular sector contains a Verified Client deal, i.e., a deal with higher Sector Quality, then the miner's storage power will increase accordingly.
+
+Upgrading capacity currently involves resealing, creating a unique representation of the data through a computationally intensive process. Looking ahead, committed capacity upgrades should eventually be possible without a reseal. A succinct and publicly verifiable proof that the committed capacity has been correctly replaced with replicated data should achieve this goal. However, this mechanism must be fully specified to preserve the security and incentives of the network before it can be implemented and is, therefore, left as a future improvement.
+
+
+
+

content/systems/filecoin_mining/sector/lifecycle.md

@@ -1,38 +1,35 @@
 ---
 title: Sector Lifecycle
+weight: 2
 dashboardWeight: 2
-dashboardState: reliable
+dashboardState: stable
 dashboardAudit: n/a
 dashboardTests: 0
 ---
 
 # Sector Lifecycle
-In Filecoin, miners contribute storage capacity to the network in units of _sectors_. These sectors work similar to real-life shipping containers; they are used provide a unique ID for storage / retrieval processes as well as ensuring the data's *dimensions* conform with all other sectors in the network.
 
-## Sector creation
+Once the sector has been generated and the deal has been incorporated into the Filecoin blockchain, the storage miner begins generating Proofs-of-Spacetime (PoSt) on the sector, starting to potentially win block rewards and also earn storage fees. Parameters are set so that miners generate and capture more value if they guarantee that their sectors will be around for the duration of the original contract. However, some bounds are placed on a sectorʼs lifetime to improve the network performance.
 
-At creation, a sector's space (raw-byte power or sector size) and time (lifetime or duration) are defined. Together, these are referred to as 'spacetime'. The new sector may contain deals (Deals and/or VerifiedDeals), Committed Capacity, or a mixture of both. 
+In particular, as sectors of shorter lifetime are added, the networkʼs capacity can be bottlenecked. The reason is that the chainʼs bandwidth is consumed with new sectors only replacing expiring ones. As a result, a minimum sector lifetime of six months was introduced to more effectively utilize chain bandwidth and miners have the incentive to commit to sectors of longer lifetime. The maximum sector lifetime is limited by the security of the present proofs construction. For a given set of proofs and parameters, the security of Filecoinʼs Proof-of-Replication (PoRep) is expected to decrease as sector lifetimes increase.
 
-The sector is then assigned a `SectorQuality`. which determines its Quality-Adjusted Power in the network, or consensus power.
+It is reasonable to assume that miners enter the network by adding Committed Capacity sectors, that is, sectors that do not contain user data. Once miners agree storage deals with clients, they upgrade their sectors to Regular Sectors. Alternatively, if they find Verified Clients and agree a storage deal with them, they upgrade their sector accordingly. Depending on whether or not a sector includes a (verified) deal, the miner acquires the corresponding storage power in the network.
 
-`SectorQuality` is determined through a weighted average of multipliers, based on spacetime occupied by their contents:
+All sectors are expected to remain live until the end of their sector lifetime and early dropping of sectors will result in slashing. This is done to provide clients a certain level of guarantee on the reliability of their hosted data. Sector termination can comes with a corresponding _termination fee_.
 
-* Sectors full of `VerifiedDeals` will have a `SectorQuality` of `VerifiedDealWeightMultiplier/BaseMultiplier`.
-* Sectors full of regular `Deals` will have a `SectorQuality` of `DealWeightMultiplier/BaseMultiplier`.
-* Sectors with neither will have a `SectorQuality` of `BaseMultiplier/BaseMultiplier`.
+As with every system it is expected that sectors will present faults. Although this might degrade the quality offered by the network, the reaction of the miner to the fault drives system decisions on whether or not the miner should be penalized. If a fault is reported immediately after it is detected by the miner, then the penalty fee is much lower than if the system detects the defect through wrong (or non-existent) PoSt submission.
 
-Once `SectorQuality` has been assigned, the sector is now ready to be added to a miner's claimed power.
+An adjacent concept to the sector _fault_ is sector _recovery_, that is, how quickly and if the miner attempts to recover the sector and bring it back to normal operation. Therefore, in case of a faulty sector, a small penalty fee approximately equal to the block reward that the sector would win per day is applied. The fee is calculated per day of the sector being unavailable to the network.
 
-## Committed Capacity (CC) upgrades
-
-A sector entirely composed of Committed Capacity can later be upgraded to a Deals sector. This is currently done by resealing, though there are plans to make CC upgrades more efficient and cost-effective after the launch of mainnet.
-
-## Sector termination 
-
-All sectors are expected to remain live until the end of their sector lifetime and early dropping of sectors will result in slashing. This is done to provide clients a certain level of guarantee on the reliability of their hosted data.
-
-Miners may also choose to terminate a sector voluntarily and accept a Termination Fee penalty.
+Miners can extend the lifetime of a sector at any time, though the sector will be expected to remain live until it has reached the end of the new sector lifetime. This can be done by submitting a `ExtendedSectorExpiration` message to the chain.
 
-## Sector extensions
+A sector can be in one of the following states.
 
-Miners can extend the lifetime of a sector at any time, though the sector will be expected to remain live until it has reached the end of the new sector lifetime. This can be done by submitting a `ExtendedSectorExpiration` message to the chain.
+| State          | Description                                                                                                                                           |
+|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `Precommitted` | Miner seals sector and submits `miner.PreCommitSector`                                                                                                |
+| `Committed`    | Miner generates a Seal proof and submits `miner.ProveCommitSector`                                                                                    |
+| `Active`       | Miner generate valid PoSt proofs and timely submits `miner.SubmitWindowedPoSt`                                                                        |
+| `Faulty`       | Miner fails to generate a proof (see Fault section)                                                                                                   |
+| `Recovering`   | Miner declared a faulty sector via `miner.DeclareFaultRecovered`                                                                                      |
+| `Terminated`   | Either sector is expired, or early terminated by a miner via `miner.TerminateSectors`, or was failed to be proven for 14 consecutive proving periods. |