build: release: v1.21.0-rc2

CHANGELOG.md

@@ -1,6 +1,6 @@
 # Lotus changelog
 
-# v1.21.0-rc1 / 2023-03-21
+# v1.21.0-rc2 / 2023-03-29
 
 This is an optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
 
@@ -11,7 +11,10 @@ Before upgrading to this feature release read carefully through these bullet poi
 - Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the `EnableSplitstore` option in your `config.toml` file. To enable SplitStore, set `EnableSplitstore=true`, and to disable it, set `EnableSplitstore=false`. **It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!**
 - This feature release requires a **minimum Go version of v1.19.7 or higher to successfully build Lotus**. Additionally, Go version v1.20 and higher is now also supported.
 - **Storage Providers:** The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the `FFI_USE_OPENCL=1` flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the `FFI_NO_GPU=1` environment variable when building from source.
-- **Exchanges:** Execution traces (returned from `lotus state exec-trace`, `lotus state replay`, etc.), has changed to account for changes introduced by the by the FVM. **Please make sure to read the `Execution trace format change` section carefully, as these are interface breaking changes**
+- **Storage Providers:** The `lotus-miner sectors extend` command has been refactored to the functionality of `lotus-miner sectors renew`. The issue where extions did not work has been fixed in this release candidate.
+- **Exchanges/Node operators/RPC-providers::** Execution traces (returned from `lotus state exec-trace`, `lotus state replay`, etc.), has changed to account for changes introduced by the by the FVM. **Please make sure to read the `Execution trace format change` section carefully, as these are interface breaking changes**
+- **Syncing issues:** If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the `LOTUS_FVM_CONCURRENCY` enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 128. If you are a Storage Provider and are pushing many messages within a short period of time, exporting `LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1` will also help with keeping in sync.
+- **Catching up from a Snapshot:** Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You [can read the FIP here.](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0060.md)
 
 ## Highlights
 
@@ -26,8 +29,10 @@ SplitStore also has some new configuration settings that you can set in your con
 
 The SplitStore also has two new commands:
 
-- `lotus chain prune hot` will run an online (badger vlog) garbage collection on the hotstore.
-- `lotus chain prune hot-moving` will run an moving garbage collection the hotstore
+- `lotus chain prune hot` is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
+- `lotus chain prune hot-moving` will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.
+
+You can read more about the new SplitStore commands in [the documentation](https://lotus.filecoin.io/lotus/configure/splitstore/#manual-chain-store-garbage-collection).
 
 **RPC API improvements**
 
@@ -98,6 +103,12 @@ The `lotus-Miner sector list` is now running in parallel - which should speed up
 
 ## Improvements
 
+- backport: fix: miner: correctly count sector extensions (10555) ([filecoin-project/lotus#10555](https://github.com/filecoin-project/lotus/pull/10555))
+   - Fixes the issue with sector extensions.
+- fix: proving: Initialize slice with with same length as partition (#10574) ([filecoin-project/lotus#10574])(https://github.com/filecoin-project/lotus/pull/10574)
+   - Fixes an issue where `lotus-miner proving compute window-post` paniced when trying to make skipped sectors human readable.
+- feat: stmgr: speed up calculation of genesis circ supply (#10553) ([filecoin-project/lotus#10553])(https://github.com/filecoin-project/lotus/pull/10553)
+- perf: eth: gas estimate set applyTsMessages false (#10546) ([filecoin-project/lotus#10456](https://github.com/filecoin-project/lotus/pull/10546))
 - feat: config: Force existing users to opt into new defaults (#10488) ([filecoin-project/lotus#10488](https://github.com/filecoin-project/lotus/pull/10488))
    - Force existing users to opt into the new SplitStore defaults.
 - fix: splitstore: Demote now common logs (#10516) ([filecoin-project/lotus#10516](https://github.com/filecoin-project/lotus/pull/10516))
@@ -171,13 +182,17 @@ The `lotus-Miner sector list` is now running in parallel - which should speed up
 - github.com/filecoin-project/go-fil-markets (v1.25.2 -> v1.27.0-rc1):
 - github.com/filecoin-project/go-jsonrpc (v0.2.1 -> v0.2.3):
 - github.com/filecoin-project/go-statemachine (v1.0.2 -> v1.0.3):
+- github.com/ipfs/go-cid (v0.3.2 -> v0.4.0):
+- github.com/ipfs/go-libipfs (v0.5.0 -> v0.7.0):
+- github.com/ipfs/go-path (v0.3.0 -> v0.3.1):
 - deps: update go-libp2p-pubsub to v0.9.3 ([filecoin-project/lotus#10483](https://github.com/filecoin-project/lotus/pull/10483))
 - deps: Update go-jsonrpc to v0.2.2 ([filecoin-project/lotus#10395](https://github.com/filecoin-project/lotus/pull/10395))
 - Update to go-data-transfer v2 and libp2p, still wip ([filecoin-project/lotus#10382](https://github.com/filecoin-project/lotus/pull/10382))
 - dep: ipld: update ipld prime to v0.20.0 ([filecoin-project/lotus#10247](https://github.com/filecoin-project/lotus/pull/10247))
 - chore: node: migrate go-bitswap to go-libipfs/bitswap ([filecoin-project/lotus#10138](https://github.com/filecoin-project/lotus/pull/10138))
 - chore: all: bump go-libipfs to replace go-block-format ([filecoin-project/lotus#10126](https://github.com/filecoin-project/lotus/pull/10126))
 - chore: market: Upgrade to index-provider 0.10.0 ([filecoin-project/lotus#9981](https://github.com/filecoin-project/lotus/pull/9981))
+- chore: all: bump go-libipfs ([filecoin-project/lotus#10563](https://github.com/filecoin-project/lotus/pull/10563))
 
 ## Others
 - Update service_developer_bug_report.yml ([filecoin-project/lotus#10321](https://github.com/filecoin-project/lotus/pull/10321))
@@ -207,6 +222,7 @@ The `lotus-Miner sector list` is now running in parallel - which should speed up
 - chore: update ffi to increase execution parallelism (#10480) ([filecoin-project/lotus#10480](https://github.com/filecoin-project/lotus/pull/10480))
 - chore: update the FFI for release (#10435) ([filecoin-project/lotus#10444](https://github.com/filecoin-project/lotus/pull/10444))
 - build: bump version to v1.21.0-dev ([filecoin-project/lotus#10249](https://github.com/filecoin-project/lotus/pull/10249))
+- build: docker: Update GO-version (#10591) ([filecoin-project/lotus#10249](https://github.com/filecoin-project/lotus/pull/10591))
 - chore: merge release/v1.20.0 into master ([filecoin-project/lotus#10184](https://github.com/filecoin-project/lotus/pull/10184))
 - docs: API Gateway: patch documentation note about make gen command ([filecoin-project/lotus#10422](https://github.com/filecoin-project/lotus/pull/10422))
 - chore: docs: fix docs typos ([filecoin-project/lotus#10155](https://github.com/filecoin-project/lotus/pull/10155))

blockstore/ipfs.go

@@ -3,7 +3,7 @@ package blockstore
 import (
 	"bytes"
 	"context"
-	"io/ioutil"
+	"io"
 
 	"github.com/ipfs/go-cid"
 	httpapi "github.com/ipfs/go-ipfs-http-client"
@@ -103,7 +103,7 @@ func (i *IPFSBlockstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, er
 		return nil, xerrors.Errorf("getting ipfs block: %w", err)
 	}
 
-	data, err := ioutil.ReadAll(rd)
+	data, err := io.ReadAll(rd)
 	if err != nil {
 		return nil, err
 	}

build/version.go

@@ -37,7 +37,7 @@ func BuildTypeString() string {
 }
 
 // BuildVersion is the local build version
-const BuildVersion = "1.21.0-rc1"
+const BuildVersion = "1.21.0-rc2"
 
 func UserVersion() string {
 	if os.Getenv("LOTUS_VERSION_IGNORE_COMMIT") == "1" {

chain/actors/agen/main.go

@@ -4,7 +4,6 @@ import (
 	"bytes"
 	"fmt"
 	"go/format"
-	"io/ioutil"
 	"os"
 	"path/filepath"
 	"strconv"
@@ -66,7 +65,7 @@ func generateAdapters() error {
 		}
 
 		{
-			af, err := ioutil.ReadFile(filepath.Join(actDir, "actor.go.template"))
+			af, err := os.ReadFile(filepath.Join(actDir, "actor.go.template"))
 			if err != nil {
 				return xerrors.Errorf("loading actor template: %w", err)
 			}
@@ -90,7 +89,7 @@ func generateAdapters() error {
 				return err
 			}
 
-			if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("%s.go", act)), fmted, 0666); err != nil {
+			if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("%s.go", act)), fmted, 0666); err != nil {
 				return err
 			}
 		}
@@ -100,7 +99,7 @@ func generateAdapters() error {
 }
 
 func generateState(actDir string, versions []int) error {
-	af, err := ioutil.ReadFile(filepath.Join(actDir, "state.go.template"))
+	af, err := os.ReadFile(filepath.Join(actDir, "state.go.template"))
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil // skip
@@ -123,7 +122,7 @@ func generateState(actDir string, versions []int) error {
 			return err
 		}
 
-		if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("v%d.go", version)), b.Bytes(), 0666); err != nil {
+		if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("v%d.go", version)), b.Bytes(), 0666); err != nil {
 			return err
 		}
 	}
@@ -132,7 +131,7 @@ func generateState(actDir string, versions []int) error {
 }
 
 func generateMessages(actDir string) error {
-	af, err := ioutil.ReadFile(filepath.Join(actDir, "message.go.template"))
+	af, err := os.ReadFile(filepath.Join(actDir, "message.go.template"))
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil // skip
@@ -155,7 +154,7 @@ func generateMessages(actDir string) error {
 			return err
 		}
 
-		if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("message%d.go", version)), b.Bytes(), 0666); err != nil {
+		if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("message%d.go", version)), b.Bytes(), 0666); err != nil {
 			return err
 		}
 	}
@@ -165,7 +164,7 @@ func generateMessages(actDir string) error {
 
 func generatePolicy(policyPath string) error {
 
-	pf, err := ioutil.ReadFile(policyPath + ".template")
+	pf, err := os.ReadFile(policyPath + ".template")
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil // skip
@@ -187,7 +186,7 @@ func generatePolicy(policyPath string) error {
 		return err
 	}
 
-	if err := ioutil.WriteFile(policyPath, b.Bytes(), 0666); err != nil {
+	if err := os.WriteFile(policyPath, b.Bytes(), 0666); err != nil {
 		return err
 	}
 
@@ -196,7 +195,7 @@ func generatePolicy(policyPath string) error {
 
 func generateBuiltin(builtinPath string) error {
 
-	bf, err := ioutil.ReadFile(builtinPath + ".template")
+	bf, err := os.ReadFile(builtinPath + ".template")
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil // skip
@@ -218,7 +217,7 @@ func generateBuiltin(builtinPath string) error {
 		return err
 	}
 
-	if err := ioutil.WriteFile(builtinPath, b.Bytes(), 0666); err != nil {
+	if err := os.WriteFile(builtinPath, b.Bytes(), 0666); err != nil {
 		return err
 	}
 
@@ -227,7 +226,7 @@ func generateBuiltin(builtinPath string) error {
 
 func generateRegistry(registryPath string) error {
 
-	bf, err := ioutil.ReadFile(registryPath + ".template")
+	bf, err := os.ReadFile(registryPath + ".template")
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil // skip
@@ -248,7 +247,7 @@ func generateRegistry(registryPath string) error {
 		return err
 	}
 
-	if err := ioutil.WriteFile(registryPath, b.Bytes(), 0666); err != nil {
+	if err := os.WriteFile(registryPath, b.Bytes(), 0666); err != nil {
 		return err
 	}
 

chain/gen/gen.go

@@ -5,7 +5,7 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"io/ioutil"
+	"os"
 	"sync/atomic"
 	"time"
 
@@ -167,7 +167,7 @@ func NewGeneratorWithSectorsAndUpgradeSchedule(numSectors int, us stmgr.UpgradeS
 
 	maddr1 := genesis2.MinerAddress(0)
 
-	m1temp, err := ioutil.TempDir("", "preseal")
+	m1temp, err := os.MkdirTemp("", "preseal")
 	if err != nil {
 		return nil, err
 	}
@@ -179,7 +179,7 @@ func NewGeneratorWithSectorsAndUpgradeSchedule(numSectors int, us stmgr.UpgradeS
 
 	maddr2 := genesis2.MinerAddress(1)
 
-	m2temp, err := ioutil.TempDir("", "preseal")
+	m2temp, err := os.MkdirTemp("", "preseal")
 	if err != nil {
 		return nil, err
 	}

cli/state.go

@@ -9,7 +9,6 @@ import (
 	"fmt"
 	"html/template"
 	"io"
-	"io/ioutil"
 	"os"
 	"reflect"
 	"sort"
@@ -1090,7 +1089,7 @@ var StateComputeStateCmd = &cli.Command{
 
 		var stout *lapi.ComputeStateOutput
 		if csofile := cctx.String("compute-state-output"); csofile != "" {
-			data, err := ioutil.ReadFile(csofile)
+			data, err := os.ReadFile(csofile)
 			if err != nil {
 				return err
 			}

cli/util/retrieval.go

@@ -4,7 +4,6 @@ import (
 	"encoding/json"
 	"fmt"
 	"io"
-	"io/ioutil"
 	"net/http"
 	"net/url"
 	"path"
@@ -65,7 +64,7 @@ func ClientExportStream(apiAddr string, apiAuth http.Header, eref api.ExportRef,
 	}
 
 	if resp.StatusCode != http.StatusOK {
-		em, err := ioutil.ReadAll(resp.Body)
+		em, err := io.ReadAll(resp.Body)
 		if err != nil {
 			return nil, xerrors.Errorf("reading error body: %w", err)
 		}

cli/wallet.go

@@ -6,7 +6,6 @@ import (
 	"encoding/hex"
 	"encoding/json"
 	"fmt"
-	"io/ioutil"
 	"os"
 	"strings"
 
@@ -337,7 +336,7 @@ var walletImport = &cli.Command{
 			inpdata = indata
 
 		} else {
-			fdata, err := ioutil.ReadFile(cctx.Args().First())
+			fdata, err := os.ReadFile(cctx.Args().First())
 			if err != nil {
 				return err
 			}

cmd/lotus-bench/import.go

@@ -6,7 +6,6 @@ import (
 	"encoding/json"
 	"fmt"
 	"io"
-	"io/ioutil"
 	"math"
 	"net/http"
 	_ "net/http/pprof"
@@ -158,7 +157,7 @@ var importBenchCmd = &cli.Command{
 		if rdir := cctx.String("repodir"); rdir != "" {
 			tdir = rdir
 		} else {
-			tmp, err := ioutil.TempDir("", "lotus-import-bench")
+			tmp, err := os.MkdirTemp("", "lotus-import-bench")
 			if err != nil {
 				return err
 			}

cmd/lotus-bench/main.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
-	"io/ioutil"
 	"math/big"
 	"math/rand"
 	"os"
@@ -197,7 +196,7 @@ var sealBenchCmd = &cli.Command{
 				return xerrors.Errorf("creating sectorbuilder dir: %w", err)
 			}
 
-			tsdir, err := ioutil.TempDir(sdir, "bench")
+			tsdir, err := os.MkdirTemp(sdir, "bench")
 			if err != nil {
 				return err
 			}
@@ -287,7 +286,7 @@ var sealBenchCmd = &cli.Command{
 			// sectorbuilder directory... we need a better way to handle
 			// this in other cases
 
-			fdata, err := ioutil.ReadFile(filepath.Join(sbdir, "pre-seal-"+maddr.String()+".json"))
+			fdata, err := os.ReadFile(filepath.Join(sbdir, "pre-seal-"+maddr.String()+".json"))
 			if err != nil {
 				return err
 			}
@@ -637,7 +636,7 @@ func runSeals(sb *ffiwrapper.Sealer, sbfs *basicfs.Provider, numSectors int, par
 							return err
 						}
 
-						if err := ioutil.WriteFile(saveC2inp, b, 0664); err != nil {
+						if err := os.WriteFile(saveC2inp, b, 0664); err != nil {
 							log.Warnf("%+v", err)
 						}
 					}
@@ -751,7 +750,7 @@ var proveCmd = &cli.Command{
 			return xerrors.Errorf("Usage: lotus-bench prove [input.json]")
 		}
 
-		inb, err := ioutil.ReadFile(c.Args().First())
+		inb, err := os.ReadFile(c.Args().First())
 		if err != nil {
 			return xerrors.Errorf("reading input file: %w", err)
 		}

cmd/lotus-bench/simple.go

@@ -5,7 +5,6 @@ import (
 	"encoding/base64"
 	"encoding/json"
 	"fmt"
-	"io/ioutil"
 	"os"
 	"strconv"
 	"time"
@@ -444,7 +443,7 @@ var simpleCommit1 = &cli.Command{
 			return err
 		}
 
-		if err := ioutil.WriteFile(cctx.Args().Get(4), b, 0664); err != nil {
+		if err := os.WriteFile(cctx.Args().Get(4), b, 0664); err != nil {
 			log.Warnf("%+v", err)
 		}
 
@@ -478,7 +477,7 @@ var simpleCommit2 = &cli.Command{
 			return xerrors.Errorf("Usage: lotus-bench prove [input.json]")
 		}
 
-		inb, err := ioutil.ReadFile(c.Args().First())
+		inb, err := os.ReadFile(c.Args().First())
 		if err != nil {
 			return xerrors.Errorf("reading input file: %w", err)
 		}
@@ -861,7 +860,7 @@ var simpleProveReplicaUpdate1 = &cli.Command{
 			return xerrors.Errorf("json marshal vanilla proofs: %w", err)
 		}
 
-		if err := ioutil.WriteFile(cctx.Args().Get(7), vpjb, 0666); err != nil {
+		if err := os.WriteFile(cctx.Args().Get(7), vpjb, 0666); err != nil {
 			return xerrors.Errorf("writing vanilla proofs file: %w", err)
 		}
 
@@ -934,7 +933,7 @@ var simpleProveReplicaUpdate2 = &cli.Command{
 			return xerrors.Errorf("parse commr: %w", err)
 		}
 
-		vpb, err := ioutil.ReadFile(cctx.Args().Get(3))
+		vpb, err := os.ReadFile(cctx.Args().Get(3))
 		if err != nil {
 			return xerrors.Errorf("reading valilla proof file: %w", err)
 		}