feat: standardize node directory structure (gnolang#1944)

## Description

Closes gnolang#1882

This PR standardizes the node's directory structure, and does not assume
that the `genesis.json` is contained within the node directory (further
tackled in gnolang#1883).

The biggest change introduced in this PR is that the node's directory
path is treated as absolute, so there is no funny business with path
concatenation when the node is running.

This is best described with the desired structure:

```
.
├── genesis.json
└── my-node-dir/
    ├── db/
    │   ├── blockstore.db (folder)
    │   ├── gnolang.db (folder)
    │   ├── state.db (folder)
    │   └── cs.wal
    ├── secrets/
    │   ├── priv_validator_state.json
    │   ├── node_key.json
    │   └── priv_validator_key.json
    └── config/
       └── config.toml
```

**BREAKING CHANGES**:
- the `genesis.json` file is no longer contained inside the node's
working directory, but outside it by default (this is dropped entirely
in gnolang#1883)
- the configuration is moved top-level to the node's directroy
- the secrets are no longer contained in `<node-dir>/config`, but in
`<node-dir>/secrets`
- the `priv_validator_state.json` is moved to `<node-dir>/secrets` by
default

If you have an old Gno chain, and what to keep backwards compatibility,
move the `genesis.json` to just outside the node directory (to prevent
it from being regenerated), and update the relevant `config.toml` path
values (`genesis_file` and `home` for the `BaseConfig`). Additionally,
update all `home` paths in the `config.toml` to be absolute paths (this
is dropped in gnolang#1884).

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [x] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
- [ ] Added new benchmarks to [generated
graphs](https://gnoland.github.io/benchmarks), if any. More info
[here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md).
</details>

.dockerignore

@@ -5,22 +5,36 @@ misc/
 docker-compose.yml
 tests/docker-integration/
 
-# gitignore
-*.sw[pon]
-cmd/foo.go
-unsorted.*
+# Copied from .gitignore
+
+# Editor Leftovers
 .DS_Store
+.vscode
+.idea
+
+# Default node data
+gnoland-data
+genesis.json
+
+# Build leftovers
+build
+
+# Legacy .gitignore
 data/*
-proto/*
-testdir/
-pkgs/sdk/vm/_testdata
-build/*
+testdir
+
+# Log + generation leftovers
 *.tx
 *.log.*
 *.log
 *.gno.gen.go
 *.gno.gen_test.go
-.vscode
-.idea
 *.pb.go
 pbbindings.go
+*~
+.#*
+*#
+
+# Test coverage leftovers
+cover.out
+coverage.out

.gitignore

@@ -1,22 +1,31 @@
-*.sw[pon]
-cmd/foo.go
-unsorted.*
+# Editor Leftovers
 .DS_Store
+.vscode
+.idea
+
+# Default node data
+gnoland-data
+genesis.json
+
+# Build leftovers
+build
+
+# Legacy .gitignore
 data/*
 testdir
-pkgs/sdk/vm/_testdata
-build
+
+# Log + generation leftovers
 *.tx
 *.log.*
 *.log
 *.gno.gen.go
 *.gno.gen_test.go
-.vscode
-.idea
 *.pb.go
 pbbindings.go
 *~
 .#*
 *#
+
+# Test coverage leftovers
 cover.out
 coverage.out

contribs/gnokeykc/go.mod

@@ -10,6 +10,7 @@ require (
 )
 
 require (
+	dario.cat/mergo v1.0.0 // indirect
 	github.com/alessio/shellescape v1.4.1 // indirect
 	github.com/btcsuite/btcd/btcec/v2 v2.3.3 // indirect
 	github.com/btcsuite/btcd/btcutil v1.1.5 // indirect

docs/getting-started/local-setup/premining-balances.md

@@ -24,7 +24,7 @@ In order for us to premine funds on a fresh chain, we need to make sure we do no
 from previous chain runs.
 
 The blockchain node, when it runs, works with an embedded DB locally on disk to store execution data (such as
-configuration files, or the state DB). For Gno blockchain nodes, this working directory is labeled as `testdir` by
+configuration files, or the state DB). For Gno blockchain nodes, this working directory is labeled as `gnoland-data` by
 default.
 
 To clean out old blockchain data, navigate to the `gno.land` folder and run the appropriate make command:

docs/getting-started/local-setup/setting-up-a-local-chain.md

@@ -42,11 +42,11 @@ Let's break down the most important default settings:
 - `genesis-balances-file` - the initial premine balances file, which contains initial native currency allocations for
   the chain. By default, the genesis balances file is located in `gno.land/genesis/genesis_balances.txt`, this is also the
   reason why we need to navigate to the `gno.land` sub-folder to run the command with default settings
-- `root-dir` - the working directory for the node configuration and node data (state DB)
+- `data-dir` - the working directory for the node configuration and node data (state DB)
 
 :::info Resetting the chain
 
-As mentioned, the working directory for the node is located in `root-dir`. To reset the chain, you need
+As mentioned, the working directory for the node is located in `data-dir`. To reset the chain, you need
 to delete this directory and start the node up again. If you are using the default node configuration, you can run
 `make fclean` from the `gno.land` sub-folder to delete the `tempdir` working directory.
 

docs/gno-tooling/cli/gnoland.md

@@ -21,11 +21,11 @@ gnoland
 ### **Options**
 
 | Name                       | Type    | Description                                                                             |
-| -------------------------- | ------- | --------------------------------------------------------------------------------------- |
+|----------------------------| ------- | --------------------------------------------------------------------------------------- |
 | `chainid`                  | String  | The id of the chain (default: `dev`).                                                   |
 | `genesis-balances-file`    | String  | The initial GNOT distribution file (default: `./gnoland/genesis/genesis_balances.txt`). |
 | `genesis-remote`           | String  | Replacement '%%REMOTE%%' in genesis (default: `"localhost:26657"`).                     |
 | `genesis-txs-file`         | String  | Initial txs to be executed (default: `"./gnoland/genesis/genesis_txs.jsonl"`).          |
-| `root-dir`                 | String  | directory for config and data (default: `testdir`).                                     |
+| `data-dir`                 | String  | directory for config and data (default: `gnoland-data`).                                     |
 | `skip-failing-genesis-txs` | Boolean | Skips transactions that fail from the `genesis-txs-file`                                |
 | `skip-start`               | Boolean | Quits after initialization without starting the node.                                   |

gno.land/Makefile

@@ -49,7 +49,7 @@ install.gnokey:;     go install ./cmd/gnokey
 
 .PHONY: fclean
 fclean: clean
-	rm -rf testdir
+	rm -rf gnoland-data genesis.json
 
 .PHONY: clean
 clean:

gno.land/cmd/gnoland/start.go

@@ -120,12 +120,11 @@ func (c *startCfg) RegisterFlags(fs *flag.FlagSet) {
 		"the root directory of the gno repository",
 	)
 
-	// XXX: Use home directory for this
 	fs.StringVar(
 		&c.dataDir,
 		"data-dir",
-		"testdir",
-		"directory for config and data",
+		"gnoland-data",
+		"the path to the node's data directory",
 	)
 
 	fs.StringVar(
@@ -192,18 +191,14 @@ func (c *startCfg) RegisterFlags(fs *flag.FlagSet) {
 		log.ConsoleFormat.String(),
 		"log format for the gnoland node",
 	)
-
-	// XXX(deprecated): use data-dir instead
-	fs.StringVar(
-		&c.dataDir,
-		"root-dir",
-		"testdir",
-		"deprecated: use data-dir instead - directory for config and data",
-	)
 }
 
 func execStart(c *startCfg, io commands.IO) error {
-	dataDir := c.dataDir
+	// Get the absolute path to the node's data directory
+	nodeDir, err := filepath.Abs(c.dataDir)
+	if err != nil {
+		return fmt.Errorf("unable to get absolute path for data directory, %w", err)
+	}
 
 	var (
 		cfg        *config.Config
@@ -223,7 +218,7 @@ func execStart(c *startCfg, io commands.IO) error {
 		cfg, loadCfgErr = config.LoadConfigFile(c.nodeConfigPath)
 	} else {
 		// Load the default node configuration
-		cfg, loadCfgErr = config.LoadOrMakeConfigWithOptions(dataDir)
+		cfg, loadCfgErr = config.LoadOrMakeConfigWithOptions(nodeDir)
 	}
 
 	if loadCfgErr != nil {
@@ -246,7 +241,9 @@ func execStart(c *startCfg, io commands.IO) error {
 	logger := log.ZapLoggerToSlog(zapLogger)
 
 	// Write genesis file if missing.
-	genesisFilePath := filepath.Join(dataDir, cfg.Genesis)
+	// NOTE: this will be dropped in a PR that resolves issue #1883:
+	// https://github.com/gnolang/gno/issues/1883
+	genesisFilePath := filepath.Join(nodeDir, "../", "genesis.json")
 
 	if !osm.FileExists(genesisFilePath) {
 		// Create priv validator first.
@@ -270,7 +267,7 @@ func execStart(c *startCfg, io commands.IO) error {
 	cfg.TxEventStore = txEventStoreCfg
 
 	// Create application and node.
-	gnoApp, err := gnoland.NewApp(dataDir, c.skipFailingGenesisTxs, logger, c.genesisMaxVMCycles)
+	gnoApp, err := gnoland.NewApp(nodeDir, c.skipFailingGenesisTxs, logger, c.genesisMaxVMCycles)
 	if err != nil {
 		return fmt.Errorf("error in creating new app: %w", err)
 	}
@@ -280,7 +277,7 @@ func execStart(c *startCfg, io commands.IO) error {
 		io.Println(startGraphic)
 	}
 
-	gnoNode, err := node.DefaultNewNode(cfg, logger)
+	gnoNode, err := node.DefaultNewNode(cfg, genesisFilePath, logger)
 	if err != nil {
 		return fmt.Errorf("error in creating node: %w", err)
 	}

gno.land/pkg/gnoland/app.go

@@ -8,6 +8,7 @@ import (
 	"github.com/gnolang/gno/gno.land/pkg/sdk/vm"
 	"github.com/gnolang/gno/gnovm/pkg/gnoenv"
 	abci "github.com/gnolang/gno/tm2/pkg/bft/abci/types"
+	"github.com/gnolang/gno/tm2/pkg/bft/config"
 	dbm "github.com/gnolang/gno/tm2/pkg/db"
 	"github.com/gnolang/gno/tm2/pkg/log"
 	"github.com/gnolang/gno/tm2/pkg/sdk"
@@ -132,7 +133,7 @@ func NewApp(dataRootDir string, skipFailingGenesisTxs bool, logger *slog.Logger,
 	}
 
 	// Get main DB.
-	cfg.DB, err = dbm.NewDB("gnolang", dbm.GoLevelDBBackend, filepath.Join(dataRootDir, "data"))
+	cfg.DB, err = dbm.NewDB("gnolang", dbm.GoLevelDBBackend, filepath.Join(dataRootDir, config.DefaultDBDir))
 	if err != nil {
 		return nil, fmt.Errorf("error initializing database %q using path %q: %w", dbm.GoLevelDBBackend, dataRootDir, err)
 	}

gno.land/pkg/gnoland/node_inmemory.go

@@ -59,34 +59,6 @@ func NewDefaultTMConfig(rootdir string) *tmcfg.Config {
 	return tmcfg.TestConfig().SetRootDir(rootdir)
 }
 
-// NewInMemoryNodeConfig creates a default configuration for an in-memory node.
-func NewDefaultInMemoryNodeConfig(rootdir string) *InMemoryNodeConfig {
-	tm := NewDefaultTMConfig(rootdir)
-
-	// Create Mocked Identity
-	pv := NewMockedPrivValidator()
-	genesis := NewDefaultGenesisConfig(pv.GetPubKey(), tm.ChainID())
-
-	// Add self as validator
-	self := pv.GetPubKey()
-	genesis.Validators = []bft.GenesisValidator{
-		{
-			Address: self.Address(),
-			PubKey:  self,
-			Power:   10,
-			Name:    "self",
-		},
-	}
-
-	return &InMemoryNodeConfig{
-		PrivValidator:      pv,
-		TMConfig:           tm,
-		Genesis:            genesis,
-		GenesisTxHandler:   PanicOnFailingTxHandler,
-		GenesisMaxVMCycles: 10_000_000,
-	}
-}
-
 func (cfg *InMemoryNodeConfig) validate() error {
 	if cfg.PrivValidator == nil {
 		return fmt.Errorf("`PrivValidator` is required but not provided")

gnovm/pkg/doc/dirs_test.go

@@ -100,7 +100,7 @@ func TestDirs_findPackage(t *testing.T) {
 		}},
 		{"alpha", []bfsDir{
 			{importPath: "dirs.mod/dep/alpha", dir: filepath.Join(td, "dirsdep/pkg/mod/dirs.mod/dep/alpha")},
-			// no testdir/module/alpha as it is inside a module
+			// no gnoland-data/module/alpha as it is inside a module
 		}},
 		{"math", []bfsDir{
 			{importPath: "math", dir: filepath.Join(td, "dirs/math")},

misc/deployments/staging.gno.land/Makefile

@@ -11,7 +11,7 @@ logs:
 down:
 	docker compose down
 	docker volume rm -f staginggnoland_gnonode
-	docker compose run gnoland rm -rf /opt/gno/src/gno.land/testdir/data /opt/gno/src/gno.land/testdir/config
+	docker compose run gnoland rm -rf /opt/gno/src/gno.land/gnoland-data/data /opt/gno/src/gno.land/gnoland-data/config
 
 pull:
 	git pull

misc/deployments/staging.gno.land/docker-compose.yml

@@ -17,7 +17,7 @@ services:
       - --chainid=staging
       - --genesis-remote=staging.gno.land:36657
     volumes:
-      - "./data/gnoland:/opt/gno/src/gno.land/testdir"
+      - "./data/gnoland:/opt/gno/src/gno.land/gnoland-data"
     ports:
       - 36656:36656
       - 36657:36657

misc/docker-compose/docker-compose.yml

@@ -9,7 +9,7 @@ services:
       - LOG_LEVEL=4
     command: [ "gnoland", "start" ]
     volumes:
-      - "gnonode:/opt/gno/src/testdir"
+      - "gnonode:/opt/gno/src/gnoland-data"
     networks:
       - gnonode
     restart: on-failure

misc/loop/cmd/snapshotter.go

@@ -162,7 +162,7 @@ func (s snapshotter) startPortalLoopContainer(ctx context.Context) (*types.Conta
 		Binds: []string{
 			fmt.Sprintf("%s/scripts:/scripts", s.cfg.hostPWD),
 			fmt.Sprintf("%s/backups:/backups", s.cfg.hostPWD),
-			fmt.Sprintf("%s:/opt/gno/src/testdir", s.containerName),
+			fmt.Sprintf("%s:/opt/gno/src/gnoland-data", s.containerName),
 		},
 	}, nil, nil, s.containerName)
 	if err != nil {

misc/loop/scripts/start.sh

@@ -19,11 +19,11 @@ gnoland start \
     --skip-start=true \
     --skip-failing-genesis-txs
 
-sed -i "s#^moniker = \".*\"#moniker = \"${MONIKER}\"#" ./testdir/config/config.toml
-sed -i "s#laddr = \".*:26656\"#laddr = \"${P2P_LADDR}\"#" ./testdir/config/config.toml
-sed -i "s#laddr = \".*:26657\"#laddr = \"${RPC_LADDR}\"#" ./testdir/config/config.toml
+sed -i "s#^moniker = \".*\"#moniker = \"${MONIKER}\"#" ./gnoland-data/config/config.toml
+sed -i "s#laddr = \".*:26656\"#laddr = \"${P2P_LADDR}\"#" ./gnoland-data/config/config.toml
+sed -i "s#laddr = \".*:26657\"#laddr = \"${RPC_LADDR}\"#" ./gnoland-data/config/config.toml
 
-sed -i "s#seeds = \".*\"#seeds = \"${SEEDS}\"#" ./testdir/config/config.toml
-sed -i "s#persistent_peers = \".*\"#persistent_peers = \"${PERSISTENT_PEERS}\"#" ./testdir/config/config.toml
+sed -i "s#seeds = \".*\"#seeds = \"${SEEDS}\"#" ./gnoland-data/config/config.toml
+sed -i "s#persistent_peers = \".*\"#persistent_peers = \"${PERSISTENT_PEERS}\"#" ./gnoland-data/config/config.toml
 
 exec gnoland start --skip-failing-genesis-txs

tm2/pkg/bft/blockchain/reactor_test.go

@@ -119,7 +119,7 @@ func newBlockchainReactor(logger *slog.Logger, genDoc *types.GenesisDoc, privVal
 func TestNoBlockResponse(t *testing.T) {
 	t.Parallel()
 
-	config = cfg.ResetTestRoot("blockchain_reactor_test")
+	config, _ = cfg.ResetTestRoot("blockchain_reactor_test")
 	defer os.RemoveAll(config.RootDir)
 	genDoc, privVals := randGenesisDoc(1, false, 30)
 
@@ -182,7 +182,7 @@ func TestFlappyBadBlockStopsPeer(t *testing.T) {
 
 	testutils.FilterStability(t, testutils.Flappy)
 
-	config = cfg.ResetTestRoot("blockchain_reactor_test")
+	config, _ = cfg.ResetTestRoot("blockchain_reactor_test")
 	defer os.RemoveAll(config.RootDir)
 	genDoc, privVals := randGenesisDoc(1, false, 30)