feat(guildnet): added guildnet support and delete crashnet

Betanet test: ``` sandi@sandi-ThinkPad-X1-Carbon-7th ~/w/n/nearup (master)> nearup run betanet 2021-07-07 15:01:55.236 INFO nearup - run: Home directory is /home/sandi/.near/betanet... 2021-07-07 15:01:55.238 INFO nodelib - setup_and_run: Using officially compiled binary 2021-07-07 15:01:57.502 INFO util - download_binaries: Downloading latest deployed version for betanet 2021-07-07 15:01:57.503 INFO util - download_binaries: Downloading neard to /home/sandi/.nearup/near/betanet/neard from nearcore/Linux/master/b7c8a3b5d2bfaeb1d689972fd867d1c10afcc440/nightly/neard... 2021-07-07 15:08:33.410 INFO util - download_binaries: Downloaded neard to /home/sandi/.nearup/near/betanet/neard... 2021-07-07 15:08:33.411 INFO util - download_binaries: Making the neard executable... 2021-07-07 15:08:33.412 INFO nodelib - check_and_setup: Setting up network configuration. 2021-07-07 15:08:33.412 INFO nodelib - init_near: Initializing the node configuration using near binary... Jul 07 15:08:33.658 INFO neard: Version: 1.2.0, Build: b7c8a3b5, Latest Protocol: 114 Jul 07 15:08:33.701 INFO near: Downloading genesis file from: https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/betanet/genesis.json ... Jul 07 15:09:01.023 INFO near: Saved the genesis file to: /home/sandi/.near/betanet/genesis.json ... Jul 07 15:09:01.716 INFO near: Generated for betanet network node key and genesis file in /home/sandi/.near/betanet 2021-07-07 15:09:02.993 INFO nodelib - run: Node is running... 2021-07-07 15:09:02.994 INFO nodelib - run: To check logs call: `nearup logs` or `nearup logs --follow` 2021-07-07 15:09:02.994 INFO nodelib - run: Watcher is enabled. Starting watcher... 2021-07-07 15:09:02.994 INFO watcher - run_watcher: Starting the nearup watcher... sandi@sandi-ThinkPad-X1-Carbon-7th ~/w/n/nearup (master)> nearup logs 2021-07-07 15:09:20.256 INFO tailer - show_logs: 18654|neard|betanet tail: cannot open '' for reading: No such file or directory ==> /home/sandi/.nearup/logs/betanet.log <== Jul 07 15:09:17.956 DEBUG client: dropping block 7xRFoEf59GMBc3RHxRqdHSy7CW8EsB6azQWcbmaLVGHD that is too far ahead. Block height 29369096 current head height 28680172 Jul 07 15:09:17.962 DEBUG client: None Received block 7xRFoEf59GMBc3RHxRqdHSy7CW8EsB6azQWcbmaLVGHD <- G52y5g9kYVdetntdvwpbDcUvfU8SQ4BZxpga38qhgV4s at 29369096 from ed25519:AtxyywuiieiDCCH7c893rQSx3hrK8X6qr2kkkHTZWkT1, requested: false sandi@sandi-ThinkPad-X1-Carbon-7th ~/w/n/nearup (master) [1]> nearup stop 2021-07-07 15:09:23.227 WARNING nodelib - stop_nearup: Stopping the near daemon... 2021-07-07 15:09:23.228 INFO nodelib - stop_native: Near procces is neard with pid: 18654... 2021-07-07 15:09:23.228 INFO nodelib - stop_native: Stopping process neard with pid 18654... 2021-07-07 15:09:53.230 WARNING nodelib - stop_native: Timeout expired. Killing process 18654 2021-07-07 15:09:53.230 WARNING nodelib - stop_nearup: Stopping the nearup watcher... 2021-07-07 15:09:53.230 INFO watcher - stop_watcher: Stopping near watcher python3 with pid 18655... ``` Guildnet: ``` sandi@sandi-ThinkPad-X1-Carbon-7th ~/w/n/nearup (add-guildnet)> nearup run guildnet 2021-07-07 15:59:44.222 INFO nearup - run: Home directory is /home/sandi/.near/guildnet... 2021-07-07 15:59:44.222 INFO nodelib - setup_and_run: Using officially compiled binary 2021-07-07 15:59:46.528 INFO util - download_binaries: Downloading latest deployed version for guildnet 2021-07-07 15:59:46.530 INFO util - download_binaries: Downloading neard to /home/sandi/.nearup/near/guildnet/neard from nearcore/Linux/1.20.0/9655ff694e8c38277b184ca42dc3478bbd4fd86a/neard... 2021-07-07 16:00:14.955 INFO util - download_binaries: Downloaded neard to /home/sandi/.nearup/near/guildnet/neard... 2021-07-07 16:00:14.955 INFO util - download_binaries: Making the neard executable... 2021-07-07 16:00:16.016 INFO nodelib - genesis_changed: Remote genesis protocol version md5 3e2e8a1e31b85ed098feac0d32bdf2f8, ours is 727b8e66dc5619a035d08609cd6332aa 2021-07-07 16:00:16.016 INFO nodelib - check_and_update_genesis: Update genesis config and remove stale data for guildnet 2021-07-07 16:00:52.187 INFO nodelib - print_staking_key: Stake for user 'test.near' with 'ed25519:5CnhRRREbLEfPfHussAiMwUsxUkWmZQTZobgi43VmNdd' 2021-07-07 16:00:52.199 INFO nodelib - run: Node is running... 2021-07-07 16:00:52.200 INFO nodelib - run: To check logs call: `nearup logs` or `nearup logs --follow` 2021-07-07 16:00:52.200 INFO nodelib - run: Watcher is enabled. Starting watcher... 2021-07-07 16:00:52.201 INFO watcher - run_watcher: Starting the nearup watcher... sandi@sandi-ThinkPad-X1-Carbon-7th ~/w/n/nearup (add-guildnet) [1]> nearup logs 2021-07-07 16:02:07.079 INFO tailer - show_logs: 50962|neard|guildnet tail: cannot open '' for reading: No such file or directory ==> /home/sandi/.nearup/logs/guildnet.log <== Jul 07 16:00:52.212 INFO neard: Version: 1.20.0-rc.1, Build: 9655ff69, Latest Protocol: 46 Jul 07 16:00:53.223 INFO near: Did not find "/home/sandi/.near/guildnet/data" path, will be creating new store database Jul 07 16:00:59.378 INFO stats: Server listening at ed25519:KDVxu4oDnQ8mSXQJ63jBGwKYmrXqMbccDRgL8yKTtw2@0.0.0.0:24567 Jul 07 16:01:09.404 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 0%, Mem: 0 B Jul 07 16:01:19.408 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 0%, Mem: 42.9 MiB Jul 07 16:01:29.411 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 42.9 MiB Jul 07 16:01:39.414 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 2%, Mem: 42.9 MiB Jul 07 16:01:49.416 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 42.9 MiB Jul 07 16:01:59.418 INFO stats: #29756287 Waiting for peers 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 42.9 MiB 2021-07-07 16:02:07.081 ERROR tailer - show_logs: Unable to read logs. Please try again. ```
near · Jul 12, 2021 · 0a7e3e8 · 0a7e3e8
1 parent 382e1f0
commit 0a7e3e8
Show file tree

Hide file tree

Showing 7 changed files with 58 additions and 33 deletions.
diff --git a/nearup b/nearup
@@ -39,7 +39,7 @@ def cli():
 @cli.command()
 @click.argument('network',
                 type=click.Choice(
-                    {'mainnet', 'testnet', 'betanet', 'crashnet', 'localnet'}))
+                    {'mainnet', 'testnet', 'betanet', 'guildnet', 'localnet'}))
 @click.option(
     '--binary-path',
     type=str,
@@ -118,7 +118,7 @@ def stop(keep_watcher):
 
 @click.argument('network',
                 type=click.Choice(
-                    {'testnet', 'betanet', 'crashnet', 'localnet'}))
+                    {'mainnet', 'testnet', 'betanet', 'guildnet', 'localnet'}))
 @click.option(
     '--home',
     type=str,

diff --git a/nearuplib/VERSION b/nearuplib/VERSION
@@ -1 +1 @@
-1.1.0
+1.2.0
diff --git a/nearuplib/constants.py b/nearuplib/constants.py
@@ -4,6 +4,13 @@
 LOGS_FOLDER = os.path.expanduser('~/.nearup/logs')
 LOCALNET_LOGS_FOLDER = os.path.expanduser("~/.nearup/logs/localnet")
 NODE_PID_FILE = os.path.expanduser('~/.nearup/node.pid')
-S3_BUCKET = 'build.nearprotocol.com'
 WATCHER_PID_FILE = os.path.expanduser('~/.nearup/watcher.pid')
 DEFAULT_WAIT_TIMEOUT = 30
+
+S3_BUCKETS = {
+    'default': 'build.nearprotocol.com',
+    'mainnet': 'build.nearprotocol.com',
+    'testnet': 'build.nearprotocol.com',
+    'betanet': 'build.nearprotocol.com',
+    'guildnet': 'build.openshards.io',
+}
diff --git a/nearuplib/nodelib.py b/nearuplib/nodelib.py
@@ -27,8 +27,7 @@ def init_near(home_dir, binary_path, chain_id, init_flags):
     logging.info("Initializing the node configuration using near binary...")
 
     cmd = [f'{binary_path}/neard', f'--home={home_dir}', 'init'] + init_flags
-    if chain_id in ['crashnet', 'betanet', 'testnet']:
-        # force download genesis
+    if chain_id in ['betanet', 'testnet']:
         cmd.append('--download-genesis')
 
     subprocess.check_call(cmd)
@@ -90,7 +89,7 @@ def check_and_setup(binary_path, home_dir, init_flags):
                     )
                     sys.exit(1)
 
-        if chain_id in ['crashnet', 'betanet', 'testnet']:
+        if chain_id in ['guildnet', 'betanet', 'testnet']:
             check_and_update_genesis(chain_id, home_dir)
         elif chain_id == 'mainnet':
             logging.info("Using the mainnet genesis...")
@@ -103,7 +102,7 @@ def check_and_setup(binary_path, home_dir, init_flags):
     init_near(home_dir, binary_path, chain_id, init_flags)
     download_config(chain_id, home_dir)
 
-    if chain_id not in ['mainnet', 'crashnet', 'betanet', 'testnet']:
+    if chain_id not in ['mainnet', 'guildnet', 'betanet', 'testnet']:
         with open(os.path.join(home_dir, 'genesis.json'), 'r+') as genesis_fd:
             genesis_config = json.load(genesis_fd)
             genesis_config['gas_price'] = 0

diff --git a/nearuplib/tailer.py b/nearuplib/tailer.py
@@ -29,7 +29,7 @@ def show_logs(follow, number_lines):
     logging.info(pid_info)
     _, _, network = pid_info.strip().split("|")
 
-    if not network in ['crashnet', 'betanet', 'testnet', 'mainnet']:
+    if network == "localnet":
         # TODO: localnet could have several logs, not showing them all but list log files here
         # Maybe better to support `nearup logs node0` usage.
         logging.info(

diff --git a/nearuplib/util.py b/nearuplib/util.py
@@ -6,64 +6,64 @@
 from botocore import UNSIGNED
 from botocore.client import Config
 
-from nearuplib.constants import S3_BUCKET
+from nearuplib.constants import S3_BUCKETS
 from nearuplib.exceptions import NetworkError, capture_as
 
 
 @capture_as(NetworkError)
-def download_from_s3(path, filepath=None):
+def download_from_s3(bucket, path, filepath=None):
     s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
-    s3_client.download_file(S3_BUCKET, path, filepath)
+    s3_client.download_file(bucket, path, filepath)
 
 
 @capture_as(NetworkError)
-def exists_on_s3(path):
+def exists_on_s3(bucket, path):
     s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
     try:
-        s3_client.head_object(Bucket=S3_BUCKET, Key=path)
+        s3_client.head_object(Bucket=bucket, Key=path)
     except s3_client.exceptions.NoSuchKey:
         return False
 
     return True
 
 
 @capture_as(NetworkError)
-def read_from_s3(path):
+def read_from_s3(bucket, path):
     s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
-    response = s3_client.get_object(Bucket=S3_BUCKET, Key=path)
+    response = s3_client.get_object(Bucket=bucket, Key=path)
     return response['Body'].read().decode('utf-8')
 
 
 def new_release_ready(net, uname):
     """Sanity check that a new release is ready for download."""
-    commit = latest_deployed_release_commit(net)
-    branch = latest_deployed_release_branch(net)
+    if net in ["localnet", "guildnet"]:
+        commit = latest_deployed_release_commit("testnet")
+        branch = latest_deployed_release_branch("testnet")
+    else:
+        commit = latest_deployed_release_commit(net)
+        branch = latest_deployed_release_branch(net)
 
     if not commit:
         return False
 
     path = f'nearcore/{uname}/{branch}/{commit}/neard'
 
-    return exists_on_s3(path)
+    return exists_on_s3(S3_BUCKETS["default"], path)
 
 
 def download_config(net, home_dir):
-    download_from_s3(f'nearcore-deploy/{net}/config.json',
+    download_from_s3(S3_BUCKETS[net], f'nearcore-deploy/{net}/config.json',
                      os.path.join(home_dir, 'config.json'))
 
 
 def download_genesis(net, home_dir):
-    download_from_s3(f'nearcore-deploy/{net}/genesis.json',
+    download_from_s3(S3_BUCKETS[net], f'nearcore-deploy/{net}/genesis.json',
                      os.path.join(home_dir, 'genesis.json'))
 
 
 def download_binaries(net, uname, nightly=False):
-    if net != "localnet":
-        commit = latest_deployed_release_commit(net)
-        branch = latest_deployed_release_branch(net)
-    else:
-        commit = latest_deployed_release_commit("testnet")
-        branch = latest_deployed_release_branch("testnet")
+    commit = latest_deployed_release_commit(net)
+    branch = latest_deployed_release_branch(net)
 
     if commit:
         logging.info(f'Downloading latest deployed version for {net}')
@@ -78,7 +78,7 @@ def download_binaries(net, uname, nightly=False):
 
         logging.info(
             f"Downloading {binary} to {download_path} from {download_url}...")
-        download_from_s3(download_url, download_path)
+        download_from_s3(S3_BUCKETS['default'], download_url, download_path)
         logging.info(f"Downloaded {binary} to {download_path}...")
 
         logging.info(f"Making the {binary} executable...")
@@ -92,7 +92,11 @@ def download_binaries(net, uname, nightly=False):
 
 
 def latest_deployed_release_commit(net):
-    return read_from_s3(f'nearcore-deploy/{net}/latest_deploy').strip()
+    if net in ["localnet", "guildnet"]:
+        return read_from_s3(S3_BUCKETS['default'],
+                            'nearcore-deploy/testnet/latest_deploy').strip()
+    return read_from_s3(S3_BUCKETS['default'],
+                        f'nearcore-deploy/{net}/latest_deploy').strip()
 
 
 def latest_deployed_release_commit_has_changed(net, commit):
@@ -108,20 +112,35 @@ def latest_deployed_release_commit_has_changed(net, commit):
 
 
 def latest_deployed_release_branch(net):
-    return read_from_s3(f'nearcore-deploy/{net}/latest_release').strip()
+    if net in ["localnet", "guildnet"]:
+        return read_from_s3(S3_BUCKETS['default'],
+                            'nearcore-deploy/testnet/latest_release').strip()
+    return read_from_s3(S3_BUCKETS['default'],
+                        f'nearcore-deploy/{net}/latest_release').strip()
 
 
 def latest_deployed_release_time(net):
-    return read_from_s3(f'nearcore-deploy/{net}/latest_deploy_at').strip()
+    if net in ["localnet", "guildnet"]:
+        return read_from_s3(S3_BUCKETS['default'],
+                            'nearcore-deploy/testnet/latest_deploy_at').strip()
+    return read_from_s3(S3_BUCKETS['default'],
+                        f'nearcore-deploy/{net}/latest_deploy_at').strip()
 
 
 def latest_genesis_md5sum(net):
-    return read_from_s3(f'nearcore-deploy/{net}/genesis_md5sum').strip()
+    if net in ["localnet", "guildnet"]:
+        return read_from_s3(S3_BUCKETS['default'],
+                            'nearcore-deploy/testnet/genesis_md5sum').strip()
+    return read_from_s3(S3_BUCKETS['default'],
+                        f'nearcore-deploy/{net}/genesis_md5sum').strip()
 
 
 def latest_genesis_md5sum_has_changed(net, md5_sum):
     latest_md5sum = latest_genesis_md5sum(net)
 
+    if net in ["localnet", "guildnet"]:
+        latest_md5sum = latest_genesis_md5sum("testnet")
+
     logging.info(f"Current genesis md5sum is {md5_sum}")
     logging.info(f"Latest genesis md5sum is {latest_md5sum}")
 

diff --git a/watcher b/watcher
@@ -42,7 +42,7 @@ def main():
 
 @main.command()
 @click.argument('network',
-                type=click.Choice({'crashnet', 'betanet', 'testnet',
+                type=click.Choice({'guildnet', 'betanet', 'testnet',
                                    'mainnet'}))
 @click.argument('home', type=str)
 @click.option('--force-restart',