-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NEAR CLI enhancements (#31) #31
Conversation
I like this interface adding this to the mix |
text/0000-near-shell.md
Outdated
### `near key <command>` | ||
|
||
### `near contract <command>` | ||
* `near contract list` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be incredibly useful. Would this information be stored from the user's history? Is there some way we can use a library/RPC to get this info?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure what this is supposed to do. Will it just return a list of contract account ids, or contract metadata, or contract code itself?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the unique identifier for a contract? Do contracts have names? The idea was to list contracts for a given account. It would be good to have an ID and an contract name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The unique identifier is the account name, which is a regular, human-readable NEAR account. Thanks for bringing this up, Bowen. I am now considering removing that list of commands.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@damons what do you mean by "contracts for a given account"? There can be zero or one contract deployed on a given account. If you are talking about contracts that an account interact with, that is much harder to do because user data are stored under the contract account, not the user account.
Yeah, this is great. I think besides adding the validator aspect to
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah this is looking great, let's continue to iterate.
Still have more work to do on this NEP, committing progressively |
text/0000-near-shell.md
Outdated
* `near contract status` | ||
* `near contract add` | ||
* `near contract remove` | ||
* `near contract build` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just highlighting a discussion here: near/near-cli#275
Since the build process for AssemblyScript and Rust differs, it may make sense to either:
- Remove this and lean on the tools for building in the language ecosystems
- In implementation, the build process has a step where the language is determined.
@vgrichina makes good points for (1) here: near/near-cli#275 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is another important aspect to this.
We don't currently have support for multi-contract projects.
And I don't think we can easily provide anything like near build
for these, as they can easily be mix of Rust / AssemblyScript / whatever else contracts which all need to be deployed together.
We need to think of some build system supporting dependencies (i.e. something like make
or gulp
) to put this all together and provide ability for developer to customize individual steps as necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to think of some build system supporting dependencies (i.e. something like make or gulp) to put this all together
near build
currently runs gulp. if the suggestion is to remove near build
would we be adding this back in later?
https://github.com/nearprotocol/near-shell/blob/7d9e8601b0eeb05872406e8bf5eca3e14098224c/bin/near-cli.js#L118-L122
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should:
- remove
near build
- have design for multi contract projects if we want to move away from gulp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed near build
from the spec
Haven't added a recommended method for multi-contract projects, as I think we haven't discussed the alternatives yet. Honestly, I don't have a good gut instinct for that task and would like to hear from others.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about something like lerna? We could probably abstract away all of the lerna operations with near
commands.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sladuca Lerna solves somewhat orthogonal problem, I'm talking more about stuff how you can have one project with multiple deployments (e.g. given this is shared system you may want to deploy single contract on multiple accounts).
Other than that I think it should be possible to use Lerna with near-shell as an option, no need to wrap around with near-shell commands. We want to provide dev tools that fit into different workflows (i.e. not tied to specific build tools ideally) + provide reasonable default workflows in examples.
Few comments:
|
@ilblackdragon when you say:
I want to confirm that having environment variables containing private keys is not considered overriding, correct? If I'm spinning up a Docker instance on the cloud, it would be great to have the option to include these are env vars, fallback to key files in the project (cwd), fallback to key files in the home directory… |
A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed. View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few comments
|
||
These are key-values reflecting how `near-shell` behaves when called within a NEAR project. | ||
|
||
Example: a developer using OS X uses `near login` with access to a browser, whereas a validator runs `near login` on a CentOS box with no UI or browser. A user prompt may ask "Is this login from a computer with a browser?" The answer is then stored in key-value format in the project-level directory so it can be referenced later, skipping the prompt in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is global configuration, not per project
|
||
Example: a developer using OS X uses `near login` with access to a browser, whereas a validator runs `near login` on a CentOS box with no UI or browser. A user prompt may ask "Is this login from a computer with a browser?" The answer is then stored in key-value format in the project-level directory so it can be referenced later, skipping the prompt in the future. | ||
|
||
Besides answers to user prompts, shell-experience settings also store information about the last version of `near-shell` used in this project. For more information, please see [Upgradability](#upgradability). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like work for package.json
file not for a .<something>
folder. Usually .
folders are added to gitignore as they contain local configuration.
|
||
`.near-config/settings.js` | ||
|
||
In summary, "settings" are key-value pairs that are set by `near-shell` and relate to the behavior of how it operates in a given project directory. They do not relate to the development or deployment of smart contracts, except for providing default values like `accountId`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
default app name can also go to package.json
|
||
``` | ||
─ awesome-near-project ⟵ NEAR dApp | ||
├── .near-config ⟵ Stores shell-experience settings and connection configuration |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would really prefer not to have any settings folders inside the project folder.
│ │ ├── default.js ⟵ Default configuration is for development | ||
│ │ └── localnet.js ⟵ Custom connection added by user for localnet | ||
│ └── settings.js ⟵ Stores near-shell settings, how shell behaves when run in this project | ||
└── .near-credentials ⟵ Previously "neardev" this directory contains private key inforamtion |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
credentials for sure should not be here. this is both easiest way to delete them (delete & reclone the project) or expose them because forgot to add the folder into .gitignore.
``` | ||
─ awesome-near-project ⟵ NEAR dApp | ||
├── .near-config ⟵ Stores shell-experience settings and connection configuration | ||
│ ├── connections ⟵ Contains files used to connect and deploy a contract (except keys) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
connections should be global for the computer. In cases where developers really need their non-local and non-standard environment, we can be near-provider.js
or something in project folder to extend and be similar to truffle?
|
||
As shown in the directory structure earlier, this file is located in the project directory at: | ||
|
||
`.near-config/connections/default.js` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's stop use default, as it's confusing. we have testnet and mainnet
A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed. View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg. |
* `network` : The NEAR network (example, "betanet", "testnet", etc.) | ||
* `private_key` : The plain-text private key, used when `type = "unencrypted"` | ||
|
||
`near-shell` will look for keys in a specific order. This list is in the prioritized order and can be understood to mean, "if the key is not found here, then try the next location/store." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about not having a complex order, but instead a simple default + a way to set this in config
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we even need complex configurable order? Seems like it always can be:
project folder -> global folder -> protected OS key store
External stuff like Ledger
likely makes sense to specify explicitly (as it's generally not going to be based on account ID).
@mikedotexe I also wanted to mention that there are a bunch of cases, when user wouldn't want to sign transaction on the same device. People would use custody, CloudHSM, HSMs that our shell doesn't support, etc. E.g., near shell should be able to create transactions without signing or sending them. We should make this at least possible and as secure as we can. Note: that we haven't had this asked by people yet, so we shouldn't actually build this. More to make sure we have a way in our CLI APIs to add this. It's possible that people can just use near-api-js instead. |
A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed. View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg. |
Just wanted to x-link this request from a while back: near/near-cli#57 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have some more work to do here, reading through this and understanding everything at a deep enough level to help guide the spec. But I wanted to leave some thoughts now, as I might not have time to return to this for a couple weeks.
As I'm reading this, I'm also thinking about how we can make the interface for near-shell
match the interface for near-api-js
. I think there should be a basic set of functionality that is closely parallel in each. See near/near-api-js#341
I believe my suggestions here will help simplify the experience of near-shell on its own, and will furthermore make it simpler to have matching experiences in both shell & JS.
|
||
* `networkId` : Similar to an environment to work on (ex: 'staging') | ||
* `nodeUrl` : URL to the NEAR node | ||
* `contractName` : NEAR account name for the smart contract |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I doubt this should be here. It's commonly included in src/config.js
today, but that doesn't seem right to me.
Let's consider your example command below for calling a contract:
near contract view near-game topPlayers --env localnet
What would having contractName
in my config file give me? It doesn't make sense to call a function on a contract without specifying which contract. I doubt any sort of "default contract" makes sense.
|
||
`near contract view near-game topPlayers --env localnet` | ||
|
||
will call the function `topPlayers` on the contract `near-game` that is deployed to localnet. The connection information will be read from the file: `.near-config/connections/localnet.js`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think these connection settings are complicated enough to each justify their own file. Here's the alternative I propose:
-
We use sensible defaults based on
NEAR_ENV
/NODE_ENV
, no files required on local machine -
NEAR_ENV=betanet near do thing
is equivalent tonear do thing --networkId=betanet
(probably better to use--network-id
, dash-case, for consistency with other CLIs) -
You can override specific connection settings with
near config connection.nodeUrl https://myownnode.cool
. This follows the same interface asgit config
– by default, it sets it for the local folder. Unlike git, we probably don't actually need a whole folder for the local project, but can use a.nearconfig
file. So this command would result in a./.nearconfig
that looks like this:[connection] nodeUrl = https://myownnode.cool
Like
git config
, you can usenear config --global
to set the value in~/.nearconfig
instead. -
It doesn't make sense to store
contractName
in this file. The name of a contract/account is needed in specific commands; it is not part of configuring a connection. -
It doesn't make sense to store
networkId
in this file. Network ID is a command line switch (--network-id=x
) or an environment variable (NEAR_ENV=x
); giving this non-standard third way to set it adds confusion rather than clarity. This switch/env var gives you a way to select a predetermined set of connection configs; what does it mean for a network id itself to be part of the connection config that a network id is used to select? -
You can also set each individual setting with its own env var. If you have no
.nearconfig
file anywhere on your machine but you haveNEAR_CONNECTION_NODE_URL=https://myownnode.cool
inenv
, thennear
commands will use that forconnection.nodeUrl
. -
Any of these settings can be configured on a per-command basis, using a switch such as
--node-url
-
Settings priority order:
- per-command setting switch, e.g.
--node-url
- environment variable, e.g.
NEAR_CONNECTION_NODE_URL
- local
.nearconfig
- closest ancestor directory
.nearconfig
- default for network specified in
--network-id
switch - default for network specified in
NEAR_ENV
env var - default for network specified in
NODE_ENV
env var - default for
testnet
- per-command setting switch, e.g.
This results in a wonderfully simple first-use experience with sensible defaults, with customization options that fit a variety of workflows and that can be discovered as-needed.
Since the lookup priority order could potentially cause some confusion, I suggest one more item:
-
near config
default command lists current config and where the value came from. Example:$ NEAR_ENV=devnet NEAR_CONNECTION_NODE_URL=https://myownnode.cool near config connection.nodeUrl=https://myownnode.cool (from environment variable NEAR_CONNECTION_NODE_URL) connection.walletUrl=https://wallet.devnet.near.org (default for NEAR_ENV=devnet) connection.helperUrl=https://someotherhelper.dance (from ../../.nearconfig)
|
||
will call the function `topPlayers` on the contract `near-game` that is deployed to localnet. The connection information will be read from the file: `.near-config/connections/localnet.js`. | ||
|
||
Users may add, remove, or modify connection environments using commands detailed later in this spec. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I question how often people need to manage multiple customized environments. For people who do, providing the environment-variable-based customization option allows them to use tools like dotenv or simple shell scripts to quickly switch between sets of custom settings.
I want to dump my vision on near-cli and tooling design in general. (moving it from a private discussion to this public NEP discussion) TL;DR: A well-designed tool should drive its own usage instead of requiring the user to learn about everything before they can even start using it. I want to make the number of available options on each step to be as minimal as possible (as opposed to 18 top-level subcommands in near-cli JS), and I don't want to hide flows behind the scenes ( For example, I consider having the following command to do a transfer:
it is quite a lengthy command, but a user is going to be guided through it as follows:
I initially considered printing a help message here listing all the required options, but we can have it for
Offline flow example
Merged flows into a single scenario (just to understand the full picture)
The flow is lengthy with various branches for online/offline and early-exit scenarios, but it guides the users through quite confidently. Here is a demo of the CLI prompts from enquirer crate: Based on some feedback I want to clarify that the non-interactive mode will co-exist and it will just have the values prepopulated into the interactive flow, and those will not be prompted for. There are basically four scenarios I see based on the online or offline mode of fetching nonce and recent block hash, and automatic or manual signing process:
Extra examples: Ledger support:
Having all of those as nested subcommands we can have help message on every single level with only the relevant options to choose from. For example:
The next stop would be:
Notice how you progressed from |
FYI, I want to share that there has been great progress on my proposed design of the new near-cli. The project lacks the README and a proper show-case at the moment and lacks a few features here and there, but overall, it is on track, and it can be ready for release in April! you can also skip a few steps (or even all of them) providing more arguments: Full command to send a transaction to transfer tokens would be:
If you split it into lines for readability:
Demo: https://asciinema.org/a/wPtlwPbJveOIpwWEABGaCjQeT The project is implemented from scratch here: https://github.com/FroVolod/near-cli |
Do we still retain the possibility to use in-line command parameters instead of this interactive interface? This is an example of the many creations we have out there: https://gist.github.com/gaia/cff45baf3fa710a42c3fc4cdaafe8edc |
@jimmy3dita Look at the second screenshot, it is there 😄 |
This looks awesome. Thanks! I do wonder if some of the subcommands given to Maybe something like: near-cli \
construct-transaction \
from frol4.testnet \
to illia.testnet \
--online \ # could have alias `--network` and a `--no-network` flag to accompany it
--rpc testnet \ # RPC endpoint seems like a separate concern from `--online`
--action transfer-near-tokens 1NEAR \ # allow adding multiple `--action`s
sign-private-key \ # do we need this? maybe just use flags below
--signer-public-key ed25519:3oN25HE9tf9YUGXQyXCFB2rEtpomijG8ghVV2JvrA4zJ \
--signer-secret-key ed25519:... |
@chadoh I initially started with an idea that looks somewhat similar to yours: I ended up with tons of unstructured arguments, that are hard to follow for a newbie, and it was not even feature-complete (there is no online/offline mode, network selection, and signing options). In my new proposal, I decided to try a guided step-by-step process with exhaustive parameters matching, for example, when you select So I defined the constraints that I wanted to satisfy:
The first point is my main pain-point with the current near-cli (in JS): I find this approach much more friendly: (surely help messages needs to be added, but you have a clear idea which options you need to specify at a given stage) |
The next step would be to design a bunch of shortcuts, and I plan to group them into five categories: transfer, execute, add, delete, and view. All of them should have feature-complete CLI and interactive modes. I want to invite everyone to review the proposed flows. The order of parameters is fixed on purpose; I believe that will contribute to a clear mental model and also can guide users nicely through the process even when they don't know where to start, and especially, where to go next. See my previous posts and examples on how Transfer NEAR [done]near-cli transfer NEAR \
network testnet \
sender 'frol.testnet' \
receiver 'illia.testnet' \
amount '1 NEAR' \
sign-with-keychain \
send Note: I use Same command as a one-liner: near-cli transfer NEAR network testnet sender 'frol.testnet' receiver 'illia.testnet' amount '1 NEAR' sign-with-keychain send Execute a change method (function) [done]Note: the reasoning for the argument order is similar to the near-cli execute change-method \
network testnet \
contract 'meta.pool.testnet' \
call 'distribute_staking' '{}' \
--attached-deposit '0 NEAR' \
--prepaid-gas '1 Tgas' \
signer 'frol.testnet' \
sign-with-keychain \
send Note: we may consider the following options to the call arguments syntax:
Execute a view method (read-only function) [done]Note: to do a view method call, we don't need to create a transaction, we don't need a signer, we don't need keys etc. It is just a JSON RPC call to near-cli execute view-method \
network testnet \
contract 'meta.pool.testnet' \
call 'distribute_staking' '{}' Add Access Key (and "login") [partially done]near-cli add access-key \
network testnet \
account 'frol.testnet' \
public-key 'ed25519:...' \
grant-full-access \
sign-with-keychain \
send near-cli add access-key \
network testnet \
account 'frol.testnet' \
generate-keypair \
grant-function-call-access \
--receiver-id 'meta.pool.testnet' \
--allowance '10 NEAR' \
--method-names 'set_a, set_b' \
sign-with-keychain \
send Here is how near-cli add access-key network testnet generate-keypair grant-by-near-wallet One could argue that Delete Access Key [done]near-cli delete access-key \
network testnet \
account 'frol.testnet' \
public-key 'ed25519:...' \
sign-with-keychain \
send Stake [planned]near-cli add stake-proposal \
network testnet \
validator 'staking.poolv1.testnet' \
amount '10_000_000 NEAR' \
transactions-signing-public-key 'ed25519:...' \
sign-with-keychain \
send Create New Sub-Account [done]near-cli add sub-account \
network testnet \
owner-account 'frol.testnet' \
sub-account 'sub.frol.testnet' \
sub-account-full-access \
public-key 'ed25519:...' \
deposit '1 NEAR' \
sign-with-keychain \
send near-cli add sub-account \
network testnet \
owner-account 'frol.testnet' \
sub-account 'sub.frol.testnet' \
sub-account-full-access \
generate-keypair \
deposit '1 NEAR' \
sign-with-keychain \
send "Create" New Implicit Account [planned]near-cli add implicit-account \
generate-keypair This effectively does not issue any transaction, the only action this command does is generating a new keypair, and prints the public key as in hex, which can be used as an account id for transfers. The account is not going to appear on the network with this command, but the account id can be used to receive tokens, and later issue transactions on behalf of this account. Delete Account itself [done]near-cli delete account \
network testnet \
account 'frol.testnet' \
beneficiary 'illia.testnet' \
sign-with-keychain \
send Deploy Contract Code [done]near-cli add contract-code \
network testnet \
account 'frol.testnet' \
contract-file './frol.testnet.wasm' \
no-initialize \
sign-with-keychain \
send When you need to initialize the contract: near-cli add contract-code \
network testnet \
account 'frol.testnet' \
contract-file './frol.testnet.wasm' \
initialize 'new' '{}' \
--attached-deposit '0 NEAR' \
--prepaid-gas '1 TGas' \
sign-with-keychain \
send View Account Details [done]Displays the non-staked balance, staked balance, storage usage, whether a contract is deployed to it, and lists access keys: near-cli view account-summary network testnet account 'frol.testnet' Download Contract Code [done]near-cli view contract-code network testnet contract 'meta.pool.testnet' download './meta.pool.testnet.wasm'
near-cli view contract-code network testnet contract 'meta.pool.testnet' hash View Contract State [partially done]nenar-cli view contract-state network testnet account 'meta.pool.testnet' View Transaction Status [done]near-cli view transaction network testnet transaction-hash '7K1sVuE2tY8kjRj9TcututW6TGJTZAmryrprjv9hYpnQ' signer 'frol4.testnet' Note: due to sharding we need to specify the signer account id when lookup the transaction status |
https://near.cli.rs is now in a pretty usable state (it does everything that was laid out above, and has NEAR Wallet login and Ledger support) There is also the next iteration of a discussion on where to move forward with NEAR CLI: https://gov.near.org/t/new-near-cli-design-proposal/3616 |
Initial draft of NEP for NEAR Shell enhancements. Comments required and appreciated.