Off-chain evaluation of Solidity snippets

[chriseth]

Motivation

Some functions of a contract are often too expensive to be run on-chain (because they have expensive loops). Furthermore, for debugging and other purposes, it might be interesting to inspect the state of a contract with functions that are not provided by the deployed contract. Currently, all functions executed in the context of a contract have to be part of their deployed bytecode even if they are never run on chain, but only via eth_call.

Abstract

This proposal provides a way to execute snippets of Solidity code (or raw EVM bytecode) from the front-end part of a dapp in the context of a deployed smart contract and thus allows querying internals of the contract (computing a sum, etc) with a single call to the backend node and without having to pay gas for it.

Example js snippet (web3 and solidity compiler need to be present):

// usual contract object creation that also provides the full source code
var contract = ...;
contract.eval(
   "proposals[$1].votes",
   2,
   function(numberOfVotes) { ... }
);
// The 'eval' function substitutes '$1' by '2', compiles the snippet
// in the context of the contract (i.e. the state variable array
// 'proposals' is visible), calls the backend node via a special
// RPC endpoint and returns the result to the given callback
// function.

This can also be used to evaluate more complex expressions:

contract.eval(
  "uint voteSum = 0;" +
  "for(uint i = 0; i < proposals.length; ++i)" +
  "  voteSum += proposals[i].votes;" +
  "return voteSum;",
  function(voteSum) { ... }
);

The general idea is that it becomes trivial to retrieve the values of state variables even if they do
not have the public modifier (note that all internals of a smart contract are visible to the
outside world anyway, even if they are marked private). Furthermore, you do not have to
pay for the gas costs of deploying this "dead code".

Implementation

Changes required in RPC:

Provide an alternate implementation of the eth_call interface where the code to be executed
can be specified (similar to the delegatecall setting).

Changes required in web3.js / solidity-js:

Compile a snippet of solidity code providing access to state variables.

Discussion

The disadvantage here is that contract authors might write their code in a way that essential
components of a contract are not accessible to other contracts anymore, only to the off-chain world.

[mchlmicy]

Instead of rewriting the functionality of the RPC, Solidity, and web3 couldn't you modify the approach to simply return the state of the contract and then perform any calculations you want off chain? If these actions aren't going to be performed on the blockchain, then what is the benefit of encoding your logic in Solidity / Serpent? This also has the benefit that the user can choose which variables can and can't be returned off chain (wouldn't change the functionality of private). Instead you could simply have another identifier (between public and private) that denotes that data to be retrieved.

[chriseth]

@mchlmicy you are right, there is a tradeoff, but I think transferring the full storage content via RPC and running a js implementation of the VM in the frontend is worse than adding a tiny feature to eth_call. With the same argument, we could also just remove eth_call.

The benefit of using Solidity directly is that you do not have to switch the language. Think of a database: You use the same language for updating and for querying your entries.

I don't fully understand your reasoning behind "the user can choose which variables can and can't be returned off chain" - they all can be returned off chain, the functionality of "private" is not changed at all with this proposal.

[mchlmicy]

You definitely have a point about "private", which I somehow managed to miss. I guess my only response would be that I wouldn't use SQL (if we're going to use the databases metaphor) for anything more complex than basic summation - which is a low gas cost - and my understanding is that you're trying to offload gas-intensive non-state changing calculations (such as hashing).

[chriseth]

This is not about offloading gas-intensive calculations. It is about

1. offloading code-intensive calculations that will not be used on chain anyway
2. extending the flexibility of how to query a contract even after it has been deployed.

[vbuterin]

I fully support this. Also, in a light client context, you may want the solidity snippet to be sent to a full node, executed on the full node, and then the answer sent back to the light client along with a Merkle proof so the light client can repeat and verify the computation itself.

One oossible concern is that I feel like we are duplicating functionality. Namely, there would now be two ways of querying a contract using solidity in web3: (1) calling a const function, (2) this. In some cases, you want the EVM code to be on the chain so that other contracts can query it, but in other cases you do not. Perhaps having two ways to do the same thing isn't too bad if they have different properties, but it might hint at the fact that there's some even better abstraction lurking in the shadows that we haven't found yet....

[danfinlay]

I am a big fan of this concept. Allowing the client to specify sub-queries in solidity could allow sophisticated, client-side filtering that is performed on the server, allowing a much more efficient relationship between UI and the web3 provider than is currently possible.

This solves an important issue where right now clients often load entire contract states, which increasingly resemble databases, just to filter the results on the UI, creating very sluggish experiences.

Consider the case of a decentralized twitter: One person might want to filter by followers, another person might filter by tag, but neither wants to load all of the records just to do the filtering on the client side. By allowing custom snippets, the filtering logic itself can be part of a decentralized UI, while the main relationships and rules of the contract are kept on-chain.

Adding merkle proofs would be a good extra benefit, as this request could then be made to an untrusted peer.

[wighawag]

I am in favor of this too.
It turns out that many contracts require getter functions and all of these add to the contract bytecode size increasing the gas cost to deploy the contract.
For my own use case, removing the getter functions reduced the contract deployment gas cost by more than 30% (Probably a lot more if I considered the original contract version which has far more getter functions)

If we could get this getter functions out of the blockchain we could reduce the deployment gas cost of many contracts

[gavofyork]

Think it's better to pass EVM code rather than anything higher level. It would also be sensible to consider a delivery mechanism for these off-chain calls and how they can be integrated into the existing contract workflow. Could be as simple as ABIs providing an extra item type (localfunction or some such) which contain compiled code of any function tagged with the offchain modifier.

[chriseth]

@gavofyork passing EVM code was of course the intention. By the way, the corresponding RPC endpoint is currently being implemented and tested here: ethereum/go-ethereum#3612

[kumavis]

@chriseth could you add spec for the rpc request, perhaps in the format of https://github.com/ethereum/wiki/wiki/JSON-RPC

[obscuren]

We are currently implementing this as eth_injectCall. EIP will follow.

[pipermerriam]

I've just opened up EIP199 which tries to fill in the other side of this functionality from the perspective of contracts. This EIP nicely gives the ability for external entities to read arbitrary contract data. The goal of EIP199 would be to enable that same functionality for contracts.

[Georgi87]

This is a great fature, which can reduce complexity of smart contracts significantly. In my opinion one of the most important new additions to the RPC interface. @gavofyork Are you planning to integrate it into parity too? Just checked, but couldn't find the call in parity.

[chriseth]

The master branch of geth now has an extension of eth_call that can be used for this: ethereum/go-ethereum#19836

[karalabe]

Here are the docs for the API https://geth.ethereum.org/rpc/eth_call

[rmeissner]

the geth docs for this can now be found at https://geth.ethereum.org/docs/rpc/ns-eth (just if someone is looking for it ;) )

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[github-actions]

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.