Pseudo-Introspection, or standard interface detection

[chriseth]

This EIP is being further developed as #881

---

For some "standard interfaces" like the token interface ( #20 ), it is sometimes useful to query whether a contract supports the interface and if yes, which version of the interface, in order to adapt the way in which the contract is interfaced with. Specifically for #20, a version identifier has already been proposed. This proposal wants to generalize the concept of interfaces and interface versions to interface identifiers:

Every "standard interface" should be assigned a unique identifier (the sha3 hash of its source representation would be suitable) as a bytes32 value. Interfaces that support pseudo-introspection should provide the following methods. Note that this is especially targeted at contracts that can also provide multiple non-conflicting interfaces. This is useful if e.g. a standard token contract also allows a "cheque" functionality, which would have its own interface ID, or if the token does not support allowances.

/// @returns true iff the interface is supported
function supportsInterface(bytes32 interfaceID) constant returns (bool);
/// @returns the (main) interface ID of this contract
function interfaceID() constant returns (bytes32);
/// @returns a bit mask of the supported interfaces.
function supportsInterfaces(bytes32[] interfaceIDs) constant returns (bytes32 r) {
  if (interfaceIDs.length > 256) throw;
  for (uint i = 0; i < interfaceIDs.length; i++)
    if (supportsInterface(interfaceIDs[i]))
      r |= bytes32(2**i);
}

Example implementation and usage:

contract GenericInterfaceContract {
    function interfaceID() constant returns (bytes32);
    function supportsInterface(bytes32 _interfaceID) constant returns (bool);
    function supportsInterfaces(bytes32[] interfaceIDs) constant returns (bytes32 r) {
      if (interfaceIDs.length > 256) throw;
      for (uint i = 0; i < interfaceIDs.length; i++)
        if (supportsInterface(interfaceIDs[i]))
          r |= bytes32(2**i);
    }
}
contract SimpleToken is GenericInterfaceContract {
    bytes32 public constant interfaceID = 0xf2a53462c853462ca86b71b97dd84af6a2f7689fc12ea917d9029117d32b9fde;
    event Transfer(address indexed _from, address indexed _to, uint256 _value);

    function totalSupply() constant returns (uint256 supply);
    function balanceOf(address _owner) constant returns (uint256 balance);
    function transfer(address _to, uint256 _value) returns (bool success);

    function supportsInterface(bytes32 _interfaceID) constant returns (bool) {
        return _interfaceID == interfaceID;
    }
}
contract Token is SimpleToken {
    bytes32 public constant interfaceID = 0xa2f7689fc12ea917d9029117d32b9fdef2a53462c853462ca86b71b97dd84af6;
    event Approval(address indexed _owner, address indexed _spender, uint256 _value);

    function transferFrom(address _from, address _to, uint256 _value) returns (bool success);
    function approve(address _spender, uint256 _value) returns (bool success);
    function allowance(address _owner, address _spender) constant returns (uint256 remaining);

    function supportsInterface(bytes32 _interfaceID) constant returns (bool) {
        return _interfaceID == interfaceID || super.supportsInterface(_interfaceID);
    }
}
contract C {
    function retrieveRemainingAllowance(GenericInterfaceContract c, address owner) returns (uint) {
        if (c.supportsInterface(Token.interfaceID)) {
            Token t = Token(c);
            return Token(c).allowance(owner, msg.sender);
        } else if (c.supportsInterface(SimpleToken.interfaceID)) {
            return 0;
        } else {
            throw;
        }
    }
}

Note:
Since the EVM does not provide any guarantees for the semantics of contracts, any such information returned by other contracts can only be used as a guideline.

Note2:
The convenient access to the interface ID via Token.interfaceID is not yet supported by solidity ( ethereum/solidity#1290 ).

Credits: This is the result of a discussion with @konradkonrad and @frozeman

[Arachnid]

This would be particularly useful for ENS resolvers, and would allow removing the has() method, which implements an equivalent (ENS-specific) operation.

supportsInterfaces seems like a lot of extra overhead for relatively little gain over just calling the contract multiple times, however, especially given the low number of interfaces one is likely to need to check.

[Arachnid]

A couple of other suggestions:

1. interfaceID assumes the existence of a 'primary' interface, and doesn't seem a-priori terribly useful. Unless there's compelling use cases for it, I'd suggest dropping it.
2. Making the interface ID the 32 bit XOR of all the function signatures the interface contains would make calculating it simpler, relying only on existing code to generate function hashes and simple arithmetic ops.

[Arachnid]

Discussing this with Chris in the Solidity channel, he pointed out that an interface is more than its ABI; it's possible for two interfaces with the same set of function signatures to have different expectations of how they operate (especially since function signatures don't include parameter names!). So, the interface ID should be a pseudorandom value with low probability of colliding with any other interface ID; it doesn't need to bear a strict relationship to the source (and arguably, shouldn't, so people don't make assumptions about it being usable as some kind of test).

I agree - so ignore my earlier suggestions about constructing the interface ID from the constituent functions.

Edit: On the other hand, Go does 'duck typed interfaces' and generally doesn't have a problem; although you can have two interfaces with the same ABI but different intents, the chances of both of them being relevant in a given scenario seem vanishingly small. For instance, it's easy to avoid a collision in interface definitions for ENS resolvers, and nobody's going to ask an ENS resolver if it implements, say , a token-related interface.

[chriseth]

After some more discussion, I think we did settle on just using the xor of the function identifiers that are part of the interface as a canonical identifier.

[VoR0220]

@chriseth @Arachnid I am of the belief that we should be either copying how Go does this or how Rust does this in regards to our interfaces. They should be their own type. Something looking like this:

contract C {
    interface I {
         foo(uint256) returns (uint256);
         bar(bool, string) returns (bool);
    }
}

contract A {
     struct MyStruct {
          function foo(uint256) returns (uint256);
          function bar(bool, string) returns (bool);
     }
     MyStruct _myStruct;
     D someContract;
     function a_call() {
          d.f(_myStruct); //passes in _myStruct as interface to D
     }
}

contract B {
  E _e; 
  D _d;

  function call_D() {
       _d.f(_e); //passes in E to D and thus will clear because functions align.
   }
} 

contract E {
    function foo (uint256) returns (uint256) {
       //some function
   }
   function bar (bool, string) returns (bool) {
     //some function
   }
} 
contract D is C {
     function f(_interface I) {
            switch type(_interface) { //switch on the types of objects passed into the interface (i think this should be mandatory to some extent)
                 case A:
                      uint a = _interface.foo(1);
                 case B: 
                       bool truth = _interface.bar(true, "something");
                 default: throw;
            }
     }
}

I'm not sure about xor-ing it...that may work, I'm just not sure I understand it yet, need to look. What I think we can do for the type assertion is to hash for these types based on their actual function opcodes that they are utilizing/generating on the stack portion. This will make it so that we can verify that only these contracts are calling externally and can make the interface quite a useful safety mechanism.

[Arachnid]

@VoR0220 I think what you're talking about is language design, and thus orthogonal to this EIP, which is mostly about how to implement the feature at the EVM level.

[VoR0220]

I believe you might be correct. Will reread this in the morning.

[VoR0220]

Okay. 2 Months later, I have reread it and with some help I think I understand the proposal better. Here's my question. How might we distinguish between different versions of a contract interface? Would that need to be included? Or would it be "Version independent"?

[jbaylina]

Here are the modifications that I would do in the standard:

1.- I would keep GenericInterface very simple with just one single method: supportsInterfaces(bytes32). I would not include the other two methods: supportsInterfaces and interfaceID.
2.- This interface should always return true when called with 0xf1753550 as a parameter. (This is the interface that implements supportsInterfaces(bytes32).
3.- This function should always return false when called with 0xff···ff parameter.

With this definition, You can test if a contract implements EIP165 or not by trying to call supportInterface for GenericInterface (Should always return true) and an Invalid interface (Should always return false). If an exception, out of gas or the results are not true/false, then this means that this contract does not implement EIP165

For saving gas, a single generic EIP165Cache contract can exist in the blockchain.

You can see an how this implementation would be here:

https://github.com/jbaylina/EIP165Cache/blob/master/contracts/EIP165Cache.sol

[Arachnid]

I broadly agree, although I'm not sure it's necessary to be able to test supportsInterface explicitly like that; if you call it and it throws, it's safe to say the method isn't supported (throwing is the default fallback behaviour for some time now).

[jbaylina]

@Arachnid I believe that there are some contracts (like wallets or normal addresses) that don't throw. They just do nothing.
In those old contracts, they may return false or true. What I am testing is that they return true in a "must Implement" interface and false in a "must not implement" interface. This way we guaranty that it will work in most (if not all) old contracts.

If you agree on this, I will make some tests with the most used contracts so that they return false. We will see if this "double check" is necessary or not.

[jbaylina]

@Arachnid Another question, I see that in the ENS implementation, the fingerprint is:

function supportsInterface(bytes4 _interfaceID) constant returns (bool);

instead of

function supportsInterface(bytes32 _interfaceID) constant returns (bool);

I think it's better to use bytes32 instead of bytes4 even if we use only 4 bytes. This will give space for future ampliations.

And may be it's too late now, but I think it would be good to also return true for thesupportsInterface interface:

0xf1753550 -> supportsInterface(bytes32)
0x01ffc9a7 -> supportsInterface(bytes4)

[Arachnid]

bytes4 was what was agreed on in the Solidity channel, but @chriseth never updated the draft accordingly. I don't think it's possible to change how many bytes we use without breaking backwards compatibility, so I'd vote for keeping it as bytes4.

There's certainly no harm in having contracts return true for the supportsInterface interface. I'll update the ENS contracts accordingly.

Since you're working on this, any interest in submitting a PR that formalises it?

[jbaylina]

@Arachnid: You mean a PR for the standard or for the ENS?

[Arachnid]

I mean a pr for the current understanding of this standard.

…

On Wed, 3 May 2017, 18:42 Jordi Baylina, ***@***.***> wrote: @Arachnid <https://github.com/Arachnid>: You mean a PR for the standard or for the ENS? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#165 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABFyUtSWd1ksEn-ZlhPOtymbZZtsX7xks5r2LyFgaJpZM4KjYmN> .

[jbaylina]

@Arachnid Ok, I'm going to prepare it!

[jbaylina]

I have the first draft of the Specification Here:
https://docs.google.com/document/d/16AC8gBylYgPE5zr9C4mz2fS1oYLHQf1o8sGjxmw8Juw/edit?usp=sharing

I have also created and tested the EIP165Cache contract Here:
https://github.com/jbaylina/EIP165Cache

Please, comment/correct in the doc before creating the formal PR.

[Arachnid]

Looks like a good start, but it would be much easier to leave comments against the PR than in a Google Doc (and they'll be preserved for posterity).

[jbaylina]

Ok. Then I'll Create the PR.

[jbaylina]

Just deployed the contract in testnet:

https://ropsten.etherscan.io/address/0xb60739c904b6d202c7cb2ba4be0fd4b733374c4e#readContract

[fulldecent]

Right now we have a few solutions for what we are trying to solve:

	165-VIEW	165-PURE	820
Can express if you implement an interface	YES	YES	YES
Can change if you implement an interface	YES	NO	YES
Can accurately cache results with a singleton contract	POSSIBLE	YES	YES
Can accurately cache results without a singleton contract	NO	YES	NO
Can delegate compliance to a separate contract	NO	NO	YES
Uses function signatures as interface	YES	YES	NO
Uses string as interface	NO	NO	YES
Allows third-party to change interface implementedness	POSSIBLE	NO	YES

[fulldecent]

VERSION 3: Added assembly parts

---

Also, we could just be a boss and do actual introspection on the blockchain from a contract.

• This only works from Solidity
• This exploits the fact that Solidity inserts function dispatch boilerplate at the beginning of all contracts
• Function dispatch knows of all defined functions
• This can return false negatives if the contract has some fancy assembly in its fallback function (i.e. it's own function dispatch)

pragma solidity ^0.4.19;

contract ContractIntrospector {
    enum Result {No, Yes, PleaseCallAgainToContinueSearching}
    mapping (address => mapping(bytes4 => bool)) private contractFunctionSelectorSupported;
    mapping (address => bytes4[]) public contractFunctionSelectors;
    mapping (address => uint) private searchProgress;
    mapping (address => bool) public doneSearching;

    function getFunctionSignature(string _function) external pure returns (bytes4) {
        return bytes4(keccak256(_function));
    }

    // Save lookup table in memory
    function getBytesForOpcodes() private pure returns (uint256[256] retval) {
        // Initialize bytesForInstruction
        // Read the Yellow Paper, find α for each opcode 
        for (uint i = 0; i < 256; i++) {
            retval[i] = uint256(-1); // Default return value
        }
        retval[0x00] = uint256(-1); // HALT
        retval[0x01] = 1 + 32*1; // ADD
        retval[0x02] = 1 + 32*1; // MUL
        //...
    }

    function getSize(address _addr) public view returns (uint256 _size) {
        assembly {
            _size := extcodesize(_addr)
        }
    }
    
    function getCode(address _addr, uint256 _start, uint256 _length)
        public
        view
        returns (bytes _code)
    {
        _code = new bytes(_length);
        assembly {
            extcodecopy(_addr, add(_code, 0x20), _start, _length)
        }
    }
    
    function howManyBytesCanWeProcessWorstCaseWithRemainingGas()
        private
        returns (uint256 _size)
    {
//TODO: implement this better        
        return 20;        
    }
    
    function doesContractSupportFunctionSelector (
        address _addr, 
        bytes4 _functionSelector
    )
        public
        returns (Result)
    {
        if (contractFunctionSelectorSupported[_addr][_functionSelector]) {
            return Result.Yes;
        }

        uint contractSize = getSize(_addr);
        uint start = searchProgress[_addr]; // starts at zero
        if (start >= contractSize) {
            return Result.No;
        }
        uint256[256] memory bytesForOpcode = getBytesForOpcodes();
        uint length = howManyBytesCanWeProcessWorstCaseWithRemainingGas();
        bytes memory code = getCode(_addr, start, length);
        uint ptr = 0;        
        while (ptr < length) {
            if (code[ptr] == 0x63) { // PUSH4
                bytes4 selector = code[ptr+1] | // TODO: is endian here correct?
                    code[ptr+2] << 8 |
                    code[ptr+3] << 16 |
                    code[ptr+4] << 24;
                contractFunctionSelectorSupported[_addr][selector] = true;
                contractFunctionSelectors[_addr].push(selector);
                if (selector == _functionSelector) {
                    searchProgress[_addr] = start + ptr;
                    if (start + ptr > contractSize) {
                        doneSearching[_addr] = true;
                    }
                    return Result.Yes;
                }
            }
            if (code[ptr] == 0x5b) { // JUMPDEST
                searchProgress[_addr] = uint256(-1);
                doneSearching[_addr] = true;
                return Result.No;
            }
            ptr += bytesForOpcode[uint256(code[ptr])];
        }
        if (ptr >= length) {
            searchProgress[_addr] = uint256(-1); // the maximum uint value
            return Result.No;
        }
        return Result.PleaseCallAgainToContinueSearching;
    }
}

Yeah, this is mostly a joke/though experiment. But maybe you were thinking it too, so just wanted to say it.

[recmo]

Haha, yes, I non-seriously considered decompiling too. (Btw, the current selector prequel has a gas cost linear in number of functions, but low constant cost is quite doable using a map, but would break decompiling approaches.)

Just to be clear. With 'heuristics' in my earlier comment I meant something similar to @jbaylina's noThrowCall to check for the existence of never-throw functions like owner, balanceOf, allowance, name, etc. I think such a heuristic would be able to distinguish the many of the current interfaces that don't support EIP165 yet.

I'm not a big fan of EIP-820. 1) It complicates deployment, which will complicate development and testing and hurt adoption. This standard is pretty much code-copy paste to implement, easy for adoption. 2) EIP820 in its current form relies on user-provided strings for ids. I foresee contracts incorrectly claiming to support the ERC20 interface. 3) It allows for other contracts to implement an interface on behalf of the contract, this adds an extra layer of complication to using interfaces on contracts as you now need to support a layer of indirection. This indirection can change at any moment, so you can not cache it. This seems like a very big gas cost overhead to me. In fact, this indirection can change mid-transaction. Finally, what happens if a malicious token contract 'claims' that its 'transfer' interface is implemented by an unrelated valuable token?

My favorite is 165-view. The only problematic caching case is when a contract drops support for a interface. (Adding can be solved by manually invalidating cache, which is trustless) But this is already a problematic case without EIP165. Contracts that plan to do this should say so in advance (lest developers will depend on an interface) and when they plan in advance they can also say not to rely on interface caches.

[fulldecent]

Quick updates:

• I need to hear from somebody who would vote on this ERC-165 as to their thoughts on which direction we should take.
• That gives me a reasonable hope that my effort here can result in a win.
• We can harmonize interface names with EIP 820: Pseudo-introspection using a registry contract. #820
• Maybe we should separate interface names from XOR function selectors
• I built an awesome Solidity decompiler, almost works. Does introspection, not pseudo
• Use this.f.selector. It is way better

[fulldecent]

OK. I am going to go forward with writing this up for EIP165-VIEW variant. PR coming.

[fulldecent]

@chriseth Responding to your offer to hand off this work, I have accepted the challenge and here is #881.

If you believe my effort should be canonical, would you please add a large note to the top of this issue:

THIS DISCUSSION MOVED TO #881

[fulldecent]

@recmo Sorry, that link broke, can you please send again?

[fulldecent]

Hello channel. #881 is very far along. I have described it faithfully as was discussed here ("165-VIEW" as above). As yet, I have seen no controversy in the 165-VIEW approach.

Previously there was an inconsistency with the 165-VIEW approach and the EIP165Cache from @jbaylina. I have resolved this by rewriting the cache.

Previously there was an open item from 165-PURE discussion about the gas cost of using an interface mapping versus using the PURE approach. Both implementations are shown and the gas costs are discussed.

Please support me by going to #881 and reacting with a party emoji 🎉 if you believe this EIP should be accepted. Acceptance is NOT final. The significance of acceptance is detailed in EIP-1:

If the EIP collaborators approve, the EIP editor will assign the EIP a number (generally the issue or PR number related to the EIP), label it as Standards Track, Informational, or Meta, give it status “Draft”, and add it to the git repository.

[fulldecent]

@jbaylina,

The supporting documentation for RJ @VoR0220, Hudson @Souptacular and Griff @GriffGreen
being listed as authors in this standard are inside your Google Doc version at https://docs.google.com/document/d/16AC8gBylYgPE5zr9C4mz2fS1oYLHQf1o8sGjxmw8Juw/edit#heading=h.kkpgop5ftnhc

I do not have access to that history. FILE-> VERSION HISTORY -> PRIOR VERSIONS.

Would you please review this history and attest that these three individuals should be listed as authors?

If you recommendation a deletion, please suggest it here and at #881. If you can make the attestation, this will be helpful to document for posterity.

[nicksavers]

The PR for this in #881 has been merged

[mwherman2000]

FYI: On the NEO blockchain, this requirement was solved in the following way: https://github.com/neo-project/proposals/blob/master/nep-10.mediawiki

[frangio]

There is a bug in the code in this EIP caused by reading from reused memory that is not zeroed out.

success := staticcall(
                    30000,         // 30k gas
                    _contract,     // To addr
                    x,             // Inputs are stored at location x
                    0x24,          // Inputs are 36 bytes long
                    x,             // Store output over input (saves space) <-----
                    0x20)          // Outputs are 32 bytes long

result := mload(x) // <-----

If the call reverts, or if it succeeds but does not return anything (e.g an EOA), x will contain erc165ID. I believe it will also contain a bad value if the call returns less than 32 bytes. This results in an incorrect doesContractImplementInterface implementation because result is compared as a number with 1 and 0.

OpenZeppelin's implementation does not have this bug. Has anyone deployed the code in the EIP?

[fulldecent]

Thanks for sharing. So if the call returns less that 32 bytes wont that just only lay down the bytes that returned? ERC-165 is pretty extensive with the three-part test. I can't think of a case that would break that implementation when an unexpected short response is produced.

Also interested to hear if anybody is use the reference code as-is.

In general, I will ask please that if allegations of a EIP are raised after final status, that they include runnable test cases. Preferably in a minified repository that also runs with Travis CI or similar.

[frangio]

It's true that the three-part test mitigates the bug in a way that it doesn't seem to have any practical consequences.

[daweth]

Is interface detection implemented automatically, or does registration have to manually occur?