ERC-1319 Smart Contract Package Registry Interface

[cgewecke]

Simple Summary

A standard interface for smart contract package registries.

Abstract

This EIP specifies an interface for publishing to and retrieving assets from smart contract package registries. It is a companion EIP to 1123 which defines a standard for smart contract package manifests.

Motivation

The goal is to establish a framework that allows smart contract publishers to design and deploy code registries with arbitrary business logic while exposing a set of common endpoints that tooling can use to retrieve assets for contract consumers.

A clear standard would help the existing EthPM Package Registry evolve from a centralized, single-project community resource into a decentralized multi-registry system whose constituents are bound together by the proposed interface. In turn, these registries could be ENS name-spaced, enabling installation conventions familiar to users of npm and other package managers.

Examples

$ ethpm install packages.zeppelin.eth/Ownership

const SimpleToken = await web3.packaging
                              .registry('packages.ethpm.eth')
                              .getPackage('simple-token')
                              .getVersion('^1.1.5');

Specification

The specification describes a small read/write API whose components are mandatory. It allows registries to manage versioned releases using the conventions of semver without imposing this as a requirement. It assumes registries will share the following structure and conventions:

• a registry is a deployed contract which manages a collection of packages.
• a package is a collection of releases
• a package is identified by a unique string name and unique bytes32 packageId within a given registry
• a release is identified by a bytes32 releaseId which must be unique for a given package name and release version string pair.
• a releaseId maps to a set of data that includes a manifestURI string which describes the location of an EIP 1123 package manifest. This manifest contains data about the release including the location of its component code assets.
• a manifestURI is a URI as defined by RFC3986 which can be used to retrieve the contents of the package manifest. In addition to validation against RFC3986, each manifestURI must also contain a hash of the content as specified in the EIP 1123.

Examples

Package Names / Release Versions

"simple-token" # package name
"1.0.1"        # version string

Release IDs

Implementations are free to choose any scheme for generating a releaseId. A common approach would be to hash the strings together as below:

// Hashes package name and a release version string
function generateReleaseId(string packageName, string version)
  public
  view
  returns (bytes32)
  {
    return keccak256(abi.encodePacked(packageName, version));
  }

Implementations must expose this id generation logic as part of their public read API so
tooling can easily map a string based release query to the registry's unique identifier for that release.

Manifest URIs

Any IPFS or Swarm URI meets the definition of manifestURI.

Another example is content on GitHub addressed by its SHA-1 hash. The Base64 encoded content at this hash can be obtained by running:

$ curl https://api.github.com/repos/:owner/:repo/git/blobs/:file_sha

# Example
$ curl https://api.github.com/repos/rstallman/hello/git/blobs/ce013625030ba8dba906f756967f9e9ca394464a

The string "hello" can have its GitHub SHA-1 hash independently verified by comparing it to the output of:

$ printf "blob 6\000hello\n" | sha1sum
> ce013625030ba8dba906f756967f9e9ca394464a

Write API Specification

The write API consists of a single method, release. It passes the registry the package name, a
version identifier for the release, and a URI specifying the location of a manifest which
details the contents of the release.

function release(string packageName, string version, string manifestURI) public
  returns (bytes32 releaseId);

Read API Specification

The read API consists of a set of methods that allows tooling to extract all consumable data from a registry.

// Retrieves a slice of the list of all unique package identifiers in a registry.
// `offset` and `limit` enable paginated responses / retrieval of the complete set.  (See note below)
function getAllPackageIds(uint offset, uint limit) public view
  returns (
    bytes32 packageIds,
    uint offset
  );

// Retrieves the unique string `name` associated with a package's id.
function getPackageName(bytes32 packageId) public view returns (string name);

// Retrieves the registry's unique identifier for an existing release of a package.
function getReleaseId(string packageName, string version) public view returns (bytes32);

// Retrieves a slice of the list of all release ids for a package.
// `offset` and `limit` enable paginated responses / retrieval of the complete set. (See note below)
function getAllReleaseIds(string packageName, uint offset, uint limit) public view
  returns (
    bytes32[] ids,
    uint offset
  );

// Retrieves package name, release version and URI location data for a release id.
function getReleaseData(bytes32 releaseId) public view
  returns (
    string packageName,
    string version,
    string manifestURI
  );

// Retrieves the release id a registry *would* generate for a package name and version pair
// when executing a release.
function generateReleaseId(string packageName, string version) 
  public
  view
  returns (bytes32);

Pagination

getAllPackageIds and getAllReleaseIds support paginated requests because it's possible that the return values for these methods could become quite large. The methods should return an offset that is a pointer to the next available item in a list of all items such that a caller can use it to pick up from where the previous request left off. (See here for a discussion of the merits and demerits of various pagination strategies.) The limit parameter defines the maximum number of items a registry should return per request.

Rationale

The proposal hopes to accomplish the following:

• Define the smallest set of inputs necessary to allow registries to map package names to a set of
  release versions while allowing them to use any versioning schema they choose.
• Provide the minimum set of getter methods needed to retrieve package data from a registry so that registry aggregators can read all of their data.
• Define a standard query that synthesizes a release identifier from a package name and version pair so that tooling can resolve specific package version requests without needing to query a registry about all of a package's releases.

Registries may offer more complex read APIs that manage requests for packages within a semver range or at latest etc. This EIP is agnostic about how tooling or registries might implement these. It recommends that registries implement EIP 165 and avail themselves of resources to publish more complex interfaces such as EIP 926.

Backwards Compatibility

No standard exists for package registries. The package registry currently deployed by EthPM would not comply with the standard since it implements only one of the method signatures described in the specification.

Implementation

A reference implementation of this proposal is in active development at the EthPM organization on Github here.

Copyright

Copyright and related rights waived via CC0.

[xinbenlv]

Thanks for composing this ERC, this is very interesting. I look forward to more updates on the draft. Cheers!

[tjayrush]

This is gorgeous. Thanks for posting it. Does it easily expand out to plain old-fashioned software distribution as well (or any digital content for that matter), or is there some reason why it would only apply to smart contracts? Asking out of ignorance more than anything else.

[gnidan]

@tjayrush in theory, EthPM works just fine with traditional software distribution, but it's overkill and not the use case for which it was designed.

You would include sources in your manifest file, as described by EIP-1123 (discussion: #1123), but leave out contract_types and deployments. See also: EthPM Use Cases for descriptions and links to examples.

Here's a basic manifest file you might use, as example:

{
  "manifest_version": "2",
  "version": "1.0.0",
  "package_name":"cool-program",
  "sources":{
    "./bin/a.out": "ipfs://Qma8ti...",
    "./src/index.cpp": "ipfs://QmRJHL..."
    /* ... other files to include in the package ... */
  }
}

[cbdotguru]

@gnidan unless we're migrating software registries to blockchain as whole right? I think a case can be made about this serving as a foundation to a much larger ecosystem. :)

What is the status of any of the web3js or any other current web3 api adopting this if we know? Are they going to wait for this to move up in status beforehand? Also, I haven't looked, but just want to verify that the ethpm registry is live on mainnet?

[cgewecke]

Hi @HACKDOM :)

• A Web3.js module is part of the work EthPrize is sponsoring at EthPM. We've talked to the maintainers at Web3.js and have their encouragement as well as advice about how to structure the API and integrate with an ENS library they're rolling out. There should be substantial progress on that over the next month.

• There's active work at EthPM on a reference implementation for a registry whose interface matches this ERC's (hopefully completed by next week). That project includes deployment and temporary ENS namespace registration scripts that should allow contract publishers to quickly deploy a registry and/or experiment with whatever registry model they're interested in. The standard proposed here is meant to make it possible for tooling to pull packages down from a multiple-registry ecosystem with the minimum fuss.

• Finally there's ongoing work at EthPM developing Python and JS libraries to facilitate package publication and installation in accordance with the package manifest standard proposed by EIP ERC-1123 – Revised Ethereum Smart Contract Packaging Standard #1123, tying all these pieces together.

In short there's WIP for all of this but ultimately community agreement about a reasonable interface is necessary before those implementations can be locked down.

The current EthPM Registry (which has been active for more than a year) is deployed to Ropsten at 0x8011df4830b4f696cd81393997e5371b93338878.

[hiddentao]

When publishing a new release it would be good if multiple manifest URIs could be provided, as fallbacks. So perhaps a function which allows you to associate additional URIs with a release id:

function addManifestUri(bytes32 releaseId, string manifestURI) public;

Then for reading, since we can't yet return an array of strings:

function getReleaseData(bytes32 releaseId) public view
  returns (
    string packageName,
    string version,
    string manifestURI, /* the first one */
    uint totalManifestURIs /* no. of URIs in total */
  );

/* Pass in 0-based index to get the URI at that index in the array */
function getManifestUri(bytes32 releaseId, uint index) public view return (string)

[cgewecke]

@hiddentao Thanks for this comment.

• Could you provide more information about why you believe fallbacks would be helpful?
• Is there an analog to this approach in other resource request patterns you know of?

[hiddentao]

@cgewecke

It's purely from a redundancy point of view. Also note that they wouldn't be mandatory - the slots are there if a package manager wishes to makes use of them.

I know that Maven allows for mirror repositories (essentially the same thing) -> https://maven.apache.org/guides/mini/guide-mirror-settings.html. And they list some good reasons:

• There is a synchronized mirror on the internet that is geographically closer and faster
• You want to replace a particular repository with your own internal repository which you have greater control over
• You want to run a repository manager to provide a local cache to a mirror and need to use its URL instead

[pipermerriam]

Looks like the return types for getAllPackageIds is incorrectly a bytes32 instead of a bytes32[].

[pipermerriam]

@hiddentao since all releases are content addressed, and we are expecting to see mostly ipfs/swarm uris, I don't see a lot of value in this. I understand there is some extra benefit in being able to provide redundancy, but for this use case, submitting both a swarm url and an IPFS url is not necessary because you can perform a transform between the two urls.

Another problem I see with this is if the different urls have different content hashes, which results in both ambiguity as to which is the right one, as well as additional load on package managers to do something like validate that all listed URLs have the same content hash.

Also, I believe that this can be accomplished by layer-2 solutions, especially since the content hash is required.

I don't think it has zero value by any means, but I don't think the value added is worth the extra complexity. 👎 from me.

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[xinbenlv]

cgewecke@ are you still working on this proposal?

[cgewecke]

@xinbenlv No, I'm not. This proposal was actually spear-headed by @pipermerriam.

@pipermerriam What should be done here?

[MicahZoltu]

Closing this for housekeeping purposes. Feel free to continue using this issue for discussion about EIP-1319.

Note: If 1319 is pulled out of stagnant, the discussions-to link should be moved to a thread on Ethereum Magicians.