IMPROVEMENT: NFT Metadata

[joshuahannan]

Issue To Be Solved

NFTs always have some sort of metadata associated with them. Historically, most of that metadata has been stored off-chain, but we would like to create a standard for metadata that allows all metadata to be stored on-chain so everything about the NFTs is truly decentralized.

This issue is meant for discussion about the possibilities of the solution. More documentation and examples will be added as we research and discuss more.

I am currently leading the charge on this, but I have a lot on my plate and don't know if I can give this issue the love it deserves, so if someone from the community wants to lead, I would love to speak with you!

Suggest A Solution

• Ideally, we'd like to try to have some sort of self-describing json format for the metadata, but more research needs to be done on how we could support this in Cadence and the flow sdk.

Context

The Avastars project is an interesting project on Ethereum that that we could potentially take inspiration from for our metadata.

[MiyaSteven]

I can solve this problem if you have the patience to walk me through some basic knowledge necessary to understand what all the moving parts are.
Questions I have:

1. Why has the metadata associated with NFT's always been stored off-chain?
2. What are current blockers that create this issue?
3. In a perfect system where everything was possible, how would the metadata of an NFT be stored?
4. Is it possible for a User to store their own metadata for their owned NFTs? Why or why not?
5. What has worked so far in existing projects that has moved us closer to solving this issue and where did they leave off and why could they not completely accomplish this?
   If you answer these, my next response might also be a wave of questions since I am pretty new to understanding data flow/storage, but my problem solving skills in general are solid as long as I understand the entire picture/pieces. Cheers, WK

[joshuahannan]

@MiyaSteven Sorry for the late response. I can answer a few of your questions, but I'll need to find some more help answering some other ones, as I am not super familiar with the state of metadata storage in other blockchains

1. Metadata associated with NFTs has been stored off-chain because on chain storage is usually expensive and hard to manage. Gas fees for storing on ethereum were high so the common recommendation was to only store the things that absolutely needed to be stored. Also, Solidity didn't provide built in libraries to manage images, videos, json, and other complex file types so apps decided to just keep most of this in their own off-chain database
2. Similar to my last answer, the main blockers are that Cadence does not provide an easy way to manage json and other complex files, so we need to either build it into to the core of the language, or we need to write functions in our smart contracts to manage these.
3. This is a hard question to answer, but I think the consensus would be to have a customizable standard for storing self-describing json files in resources that can hold almost any type of metadata, such as numbers, text, images, videos. The contract would be able to parse some of these and return their values to the caller.
4. It is definitely possible to store the metadata for an NFT in the NFT itself, because you could just add a field or fields to the NFT that holds the metadata in an encoded format.
5. Other projects on Ethereum have been trying to solve this issue and are seeing decent success. Some examples I know of are Avastars and Chainfaces so I would recommend checking those out.

Thanks for reaching out and feel free to ask more questions if you still need help!

[pizzarob]

I'm of the opinion that metadata should be flexible and scalable. You don't get any of those things with current on-chain ethereum solutions.

Metadata should be flexible
As a creator of an NFT I should be able to update the metadata as I see fit to support my project without having to rewrite my smart contract to change the shape of my data, or submit a transaction to update the data.

Metadata should be scalable
This ties into flexibility as well, but metadata needs to be scalable. As NFTs become more mainstream and projects become larger not only does metadata need to be flexible and upgradeable, but this flexibility needs to scale. If my project has 10 million NFTs it doesn't make financial sense to have to update 10million NFTs on chain (my experience is on Ethereum).

My platforms solution for this is to use a traditional storage space and have a server that dynamically retrieves metadata based on the contract address and token ID. This allows for metadata to be updated at scale to provide flexibility for enterprise projects.

If this storage space was decentralized and there could be a history of metadata updates I think that would suffice. Updating metadata needs to be cheap, fast and easy.

[bjartek]

Could a start to this be to support labels that are pure {String,String} Dictionary and expose that in the public interface and collection interface. Just doing that would make it a lot easier to experiment with.

Metadata should have a schema that could be migrated

Adding support for migrating the data in the schema to a new format would be a very nice feature here. A binary format like avro supports this. There are lots of examples on how people use it in kafka to allow sending messages that are backwards compatible with old formats.

What is the size of a flow block gonna be? Will it be feasible to store a pretty large SVG on-chain.

[joshuahannan]

We originally had a {String: String} dictionary in the NFT standard, but we felt that is wasn't necessary since we wanted the standard to be pretty minimal and didn't want to force a relatively weak standard for metadata on the users of the standard. But you can experiment with that in your own contracts that implement the NFT standard. That is similar to what top shot does.

Yes, I totally agree that the schema should be able to be migrated. We are discussing a process for how contracts are stored and upgraded in accounts in onflow/cadence#221 (comment), which includes a part about how data from contracts is migrated to the new version. We'd love some feedback or ideas if you have any.

We aren't totally clear on what the size of a flow block will be but I feel fairly confident that you'll be able to store larger files like that as long as you have the money to pay for storage.

[bjartek]

Another thing me and @psiemens talked about yesterday was implicit support for something like IPVS.

When sending a transaction with an specific type that field is not sent on chain but sent into IPVS, and if the field is accessed it is pullef from there. Or something like that.

[ericelliott]

1. Why has the metadata associated with NFT's always been stored off-chain?

Cost per GB of storage on Ethereum is measured in the millions of dollars. For comparison, price per GB of storage on cloud services like AWS is $0.05/month. Storage architecture must be radically different on Flow to make this idea even remotely cost-effective. Currently, it's not so hard to fire up an IPFS pinning service, and store only the hash on-chain (which is a URL in IPFS-land).

2. What are current blockers that create this issue?

I don't know the Flow architecture well enough to answer this question.

3. In a perfect system where everything was possible, how would the metadata of an NFT be stored?

Different NFTs can represent radically different information, needing radically different data structures. There is no one-size-fits-all solution for NFT data representation.

4. Is it possible for a User to store their own metadata for their owned NFTs? Why or why not?

That would effectively shard Flow storage. How would you achieve consensus on data representation?

5. What has worked so far in existing projects that has moved us closer to solving this issue and where did they leave off and why could they not completely accomplish this?

Store real data on IPFS. Encode only the hash on the blockchain. The primary blocker for this on Ethereum is the cost of storing data on Ethereum ($millions/GB).

[cybercent]

An idea would be to use bson (http://bsonspec.org) , and store metadata on chain.

Edit:
Instead of imposing a predefined schema, we would let the schema to be decided by the developer but add a easy way to filter items (NFTs) in a collection based on the fields the metadata would contain.

1. The NFT standard would have a metadata field of type bson.
2. Cadence would provide methods to filter items in a collection based on metadata.

Reference for querying bson data in Postgres
https://www.postgresql.org/docs/9.6/datatype-json.html

Example:

metadata = {
 "firstName": "Kevin",
 "lastName": "Durant",
  "season": 2019,
  "team": {
       "name": "Nets", 
       "primary_position": "Small forward"
   }
}

Query examples

Filter by presence of a key.
metadata ? "firstName"
Response: all items in the collection that have the key firstName set.

metadata ?| ["firstName", "season"]
Response: all items in the collection that have one of the keys firstName OR season set. Hence the |.

metadata ?& ["firstName", "season"]
Response: all items in the collection that have both keys firstName ANDseason set. Hence the &.

Filer by checking inclusion of one JSON into another one.
metadata -> team @> {"name": "Nets"}
Response: all items in the collection that have the team name set to Nets.

metadata -> team @> {"name": "Nets", "primary_position": "Small forward"}
Response: all items in the collection that have the team name set to Nets and the team position set to Small forward

Field names
If needed, the on-chain metadata could have short names for keys to save on storage.
The developer could make available the desired mapping on his website. The mapping would be used by other dapps and wallets that want to interact with NFTs created by the developer and need to display that info to the end user.

<head>
<meta name="flow-0x01.NBATopShot" content="https://my-dapp.com/0x01.NBATopShot.json">
</head>

0x01.NBATopShot.json could look like this:

"en": {
 "firstName": "Player First Name",
 "lastName": "Player Last Name",
  "season": "NBA Season",
  "team": {
       "name": "Team Name", 
       "primary_position": "Player primary postion"
   }
}

[alxocity]

Not sure if there should be a metadata standard. Perhaps just a metadata field that is a custom Metadata resource per NFT. Then any client can traverse this resource to get relavent metadata. The contract can also define a metadata template field for recommendations on displaying the data using handlebars or similar; although, these metadata display templates are probably better suited as scripts rather than contract functions.

Tldr; seems metadata can be stored however the dev wants, but have a standard for the metadata script used to read and format the data.

[alxocity]

And building standards for all complex data types will be beneficial. Although, you can jam almost anything into a string ;)

[ericelliott]

Many schemas that may be suitable for NFTs have already been defined and are available at schema.org including images, video, audio, and so on.

ERC-721 specifies a simple metadata schema that looks like this:

{
    "title": "Asset Metadata",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "description": "Identifies the asset to which this NFT represents",
        },
        "description": {
            "type": "string",
            "description": "Describes the asset to which this NFT represents",
        },
        "image": {
            "type": "string",
            "description": "A URI pointing to a resource with mime type image/* representing the asset to which this NFT represents. Consider making any images at a width between 320 and 1080 pixels and aspect ratio between 1.91:1 and 4:5 inclusive.",
        }
    }
}

ERC-721 tokens get associated with their metadata on-chain using a tokenURI field in the token contract. Further information such as description and image are parsed from the corresponding JSON record referenced by URI. (See above). That URI generally references a JSON document and assets on IPFS.

Open questions:

• How expensive is it to host data per GB on Flow? (Please show your work on calculating this - e.g., Write a script that will check storage costs with current blockchain data and show how we can estimate those costs into the future as the Flow blockchain grows).
• How do current applications such as NBA Top Shot associate Flow NFTs with the creative assets associated with them?
• Should there be a metadata table to associate metadata with NFTs (so we can make many-to-one relationships?)
• Should metadata be stored on something like IPFS instead of on-chain (so we can separately optimize for data storage use-cases)?
• Is it better to bring storage in-house into the Flow blockchain ecosystem and standardize so we can have more control over concerns such as deletion/shredding capabilities and performance characteristics?
• How do we address data stored on Flow? (IPFS multihash addresses may be good prior art to build on here) - same data always points to same multi-hash, which can simultaneously prevent data duplication on the blockchain and prevent/discover unauthorized use of IP.
• If it is discovered that a bad (or naive) actor violated IP rights and uploaded proprietary or confidential information, how can you erase/scrub/shred data that's already stored on the blockchain? (It should not be possible to permadox a user by recording legally protected, personally identifying information to a blokchain). Can we use some sort of strong encryption/data sharding, and shred the keys if the consensus is that somebody acted in bad faith when they added the data to the blockchain in the first place?

[pizzarob]

Using standard schemas for different types of assets from schema and specifying these from the start is a great idea. Case in point - the ERC-721 spec on Ethereum has a limited metadata schema definition and now different platforms have different metadata shapes for different media types which ruins the off-the-shelf interoperability between these platforms.

[ericelliott]

@pizzarob Yep. I just added the ERC-721 metadata schema to my comment, above. I agree that mappability to ERC-721 is an important feature to strive for.

[MiyaSteven]

Just catching up on this issue. RE: Joshua's June 16th's comment

Questions:

• On-Chain storage is expensive and hard to manage -> does this apply to Flow Blockchain?
• What did Apps using solidity do in order to manage metadata since there were no built in libraries for this?
• Would writing functions in Flow’s smart contracts to manage json and complex files allow things like Node, ReactJS or other tools/packages/libraries to be created in order to allow Cadence/Flow to stay relevant like how they have impacted JS language to stay competitive over time? If not, how could something like this be possible with Cadence/Flow in order to maintain developers QOL and everything they build for satisfactory UE + retention?
• If we store metadata for an NFT in the NFT itself by adding fields to the NFT that hold the metadata in an encoded format, what are the potential issues that could be caused as a side effect of this? Storage issues for users/apps holding the NFT’s, speed, errors from transactions as the NFT’s move across different platforms within Flow/Environments/Platforms?
• Also, if we store metadata in an NFT itself, could users store their own NFT’s to be responsible/pay for their own storage instead of the platforms/DApps themselves? This is how the real world works, when I buy a product or basketball card, I need to be responsible for keeping it stored safe and sleeves to protect its quality…understanding these are digital resources but the same responsibility should be “normal” for users to hold and store their own resources. It’s a lot easier than storing IRL that’s for sure and without the risk of damaging the resources which could be a huge selling point into this ecosystem from tangible “mainstreams normal” to intangible/digital “this evolutionary concept”.
• Is there a way to create a general model/schema for NFT’s metadata? Where inputs are an optional integer, optional string, optional any type of complex file datatype that could be added into a resource?

[bjartek]

I make a little something to see if Mixins could be used to solve this. https://github.com/bjartek/flow-nft-mixin

[ericelliott]

@bjartek I think that's an interesting idea to add additional features to the NFT, such as artist royalties. I think smart contract composability is a great feature.

Are smart contract mixins idiomatic on Flow? I'd love to read more about them with more descriptions of how they work and more examples of using them.

[bjartek]

Mixin might not be the correct term to use here. But interface, contract, capability are already used so it just used mixin.

My refence for it is https://docs.scala-lang.org/tour/mixin-class-composition.html

Would Trait be better?

[bjartek]

It might be better for Trait just to be an interface that requires some methods. Like

• data(): AnyStruct{}
• html() : String? return HTML version if appropraite
• string(): String, return text description
• type: String

Aso. Then you do not need for it to own a resouce since it can be the resource itself.

[dete]

Here are a series of observations, opinions, and facts that will hopefully help this discussion:

• ERC-721 made metadata optional, and I continue to think that this was the right call. Remember that any interface standards define the minimum functionality required by implementors, and don't limit additional functionality. Whenever defining a standard, the goal should be to find the absolute minimum number of hard requirements knowing that you are forcing those requirements on EVERY implementation. Metadata usually makes sense, and most implementations should use it, but there's no need to require it.
• To @alxocity's point, metadata needs to be extensible. However, I do think that having a minimum number of standard fields (namely, "Name", "Description", and "Image") that are required if the metadata exists has a huge amount of value. It makes many use cases (most notably, NFT browsers in wallets and open marketplaces like OpenSea) a million times easier to implement, and much more useful.
• Storage on Flow will be many times cheaper than Ethereum, but it's still designed to hold data to be processed directly by smart contracts, and not data intended solely for off-chain use. In other words, I think that the model of storing "heavy weight" metadata (like images or videos) in off-chain (anchored on-chain by a hash) is still correct within Flow. You'll note that we do include "light-weight" data in TopShots (player name, team name, etc.), because we expect that to provide interesting surface area for extensibility in future smart contract games.
• A future enhancement for Flow that we've mapped out, but haven't yet implemented, is the idea of "collapsing data storage into a hash". What this means is that, for some object in on-chain storage, you can ask the execution environment to purge the bytes of that object from storage, and only hold on the hash of the the contents of that object. (This object could be the metadata data for a single NFT, it could be an entire NFT, or it could be an entire collection of NFTs.) So, you could collapse a megabyte of data into a single 32-byte hash, and free up that storage allocation. It would be the responsibility of the owner of that data to arrange for off-chain storage (using IPFS or DropBox, as is their preference!), and any attempts to access that object would fail while in the collapsed state. Then, at some future date, the user could reintroduce that data, the chain could confirm the validity of the data via the hash, and the object would be "reconstituted" and fully accessible again. This scheme would work incredibly well for something like Avastars, where the smart contract itself constructs the metadata (an image in that case), but once it's been constructed, the data doesn't need to live on-chain indefinitely. Any client software could confirm that an off-chain representation of the image matches what the smart contract generated, without the network needing to maintain the long term storage of that data. Unfortunately, we can't depend on that functionality on Flow today.
• We've had conversations with the Protocol Labs team about what it might look like to include an Filecoin service on Flow that is mediated entirely by smart contracts. The basic idea would be that you could pass some on-chain data to the Filecoin smart contract (living on Flow), and it would arrange for storage inside Filecoin (using some kind of bounty system, presumably). Once the storage had been confirmed, you'd get a receipt on Flow that would let you request that data be restored to the chain later. It would only be useful for long-term storage, and there'd be significant latency when requesting both initial storage and refetches, but you could imagine this would be useful for smart contract-generated data that rarely needs to be touched on-chain. (Again, Avastars might be the easiest example for understanding this.) This idea is especially powerful when it can be coupled with the "collapse to hash" functionality described above, and can be extended to any other decentralized (or even centralized) storage solutions.

[dete]

Are smart contract mixins idiomatic on Flow?
As I pointed out above, standard interfaces (like the Non-Fungible Token interface) define the minimum functionality for compliant implementations, and don't limit additional functionality. In particular, additional standards or optional extensions to existing standards can be published on chain, and any implementation can choose to conform to as many of them as it wants, provided they don't conflict.

To use your example for royalties, you could define an extension of the NFT interface that includes royalty tracking, and (so long as it doesn't break the requirements of the base NFT interface), would be seen as a generic NFT to use cases that didn't have to worry about royalties, while adding in the royalty functionality where appropriate.

[ericelliott]

Are interfaces on Flow first-class and composable? For example, can we say a contract implements the NFT, Royalties, and timelock interface?

[dete]

Are interfaces on Flow first-class and composable? For example, can we say a contract implements the NFT, Royalties, and timelock interface?

You bet! Provided those interfaces don't have any conflicts.

[joshuahannan]

struct and resource interfaces are composable, but contract interfaces aren't, correct? @turbolent ?

[turbolent]

If you mean "composable" as in, a concrete struct/resource/contract can implement multiple interfaces, then yes, all of them can do that (if it's not working right now it is a bug), e.g.

contract interface NFT { /* ... */ }

contract interface Royalties { /* ... */  }

contract interface Timelock { /* ... */  }

contract Cool: NFT, Royalties, Timelock { /* ... */ }

[bjartek]

But it is not possible to import the code for fulfulling that interface contract from another account right? You will have to duplicate that yourself?

[dete]

That's right, @bjartek. We have some thoughts on how to implement code-reuse, but they are still only thoughts.

We definitely didn't want to use standard object inheritance, because it already has lots of weird edge cases in off-chain code, and we were pretty sure that it would be an absolute disaster in the context of smart contracts. Imagine if I could define a sub-class of CryptoKitty that overrode the breeding method! Or a subclass of Vault that "extended" the interface to include a "makeMeRich" method! Doesn't exactly fit the use case... 😀

[bjartek]

@dete exactly. That was part of the reasoning behind my mixin experiment. Each trait would store all its method and state in a seperate «namespace» inside the NFT.

Will this feature be done before main net launches? Or if not how should we structure our NFT to be compatible.

In my case I want to store Art that could be unique or editioned. Either as a single «trait» or ad two.

[bjartek]

I discussed this with @briandilley in dm a bit and with the ability to resolve a list of a type then my pervious need are covered. So using types and not strings will work

[rheaplex]

Types cannot currently be used as dictionary keys (that would be a good feature to add), but we can use Type().identifier as the key in a dictionary of {String: AnyStruct} and hide this with the API.

[bjartek]

Is it easy/possible to create a Type from fcl/go-sdk and send it as argument to a script/transacrion? Or to create a type from a string in cadence?

[rheaplex]

Is it easy/possible to create a Type from fcl/go-sdk and send it as argument to a script/transacrion? Or to create a type from a string in cadence?

It's tricky because you need the content of the type, I think. @turbolent I think I asked about this before and it wasn't practical?

[briandilley]

I updated my implementation of the above in the following gist.

You can find it in the playground here too

[joshuahannan]

@briandilley @bjartek Can one of you write up a summary of the current state of the metadata proposal? I think it is probably a little hard for any newcomers to understand what the state of the proposal is, and we'd probably like to start sharing it with some other teams soon, like OpenSea and such. It would be good to have a proposal with design rationale, trade-offs, examples, and blockers.

[dete]

FWIW- I agree that "View" is a reasonable name and will use it here. I humble suggest we consider "Atom", since that has an antecedent with metadata chunks in the MP4 file format.

One last attempt to make the case that name->Type is potentially problematic:

In practice, the code is going to be much easier to write if a particular name always maps to a particular type. It doesn't do anyone any good if there is a well defined tag called "Artist", but then there are 5 different structs that represent the artist data in different ways. Additionally, name collisions and/or squatting is bound to happen in practice (two llama projects launch at about the same time, both have metadata for "Llama Genes", but the two data types they map to are wildly different).

So, what's likely to happen in the real world is that you'll actually be looking for name/Type pairs. So, if you have to check the type anyway, why bother with the name part?

On the other hand, Types in Cadence are guaranteed to be distinct (no possibility for squatting or collisions), and code that examines the metadata views will always know the right type it can expect to be returned, because it's literally the type that it asked for. The interface can even include a postcondition that says that the returned type needs to match the type argument.

It's worth noting here that I'm optimizing for on-chain computation. On-chain code (or even Cadence scripts running on an Access Node) is always going to be more "expensive" to run (and update) than purely client-side code, so if there's ever a trade-off between making things easier on-chain or easier for client code, I'll always lean towards making things easier for the on-chain case.

In particular, it seems like some of motivation for tagging with names is imagining that, as time goes on, new View structs will replace old ones for the same tag. This sounds like a nightmare for on-chain code to me! 😁 I have a smart contract that requires that you supply a Giannis Moment in order to be granted access. If the struct that a Moment returns when I ask for the Player View changes, you've broken my whole thing. On the other hand, if a Moment starts to offer a new, "better" version of the Player type, newer contracts can use the new version, and I can keep getting the "backwards compatible" version without any code changes.

[dete]

I think there's a useful pattern that would be helpful when adding "newer and better" Atom/View structs. I think @briandilley was implying this pattern in a comment he made above, but I think it's useful to have an explicit (albeit contrived) example here:

    pub struct ArtistA {
        pub let name: String
        pub let birthYear: Int32
    }

    pub struct ArtistB {
        pub let name: String
        pub let birthYear: Int32
        pub let birthCountry: String

        pub fun asArtistA(): ArtistA {
            return ArtistA(name: self.name, birthYear: self.birthYear)
        }
    }

(Note, I'm using the names ArtistA and ArtistB here, but in practice this is more likely to be something like "anAddress.MusicMetadata.Artist" and "anotherAddress.MusicMetadataEnhanced.Artist".)

[bjartek]

Very good points in your above post @dete. I have not found time to update my reference impl to support the Type only stype but I really want to see if it works in practise because I can see that the string style can cause problems.

I also really like you optimization choice.

We had a discussion in discord the other day and I am more convinced then ever that the NFTv2 standard needs to support both immutable data and view on top of that data. One could even argue that you should not be allowed to change a view, but you should be allowed to add or even in some scenarios remove a view.

I also like your concrete example with the "newer and better" approach.

I will see if I find time to write the proposal that @joshuahannan suggests above in the next week. Down with the flu at the moment and have some other things that take up some time.

[bjartek]

I tried to hack together a small poc for the 'type-only' approach. https://github.com/bjartek/flow-nfmt/tree/types

It works, but I have to hard code the Type in the script to resolve the View since I do not know how to create that type from a string in the go-sdk. Anybody have any hints on that? So create a type from its Identifier is what I am looking for.

[bluesign]

I think one thing we are missing something important.

I want to separate for the sake of discussion, 2 parts of information, I will define them as:

• data: which is the part of NFT that is stored in the NFT resource.
• view/atom: which is the friendly/interoperable representation of the data or some derivative of the data

So technically if I have a Song NFT. I can have: ArtistId as data, then I can have an Artist view/atom as proposed by @dete as type ArtistA like:

    pub struct ArtistA {
        pub let name: String
        pub let birthYear: Int32
    }

From the discussion we had with @briandilley and @bjartek, I believe the main need from the developer side is some kind of upgrading of views. (which is pretty similar to providing ArtistB later on )

 pub struct ArtistB {
        pub let name: String
        pub let birthYear: Int32
        pub let birthCountry: String
}

So as we don't want to lose on-chain integration as @dete suggested. We should have both of them at the same time.

So when I update my NFT to provide more view/atom. It should return both ArtistA and ArtistB.

But I think there is something important to show that ArtistA and ArtistB are representing the same data. We shouldn't leave that to the presentation layer. I don't like let's say wallet has to know if there is ArtistB information, show ArtistB and not show ArtistA.

In my opinion tag makes a bit sense. But maybe mapping one tag to an array of Types are a good solution here.

[bluesign]

@dete about asArtistA part, what is your opinion about making it dynamic.

as we will have an interface like AtomProvider as follows:

pub fun availableAtoms() : [Type]
pub fun getAtom(_ type: Type): AnyStruct

pub struct ArtistB : AtomProvider {
    pub let name: String
    pub let birthYear: Int32
    pub let birthCountry: String
    pub fun availableAtoms() : [Type]{
        return [ArtistA]
     }
     pub fun getAtom(_ type: Type): AnyStruct{
         if type==ArtistA{
              return ArtistA(name: self.name, birthYear: self.birthYear)
         }
     }
}

If the presentation layer (let's say wallet) doesn't understand ArtistB it can ask for alternate representations and can go till it can recognize something.

[rheaplex]

So create a type from its Identifier is what I am looking for.

If we wish to fetch the instance of that type to work with it, we probably know its structure and can import its definition to access it better.

If we don't, maybe we should access it via an interface.

In either case, if we need to get a Type from a String we can construct a lookup table in a well-known location of {String: Type} and use that if we need to.

I'm worried that type construction from strings promises a dynamicness that Cadence doesn't have. But I've been wrong about this kind of thing before. 😺

[bluesign]

@bjartek Also we already have to get supported Type with getViews anyway, maybe we can use that result as a parameter to resolveView maybe

[bjartek]

@bluesign are you suggesting using the index of the getViews array as the parameter to resolve things? I do not see that as an improvement at all.

What if we expose the views as Type but resolve them using their type identifier? My example implementation from above already supports this.

[bluesign]

I meant a bit like this, if someone will use this, they will not just blindly call resolveView, they will first ask getViews to return supported views, then they will try to resolve views they are interested in.

Also as @rheaplex suggested, in order to use the struct probably, they will reference the contract containing that struct anyway.

I would love to be able to create the type from identifier, but In general, it is big work for a little gain. Requires importing another contract on runtime, parser, and checker running again, also seems like can be a potential security hole (at least for DoS).

[bjartek]

I meant a bit like this, if someone will use this, they will not just blindly call resolveView, they will first ask getViews to return supported views, then they will try to resolve views they are interested in.

How are you supposed to do this in any other way then using the index right now? There is AFAIK no way of creating a Type in any of the SDKs or clients.

Also as @rheaplex suggested, in order to use the struct probably, they will reference the contract containing that struct anyway.

Many people will want to do this yes. But imagine you have a block explorer. It an see your NFTs, it can see your views, it can resolve the views and peek as the data behind them as data that is exporeted as json. Personally I would love to see that, and it can be done I think.

I would love to be able to create the type from identifier, but In general, it is big work for a little gain. Requires importing another contract on runtime, parser, and checker running again, also seems like can be a potential security hole (at least for DoS).

Then I suggest we do not use Types to resolve the views, using index in views array is too brittle, atleast if you want to be able to store the "url" to a view somewhere as a stable identifier. What if somebody adds or removes some view? What if a contract want to store the view it used in a NFT as a field and fetch that view later. The 'thing' used to resolve a view must be able to support this IMHO.

The way I see it using the identifiser of the type will solve this.

[briandilley]

Having to import views from another contract is not a problem IMO. Especially since we'll likely have a set of views hosted by a known trustworthy source (ie: dapper or some standards group). That said, there is definitely something to be said for something a bit less statically typed - like JSON where you never have to import anything from anywhere.

[rheaplex]

Presented for feedback and comment based on Dete's comments above, a suggestion for supporting forward compatibility in NFT metadata without limiting experimentation and innovation:

Let us assume a metadata standard defining a resource interface that supports at least the following member functions:

pub contract NFTMetadata {
    pub interface resource NFTMetadata {
        // List the Types of metadata contained by the NFT
        pub fun metadataTypes(): [Type]
        // Get a particular Type of metadata, or nil if absent
        pub fun metadataOfType(type: Type): AnyStruct?
        // Get all available metadata (convenience function, Type keys are not yet supported in Cadence)
        pub fun metadataOfAllTypes(): {Type: AnyStruct}
    }
}

Let us assume that we do not wish this standard to specify the Types used beyond a basic core of metadata properties, for example the ERC-721 metadata fields plus a royalty spec.

To enable forward compatibility, we create type converter contracts that allow instances of types not known or supported by a given implementation of the MetaData interface to be created from that existing metadata.

Here is an example converter contract:

[Imports skipped]
...
pub contract ExampleMetadataTypeConverter {

    // This is a conversion function.
    // It takes an NFT, and requests the old Type of metadata that it understands.
    // If that Type is available, the function converts it to the new type and returns it.
    // Otherwise it returns nil.
    //
    pub fun toNewType(nft: &NonFungibleToken.NFT{NFTMetadata.NFTMetadata}): NewType? {
        if let oldType = nft.metadataOfType(type: OldType) {
            return NewType(x: oldType.h, y: - oldType.v, z: 0.0)
        }
        return nil
    }

    // This is a conversion function.
    // It takes an NFT, and requests the old Type of metadata that it understands.
    // If that Type is available, the function converts it to the new type and returns it.
    // Otherwise it tries to request two other metadata Types, in order of preference.
    // If one of those is available, the function converts it to the new type and returns it.
    // Otherwise it returns nil.
    //
    pub fun toOtherNewType(nft: &NonFungibleToken.NFT{NFTMetadata.NFTMetadata}): OtherNewType? {
        if let oldType = nft.metadataOfType(type: OtherOldTypeA) {
            return OtherNewType(r: oldType.r, g: oldType.g, b: oldType.b, a: 1.0)
        }
        if let oldTypeB = nft.metadataOfType(type: OtherOldTypeB) {
            return OtherNewType(r: 0.0, g: 0.0, b: 0.0, a: oldType.alpha)
        }
        if let oldTypeC = nft.metadataOfType(type: OtherOldTypeB) {
            return OtherNewType(r: oldTypeC.w / 7752.1, g: 7.0, b: 0,0, a: oldTypeC.alpha / 100.0)
        }
        return nil
    }

}

Note that we fetch the metadata directly from the NFT. This is intended to ensure that the data cannot be manipulated in between being fetched from the NFT and passed to the converter.

New types can be added until the contract is locked. This makes the code reliable without leaving an open attack vector.

[bjartek]

First i must mention that this discussion has progressed in the relevant flip fest repositories and we also have had a set of meetings with Glen about this.

Let us assume that we do not wish this standard to specify the Types used beyond a basic core of metadata properties, for example the ERC-721 metadata fields plus a royalty spec.

How are you going to enforce that? If those methods take a Type as a parameter? I do not see the value at this at all, just limitations.

[psiemens]

Hey everybody! @bjartek @bluesign @briandilley and @figs999 formalized the ideas from this thread into a Flow Improvement Proposal (FLIP) here: onflow/flow#636

It's on track to be the first community-written FLIP that is written, reviewed and implemented on Flow. To prevent communication fragmentation, we're encouraging everybody to move this discussion to that PR and any future extensions or amendments that result from it.

[psiemens]

I'm closing this issue because the core ideas in this thread were proposed and accepted in this FLIP: onflow/flow#636.

The new standard is flexible to support a wide variety of data formats. We'll be giving it a permanent home in this repository. We can have focused discussions about specific data formats (e.g. royalties) in dedicated issues or PRs on this repository.

Thanks again everybody!