NFT Metadata Standard

[chadoh]

This discussion split from the more general discussion in #171. While that was intended to tackle the entire upcoming NFT standard, this discussion aims to only clarify the metadata aspect. Below seems to be the current best-guess at what a v1.0.0 standard might look like, based on current discussion in #171:

Summary

An interface for a non-fungible token's metadata. The goal is to keep the metadata future-proof as well as lightweight. This will be important to dApps needing additional information about an NFT's properties, and broadly compatible with other token standards such that the NEAR Rainbow Bridge can move tokens between chains.

Motivation

The primary value of non-fungible tokens comes from their metadata. While the core standard provides the minimum interface that can be considered a non-fungible token, most artists, developers, and dApps will want to associate more data with each NFT, and will want a predictable way to interact with any NFT's metadata.

NEAR's unique storage staking approach makes it feasible to store more data on-chain than other blockchains. This standard leverages this strength for common metadata attributes, and provides a standard way to link to additional offchain data to support rapid community experimentation.

This standard also provides a spec version. This makes it easy for consumers of NFTs, such as marketplaces, to know if they support all the features of a given token.

Prior art:

• NEAR's Fungible Token Metadata Standard
• Discussion about NEAR's complete NFT standard: Improved NFT Standard #171

Interface

Metadata applies at both the contract level (ContractMetadata) and the token level (TokenMetadata). The relevant metadata for each:

type ContractMetadata = {
  spec: string, // required, essentially a version like "nft-1.0.0"
  name: string, // required, ex. "Mochi Rising — Digital Edition" or "Metaverse 3"
  symbol: string, // required, ex. "MOCHI"
  icon: string|null, // Data URL
  base_uri: string|null, // Centralized gateway known to have reliable access to decentralized storage assets referenced by `reference` or `media` URLs
  reference: string|null, // URL to a JSON file with more info
  reference_hash: string|null, // Base64-encoded sha256 hash of JSON from reference field. Required if `reference` is included.
}

type TokenMetadata = {
  title: string|null, // ex. "Arch Nemesis: Mail Carrier" or "Parcel #5055"
  description: string|null, // free-form description
  media: string|null, // URL to associated media, preferably to decentralized, content-addressed storage
  media_hash: string|null, // Base64-encoded sha256 hash of content referenced by the `media` field. Required if `media` is included.
  copies: number|null, // number of copies of this set of metadata in existence when token was minted.
  issued_at: string|null, // ISO 8601 datetime when token was issued or minted
  expires_at: string|null, // ISO 8601 datetime when token expires
  starts_at: string|null, // ISO 8601 datetime when token starts being valid
  updated_at: string|null, // ISO 8601 datetime when token was last updated
  extra: string|null, // anything extra the NFT wants to store on-chain. Can be stringified JSON.
  reference: string|null, // URL to an off-chain JSON file with more info.
  reference_hash: string|null // Base64-encoded sha256 hash of JSON from reference field. Required if `reference` is included.
}

Then a new function is needed on the NFT contract:

function nft_metadata(): ContractMetadata {}

And a new attribute must be added to each Token struct returned:

 type Token = {
   id: string,
   owner_id: string,
+  metadata: TokenMetadata,
 }

An implementing contract MUST include the following fields on-chain

• spec: a string. Should be ft-1.0.0 to indicate that a Fungible Token contract adheres to the current versions of this Metadata and the Fungible Token Core specs. This will allow consumers of the Fungible Token to know if they support the features of a given contract.
• name: the human-readable name of the set of tokens.
• symbol: the abbreviation, like MOCHI or MV3
• base_uri: Centralized gateway known to have reliable access to decentralized storage assets referenced by reference or media URLs. Can be used by other frontends for initial retrieval of assets, even if these frontends then replicate the data to their own decentralized nodes, which they are encouraged to do.

An implementing contract MAY include the following fields on-chain

For ContractMetadata:

• icon: a small image associated with this token. Encouraged to be a data URL, to help consumers display it quickly while protecting user data. Recommendation: use optimized SVG, which can result in high-resolution images with only 100s of bytes of storage cost. (Note that these storage costs are incurred to the token owner/deployer, but that querying these icons is a very cheap & cacheable read operation for all consumers of the contract and the RPC nodes that serve the data.) Recommendation: create icons that will work well with both light-mode and dark-mode websites by either using middle-tone color schemes, or by embedding media queries in the SVG.
• reference: a link to a valid JSON file containing various keys offering supplementary details on the token. Example: "/ipfs/QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm", "https://example.com/token.json", etc. If the information given in this document conflicts with the on-chain attributes, the values in reference shall be considered the source of truth.
• reference_hash: the base64-encoded sha256 hash of the JSON file contained in the reference field. This is to guard against off-chain tampering.

For TokenMetadata:

• name: The name of this specific token.
• description: A longer description of the token.
• media: URL to associated media. Preferably to decentralized, content-addressed storage.
• media_hash: the base64-encoded sha256 hash of content referenced by the media field. This is to guard against off-chain tampering.
• copies: The number of tokens with this set of metadata or media known to exist at time of minting.
• issued_at: ISO 8601 datetime when token was issued or minted
• expires_at: ISO 8601 datetime when token expires
• starts_at: ISO 8601 datetime when token starts being valid
• updated_at: ISO 8601 datetime when token was last updated
• extra: anything extra the NFT wants to store on-chain. Can be stringified JSON.
• reference: URL to an off-chain JSON file with more info.
• reference_hash: Base64-encoded sha256 hash of JSON from reference field. Required if reference is included.

No incurred cost for core NFT behavior

Contracts should be implemented in a way to avoid extra gas fees for serialization & deserialization of metadata for calls to nft_* methods other than nft_metadata or nft_token. See near-contract-standards implementation using LazyOption as a reference example.

Drawbacks

• When this NFT contract is created and initialized, the storage use per-token will be higher than an NFT Core version. Frontends can account for this by adding extra deposit when minting. This could be done by padding with a reasonable amount, or by the frontend using the RPC call detailed here that gets genesis configuration and actually determine precisely how much deposit is needed.
• Convention of icon being a data URL rather than a link to an HTTP endpoint that could contain privacy-violating code cannot be done on deploy or update of contract metadata, and must be done on the consumer/app side when displaying token data.
• If on-chain icon uses a data URL or is not set but the document given by reference contains a privacy-violating icon URL, consumers & apps of this data should not naïvely display the reference version, but should prefer the safe version. This is technically a violation of the "reference setting wins" policy described above.

Future possibilities

• Detailed conventions that may be enforced for versions.
• A fleshed out schema for what the reference object should contain.

[chadoh]

Do we like using ISO 8601 strings for dates, or would we prefer timestamps? Blocktimes? If timestamps, in seconds, milliseconds, or something else?

[mikedotexe]

On yesterday's call we decided on epoch in milliseconds. I've updated it

[chadoh]

Do we care to include issued_at and updated_at? These attributes can be resolved from block times using an off-chain indexing layer.

[mikedotexe]

I'm not a huge advocate of the various times, but I believe the generally consensus with the community is that they'd be useful.

[think-in-universe]

Is updated_at used in practice today? I think most of the token metadata are not changed after minted, but there're some cases like generative arts or games that the metadata needs to be updated.

[zahhar]

How about renaming type NFTMetadata into ContractMetadata?
NFT stands for 'Non Fungible Token', and we also have TokenMetadata that are quite similar in nature.

[chadoh]

I like that

[chadoh]

Updated

[mikedotexe]

I know there are discussions about having a generic metadata for any smart contract exposing methods and whatnot. I think I'd like to make this a little more specific, as in the future ContractMetadata will likely be confused with that standard. I think NFTContractMetadata will work better here and have slightly better SEO and a targeted meaning.

[thor314]

Sample Rust implementation:

#[derive(BorshDeserialize, BorshSerialize, Deserialize, Serialize)]
#[serde(crate = "near_sdk::serde")]
pub struct NFTContractMetadata {
  /// a version like "nft-1.0.0"
  pub spec: String,
  /// Subaccount of the contract
  pub name: String,
  /// Symbol of the Contract. 
  pub symbol: String,
  /// a small image associated with this Contract.
  pub icon: Option<String>,
  /// Centralized gateway known to have reliable access to decentralized storage
  /// assets referenced by `reference` or `media` URLs
  pub base_uri: Option<String>,
  /// URL to a JSON file with more info
  pub reference: Option<String>,
  /// Base64-encoded sha256 hash of the JSON file pointed at by the reference
  /// field. Required if `reference` is included.
  pub reference_hash: Option<Base64VecU8>,
}


impl NFTContractMetadata {
  pub fn from(args: NFTContractMetadataArgs) -> Self {
    match args.reference {
      Some(_) => assert!(args.reference_hash.is_some()),
      None => assert!(args.reference_hash.is_none()),
    }
    let store_account = format!("{}.{}", args.name, env::current_account_id());
    assert!(
      env::is_valid_account_id(store_account.as_bytes()),
      "Invalid character in store id"
    );
    assert!(args.symbol.len() <= 6);

    Self {
      spec: args.spec,
      name: args.name,
      symbol: args.symbol,
      icon: args.icon,
      base_uri: args.base_uri,
      reference: args.reference,
      reference_hash: args.reference_hash,
    }
  }
}

/// Unvalidated args
#[derive(Serialize)]
#[serde(crate = "near_sdk::serde")]
pub struct NFTContractMetadataArgs {
  pub spec: String,
  pub name: String,
  pub symbol: String,
  pub icon: Option<String>,
  pub base_uri: Option<String>,
  pub reference: Option<String>,
  pub reference_hash: Option<Base64VecU8>,
  pub owner_id: AccountId,
}

/// ref: https://github.com/near/NEPs/blob/master/specs/Standards/NonFungibleToken/Metadata.md
pub trait NonFungibleContractMetadata {
  /// Get the metadata for this Contract.
  fn nft_metadata(&self) -> &NFTContractMetadata;
}

#[near_bindgen]
impl NonFungibleContractMetadata for MintbaseStore {
  fn nft_metadata(&self) -> &NFTContractMetadata {
    &self.metadata
  }
}

#[derive(BorshDeserialize, BorshSerialize, Serialize, Deserialize, Clone)]
#[serde(crate = "near_sdk::serde")]
pub struct TokenMetadata {
  /// the Title for this token. ex. "Arch Nemesis: Mail Carrier" or "Parcel #5055"
  pub title: Option<String>,
  /// free-form description of this token.
  pub description: Option<String>,
  /// URL to associated media, preferably to decentralized, content-addressed storage
  pub media: Option<String>,
  /// Base64-encoded sha256 hash of content referenced by the `media` field.
  /// Required if `media` is included.
  pub media_hash: Option<Base64VecU8>,
  /// number of copies of this set of metadata in existence when token was minted.
  pub copies: Option<U64>,
  /// ISO 8601 datetime when token was issued or minted.
  pub issued_at: Option<String>,
  /// ISO 8601 datetime when token expires.
  pub expires_at: Option<String>,
  /// ISO 8601 datetime when token starts being valid.
  pub starts_at: Option<String>,
  /// ISO 8601 datetime when token was last updated (transferred, ).
  pub updated_at: Option<String>,
  /// anything extra the NFT wants to store on-chain. Can be stringified JSON.
  pub extra: Option<String>,
  /// URL to an off-chain JSON file with more info.
  pub reference: Option<String>,
  /// Base64-encoded sha256 hash of JSON from reference field. Required if
  /// `reference` is included.
  pub reference_hash: Option<Base64VecU8>,
  ///
  /// *Mintbase-specific metadata*
  ///
  /// The account that minted this token.
  pub minter: AccountId,
  /// Some information about the token type, eg. "art" or "ticket". Up to 15
  /// chars in length.
  pub memo: Option<String>,
}

impl TokenMetadata {
  /// Validate
  pub fn from(args: TokenMetadataArgs) -> Self {
    Self {
      title: args.title,
      description: args.description,
      media: args.media.clone(),
      media_hash: args
        .media
        .map(|media| Base64VecU8::from(env::sha256(media.as_bytes()))),
      copies: args.copies,
      issued_at: Some(env::block_timestamp().to_string()),
      expires_at: args.expires_at,
      starts_at: args.starts_at,
      updated_at: Some(env::block_timestamp().to_string()),
      extra: args.extra,
      reference: args.reference.clone(),
      reference_hash: args
        .reference
        .map(|reference| Base64VecU8::from(env::sha256(reference.as_bytes()))),
      minter: args.minter,
      memo: args.memo,
    }
  }
}

/// The unvalidated arguments for a token
#[derive(Serialize)]
#[serde(crate = "near_sdk::serde")]
pub struct TokenMetadataArgs {
  /// the Title for this token. ex. "Arch Nemesis: Mail Carrier" or "Parcel #5055"
  title: Option<String>,
  /// free-form description of this token.
  description: Option<String>,
  /// URL to associated media, preferably to decentralized, content-addressed storage
  media: Option<String>,
  /// number of copies of this set of metadata in existence when token was minted.
  copies: Option<U64>,
  /// ISO 8601 datetime when token expires.
  expires_at: Option<String>,
  /// ISO 8601 datetime when token starts being valid.
  starts_at: Option<String>,
  /// ISO 8601 datetime when token was last updated (transferred).
  updated_at: Option<String>,
  /// anything extra the NFT wants to store on-chain. Can be stringified JSON.
  extra: Option<String>,
  /// URL to an off-chain JSON file with more info.
  reference: Option<String>,
  ///
  /// *Mintbase-specific metadata*
  ///
  /// The account that minted this token.
  minter: AccountId,
  /// Some information about the token type, eg. "art" or "ticket". Up to 15
  /// chars in length.
  memo: Option<String>,
}

[zahhar]

Colleagues, shall we give a hint to potential adopters of the standard how to use reference, media and _hash attributes?

My current understanding, but please correct me if I am wrong:

• reference and media MAY store EITHER full absolute URL (in this case consumer ignores base_uri value or it can be left unset), OR part of the URL relative to the base_uri defined in the NFTContractMetadata. However, contract must be consistent here and use same approach for both reference and media attributes, as well as for all NFTs minted. Correct?
• _hash attributes propose to support only SHA-256 algorithm. But what if I want SHA-384 or even SHA-512? Potentially consumer MAY calculate the length and determine algorithm used. Or RTFM for the contract. However, probably would be better either extend NFTContractMetadata with one more attribute named hash_algorithm storing the value like sha-512 that is used for all hashed, OR store the used algorithm as a prefix for the actual value, e.g. sha-384; 3519fe5ad2c596efe3e276a6f351b8fc0b03db861782490d45f7598ebd0ab5fd5520ed102f38c4a5ec834e98668035fc — I am not sure if semicolon is good enough delimiter, or there is any standard that defines it?

[chadoh]

• Your thoughts on reference and media sound correct to me. We could further clarify that base_uri is meant to be used with decentralized storage, so that reference/media might look like /ipfs/Qmblahblahblah, while base_uri points to a known, reliable IPFS gateway.
• For *_hash attributes, I agree it seems unoptimized that we're locking ourselves into sha256. I would suggest we use multihash, the hashing algorithm created & used by IPFS, which creates self-describing hashes in one attribute. This way we don't need a separate specifier like hash_algorithm.

[dievardump]

Hello, new here. Very interested in what you're doing here for NFTs.

I would like to emit reserves about _hash attributes.

This works very well when we are working with elements that are not evolving too much over time. However for certain type of items, like game items, it can happen that the metadata change several times per minutes, or that some metadata that are not essential to the NFT but still nice to have as Metadata are added to the Metadata file describing it.

Or it can happen that some updates from a version to another of the game could force platforms to update Metadata to hundreds of thousands of items at once, which could not possibly be expected to be replicated at once on the blockchain.

That's why having to force the _hash on-chain in those conditions might be a problem and something very hard to follow, if not impossible, for platforms emitting and controlling those items.

I think keeping a hash somewhere is a good idea, but this might be more feasible if it was added to the external file itself as a property containing something like "signature": signer.sign(hash(json))

For this instead of having the hash in the contract, the contract would reference a signer address

When a platform updates those external Metadata, if it doesn't change URL, instead of having to do a tx they can simply:

• update file metadata
• signature = signer.sign(hash(metadata))
• add signature under a reserved property name, in the metadata file

Anyone can later verify the content:

• remove (and keep reference to) the signature property
• do a recover on the content without the property, and verifies it is signed by the designated signer in the contract

instead of having just signature it could be a rich object metadata_verfication that contains the signature, the algorithm used and any other information needed to recover the signer from the signature and verify the content was not tempered

[zahhar]

Hi @dievardump,
Thank you for your interest and feedback.

Your concerns are valid. Let me explain how NEAR NFT standard attempts to address them.

_hash attributes are totally optional, and you can just leave them unused, and then implement any other verification or validation approach off-chain. I'd suggest to take a look at extra attribute that is designed to accommodate any custom data you might need for it, e.g. you may store there your metadata_verrification object or whatever else is required.

Just a sidenote for others who might read this thread later: please do not confuse blockchain with traditional database. Do not attempt to store non-essential information in blockchain, or update data frequently. Blockchain is designed to store only data that requires cryptographic evidence, like proof of ownership, or proof that some event did happened at certain time. For everything else use off-chain document DB or relational DB, and refer to it from your blockchain data.

[dievardump]

Thank you for the feedback.

However the doc precise for TokenMetadata reference_hash: ... Required if reference is included
Maybe it is a mistake then?

Because this would mean that every-time the content of the file pointed by reference changes, the hash needs to be updated on the blockchain.

If I'm editing a fast pace game, and the items updates several times per minute (like strength, magic, ), it would force me to update the on-chain reference_hash very often. (and those are content that need to be in the JSON, else marketplaces like OpenSea wouldn't know about it)

Worst, if I were to add one new property to my game items after an update, I would need to updates all references. Which could count on hundred of thousands of items updated at once.

This is what I meant.

I it's optional like you say, I think it's pretty good, but here the doc marks it as required

[zahhar]

I think there is a mistake in documentation. We can only advise to have a _hash, but can't enforce it for various reasons. For example, when you store your media or reference in distributed storage like Arweare or IPFS, your URL includes a hash, and does not require this hash to be duplicated as a separate attribute.

@mikedotexe , @thor314 , @chadoh - guys, who/who could update the docs and mark *_hash optional?

[think-in-universe]

Background

As requested by some wallet and NFT users, they want to make the NFT images shown in the Wallet Collectibles being linked to the original page where the NFT is purchased or received.

For example, if anyone click on the NFT image which is purchased on Mintbase, the original link of the NFT should be opened, such as: https://www.mintbase.io/zh/thing/Zt0z2ZKZ7FA3FqrIpDdFnwy6Cx-oIs9_pHv53n7RSik:afrofuturemafia.mintbase1.near

See more details about this enhancement to wallet here: near/near-wallet#1896

reference object

One solution is to define such link in the JSON data in reference object. Should we start elaborating which info to be contained in the reference object?

[chadoh]

My understanding of reference is that it is a testing ground for rapid community experimentation. Same with extra, with reference being off-chain data and extra being on-chain. This seems like a perfect scenario to use one of them.

If we find that a majority of NFT apps all find a particular field useful, we could add it to the core metadata spec in a future version.