docs: describe elements transaction serialization format

[jamesdorfman]

Note: When reviewing this markdown file, please open it in a markdown viewer. Otherwise, the tables will be illegible. Github will show you the rendered markdown if you open the "Files changed" tab, click the three dots on the top right of the file, and select "View file".

This PR adds a new markdown document which contains a detailed description of the elements transaction serialization format.

The document includes a detailed descriptions of all fields and examples where complex raw transactions are deserialized, step-by-step.

Intended audience:

• Hardware Wallet developers implementing liquid apps
• Developers creating Liquid integrations
• Developers writing their own serialized Liquid transactions from other languages
• Bitcoin users wanting to see what's new in an Elements transactions
• People creating and using PSETs (PSETs use serialized transactions)
• People wanting to know how Liquid and Elements work

[delta1]

Great work, I have just done a first pass with mostly grammatical suggestions, and will make a second pass (when I can) to go over the examples closely.

[delta1]

Suggested change

	This document describes the format used to serialize elements transactions. Once a transaction has been converted into this raw, serialized form, it can be broadcast across the network.
	This document requires familiarity with the architecture of bitcoin and elements, as well the various parts that are expected in an elements transaction (inputs, outputs, assets, pegin data, etc.). For more information on those, please refer to the [elements code tutorial](https://elementsproject.org/elements-code-tutorial/overview).
	This document describes the format used to serialize Elements transactions. Once a transaction has been converted into this raw, serialized form, it can be broadcast across the network.
	This document requires familiarity with the architecture of Bitcoin and Elements, as well the various parts that are expected in an Elements transaction (inputs, outputs, assets, peg-in data, etc.). For more information on those, please refer to the [Elements code tutorial](https://elementsproject.org/elements-code-tutorial/overview).

[jamesdorfman]

Thanks! I will add all these, except for the "peg-in" one.

It appears that the existing elements docs use "pegin", not "peg-in".

claimpegin "bitcoinTx" "txoutproof" ( "claim_script" )
Claim coins from the main chain by creating a pegin transaction with the necessary metadata after the corresponding Bitcoin transaction.

https://elementsproject.org/en/doc/0.18.1.6/rpc/wallet/claimpegin/

[delta1]

Suggested change

	*Note:* The encoding column in the following tables corresponds to the way that the fields are serialized. In bitcoin, numbers and hashes are serialized in little endian, and this same notion carries over to elements.
	*Note:* Values are listed in multiples of 100,000,000 (this corresponds directly to how bitcoin encodes value, where 1 BTC is divisible into 100,000,000 satoshis). For example, a `ConfidentialAmount` representing 7 inflation keys would be encoded as 700,000,000.
	*Note:* The encoding column in the following tables corresponds to the way that the fields are serialized. In Bitcoin, numbers and hashes are serialized in little-endian, and this same notion carries over to Elements.
	*Note:* Values are listed in multiples of 100,000,000 (this corresponds directly to how Bitcoin encodes value, where 1 BTC is divisible into 100,000,000 Satoshis). For example, a `ConfidentialAmount` representing 7 inflation keys would be encoded as 700,000,000.

[delta1]

This is a signed int32 - link

Might be worth adding some info from https://developer.bitcoin.org/reference/transactions.html#raw-transaction-format

Maybe "Data Type" should be the actual C++ type used?

Suggested change

	| Version | Yes | 4 bytes | `UnsignedInteger` | Little endian | Protocol version (should always be 2). |
	| Version | Yes | 4 bytes | `uint32_t` | Little-Endian | Transaction version number. Currently version 2. Version 2 means that [BIP 68](https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki#specification) applies. |

[delta1]

Suggested change

	| Flags | Yes | 1 byte | | | Whether the transaction contains a witness. Either 0 or 1. |
	| Flags | Yes | 1 byte | unsigned char | | Whether the transaction contains a witness. Either 0 (no witness) or 1. |

[delta1]

Suggested change

	| Witness | Only if flags is 1 | Varies | `Witness` | | See [BIP 141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki). Note that elements witnesses contain more data than bitcoin witnesses. This extra data is described further below. |
	| Witness | Only if flags is 1 | Varies | `Witness` | | See [BIP 141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki). Note that Elements witnesses contain more data than Bitcoin witnesses. This extra data is described further below. |

[delta1]

Suggested change

	| ScriptSig Length | Yes | Varies | `VarInt` | | Set to `0x00` if the transaction is segwit and the witness contains the signature. |
	| ScriptSig Length | Yes | Varies | `VarInt` | | Set to `0x00` if the transaction is SegWit and the witness contains the signature. |

[delta1]

Suggested change

	As noted above, the witness is only present for segwit transactions.
	As noted above, the witness is only present for SegWit transactions.

[delta1]

Suggested change

	The range proofs must be empty if their asociated amount (issuance / inflation keys) is explicit.
	The range proofs must be empty if their associated amounts (issuance / inflation keys) are explicit.

[delta1]

Suggested change

	The Surjectionproof must be empty if the output’s assetId is explicit.
	The Surjection Proof must be empty if the output’s assetId is explicit.

[delta1]

Suggested change

	Refer to the [bitcoin protocol documentation](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer) for a full description on how this type is serialized.
	Refer to the [Bitcoin protocol documentation](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer) for a full description on how this type is serialized.

[roconnor-blockstream]

Null asset and null amounts are illegal in outputs. They cannot be 1 byte.

[jamesdorfman]

Fixed.

[roconnor-blockstream]

This is only freeform for new issuances. For reissuances it is required to be the asset's entropy value.

[jamesdorfman]

Added.

[roconnor-blockstream]

Note: Inflation Keys cannot be reissued.

[jamesdorfman]

I've added a note here explaining this.

[roconnor-blockstream]

Input is a pegin; Input has an asset issuance.

[jamesdorfman]

Fixed.

[philippem]

Do we need to have the serialized form listed here? The table is sequential and conveys the form. Duplicating it makes it redundant and the reader may wonder if it differs .

[philippem]

Can probably omit the NB. We can assume familiarity with little-endian and big-endian serializations.

[roconnor-blockstream]

FWIW, I think the situation here is more complex than this documentation presents. The serialization order of the hashes is the correct order of the bytes produced by SHA256. It is the elements client that takes this correct serialization order, reinterprets it as a 256 bit integer in little endian format and then displays that integer in hex, which has the effect of displaying AssetIDs backwards.

I just don't know a concise way of explaining all this.

[jamesdorfman]

I see. Thank you for the clarification.

I was actually under the (incorrect) impression that the elements client displayed the correct order of bytes produced by SHA256, and the hashes were just serialized in reverse.

Your explanation makes much more sense.

I will make this distinction more clear.

[apoelstra]

I think we should drop any reference to endianness and just say that the bytes are reversed. You can add a footnote explaining that historically this has something to do with endianness and type conversions but I don't think it would make things any clearer.

[jamesdorfman]

Done. I've stated that the bytes are reversed, and I've added a footnote explaining the historical reasons for this.

[philippem]

We should state if other values are accepted or ignored. (In this case, they are rejected)

[philippem]

Might be helpful to add a one sentence description of this field, in case someone is arriving here without the previous context.

[philippem]

the use of camel case here is inconsistent with input 1

[philippem]

what happens if both bits are 1?

[jamesdorfman]

I think it's self-evident that an asset issuance and a pegin are mutually exclusive, from a logical perspective.

I'd prefer not to enumerate every single thing that could be invalid, because that would make the document much longer and more confusing to read.

I could add a link to the actual transaction validation code here, but I'm not sure how helpful that will be.

[roconnor-blockstream]

As far as I am aware you can pegin and issue an asset at the same time. I've never tried it though.

[philippem]

might be helpful to keep with the table description for these types, for consistency and clarity (even though there are only 2 fields).

[roconnor-blockstream]

I would drop the "Big-endian" for scripts and leave the field blank.

[roconnor-blockstream]

Note: Amount and Num Inflation Keys cannot be explicitly 0. (Instead they should be NULL values).

[roconnor-blockstream]

@apoelstra correct me if I'm wrong, but how the element/green UI handles units is ... more complicated than this suggests. There is something about the asset registry etc. etc.

I think we should simply note that all unit values are in satoshis, or say all values are in the smallest indivisible unit (i.e. satoshis) or something like that.

[apoelstra]

Agreed.

Even if other assets didn't have "natural units" that differed from 10^8 I don't know what 10^8 would have to do with the encoding, so we should drop any mention of it.

[jamesdorfman]

The reason I mentioned this was because Example 2 below contains an asset issuance with the number of inflation keys encoded as:

0x29b92700 = 700 000 000 

Does this correspond to 7 keys? A single inflation key is indivisible, correct?

[apoelstra]

I think that's 700 million reissuance keys.

[jamesdorfman]

I see. So when I parse the full issuance transaction with $elements-cli decoderawtransaction <...>, the result contains:

"issuance": {
        "assetBlindingNonce": "0000000000000000000000000000000000000000000000000000000000000000",
        "assetEntropy": "503f1db03b8ed61ace8fbadc2fcb6c200bdf1ada403d74b891e55e1a047b6596",
        "isreissuance": false,
        "token": "6e88fa86a097b9ede564f67c60e8c5d24ed1f10eeb56c3a60223961f2e38737f",
        "asset": "d42c5ba3724b7f7a045968e0c04e69f81070aca8b6e2db88cd84c65dd07b8324",
        "assetamount": 33.00000000,
        "tokenamount": 7.00000000
}

Is the "tokenamount": 7.00000000 indicating that there are really 700,000,000 reissuance tokens?

[apoelstra]

lol. yes.

[jamesdorfman]

Ok that makes sense now. Thanks!

[apoelstra]

I opened #1152 about this.

[roconnor-blockstream]

@apoelstra Is there any existing wallet interpretation of an explicit nonce? Care to comment on its endianness?

[apoelstra]

No, and lol. It's hard to imagine an interpretation of "endianness" for nonces.

[roconnor-blockstream]

If you would take the explicit value as an secp256k1 scalar value and multiply the generator with it, that would give it an endianness. But I wasn't sure if a wallet is expected to do anything with an explicit nonce.

[apoelstra]

Yeah, that would make sense. But no, in practice nobody interprets "explicit nonces" at all.

[roconnor-blockstream]

I suggest commenting that the previous TXID is dfb6... here and for other examples.

[roconnor-blockstream]

I suggest commenting that the assetID is 6f02... here and in the other examples.

[roconnor-blockstream]

@apoelstra Should be be describing the pegin structure here, amount, claim_script, genesis hash, peg-in transaction? I know that the transaction tool does this.

[apoelstra]

Yeah, I think that we should. I'm tempted to say "it's a stack of byte vectors" but in this case every field has a clear meaning and fixed format, and it's simple enough to parse.

[roconnor-blockstream]

Approved, though I think we might as well inline the definition of the encoding for VarInts.

[philippem]

ack thank you!!

[philippem]

great job, thank you!