diff --git a/.aspell.en.pws b/.aspell.en.pws index e6ac81a3b..d50aaec44 100644 --- a/.aspell.en.pws +++ b/.aspell.en.pws @@ -355,7 +355,6 @@ verifier verifiers EOF monotonicity -varint optimizations structs CompactSize @@ -367,8 +366,6 @@ namespaces tlvs fips rfc -varint -CompactSize multipath mpp tlvs diff --git a/01-messaging.md b/01-messaging.md index 305d67fec..6c3c36acc 100644 --- a/01-messaging.md +++ b/01-messaging.md @@ -110,20 +110,15 @@ the backwards-compatible addition of new fields to existing message types. A `tlv_record` represents a single field, encoded in the form: -* [`varint`: `type`] -* [`varint`: `length`] +* [`bigsize`: `type`] +* [`bigsize`: `length`] * [`length`: `value`] -A `varint` is a variable-length, unsigned integer encoding using the -[BigSize](#appendix-a-bigsize-test-vectors) format, which resembles the bitcoin -CompactSize encoding but uses big-endian for multi-byte values rather than -little-endian. - A `tlv_stream` is a series of (possibly zero) `tlv_record`s, represented as the concatenation of the encoded `tlv_record`s. When used to extend existing messages, a `tlv_stream` is typically placed after all currently defined fields. -The `type` is a varint encoded using the BigSize format. It functions as a +The `type` is encoded using the BigSize format. It functions as a message-specific, 64-bit identifier for the `tlv_record` determining how the contents of `value` should be decoded. `type` identifiers below 2^16 are reserved for use in this specification. `type` identifiers greater than or equal @@ -131,7 +126,7 @@ to 2^16 are available for custom records. Any record not defined in this specification is considered a custom record. This includes experimental and application-specific messages. -The `length` is a varint encoded using the BigSize format signaling the size of +The `length` is encoded using the BigSize format signaling the size of `value` in bytes. The `value` depends entirely on the `type`, and should be encoded or decoded @@ -192,7 +187,7 @@ things, enables the following optimizations: - variable-size fields can reserve their expected size up front, rather than appending elements sequentially and incurring double-and-copy overhead. -The use of a varint for `type` and `length` permits a space savings for small +The use of a bigsize for `type` and `length` permits a space savings for small `type`s or short `value`s. This potentially leaves more space for application data over the wire or in an onion payload. @@ -236,6 +231,7 @@ The following convenience types are also defined: * `signature`: a 64-byte bitcoin Elliptic Curve signature * `point`: a 33-byte Elliptic Curve point (compressed encoding as per [SEC 1 standard](http://www.secg.org/sec1-v2.pdf#subsubsection.2.3.3)) * `short_channel_id`: an 8 byte value identifying a channel (see [BOLT #7](07-routing-gossip.md#definition-of-short-channel-id)) +* `bigsize`: a variable-length, unsigned integer similar to Bitcoin's CompactSize encoding, but big-endian. Described in [BigSize](#appendix-a-bigsize-test-vectors). ## Setup Messages @@ -474,10 +470,10 @@ decoded with BigSize should be checked to ensure they are minimally encoded. The following is an example of how to execute the BigSize decoding tests. ```golang -func testReadVarInt(t *testing.T, test varIntTest) { +func testReadBigSize(t *testing.T, test bigSizeTest) { var buf [8]byte r := bytes.NewReader(test.Bytes) - val, err := tlv.ReadVarInt(r, &buf) + val, err := tlv.ReadBigSize(r, &buf) if err != nil && err.Error() != test.ExpErr { t.Fatalf("expected decoding error: %v, got: %v", test.ExpErr, err) @@ -541,19 +537,19 @@ A correct implementation should pass against these test vectors: "name": "two byte not canonical", "value": 0, "bytes": "fd00fc", - "exp_error": "decoded varint is not canonical" + "exp_error": "decoded bigsize is not canonical" }, { "name": "four byte not canonical", "value": 0, "bytes": "fe0000ffff", - "exp_error": "decoded varint is not canonical" + "exp_error": "decoded bigsize is not canonical" }, { "name": "eight byte not canonical", "value": 0, "bytes": "ff00000000ffffffff", - "exp_error": "decoded varint is not canonical" + "exp_error": "decoded bigsize is not canonical" }, { "name": "two byte short read", @@ -604,14 +600,14 @@ A correct implementation should pass against these test vectors: The following is an example of how to execute the BigSize encoding tests. ```golang -func testWriteVarInt(t *testing.T, test varIntTest) { +func testWriteBigSize(t *testing.T, test bigSizeTest) { var ( w bytes.Buffer buf [8]byte ) - err := tlv.WriteVarInt(&w, test.Value, &buf) + err := tlv.WriteBigSize(&w, test.Value, &buf) if err != nil { - t.Fatalf("unable to encode %d as varint: %v", + t.Fatalf("unable to encode %d as bigsize: %v", test.Value, err) } diff --git a/04-onion-routing.md b/04-onion-routing.md index 801c30c1f..24d6ccda0 100644 --- a/04-onion-routing.md +++ b/04-onion-routing.md @@ -95,7 +95,7 @@ There are a number of conventions adhered to throughout this document: - Each hop in the route has a variable length `hop_payload`, or a fixed-size legacy `hop_data` payload. - The legacy `hop_data` is identified by a single `0x00`-byte prefix - - The variable length `hop_payload` is prefixed with a `varint` encoding + - The variable length `hop_payload` is prefixed with a `bigsize` encoding the length in bytes, excluding the prefix and the trailing HMAC. # Key Generation @@ -163,7 +163,7 @@ It is 1300 bytes long and has the following structure: 1. type: `hop_payloads` 2. data: - * [`varint`:`length`] + * [`bigsize`:`length`] * [`hop_payload_length`:`hop_payload`] * [`32*byte`:`hmac`] * ... @@ -523,10 +523,10 @@ For each hop in the route, in reverse order, the sender applies the following operations: - The _rho_-key and _mu_-key are generated using the hop's shared secret. - - `shift_size` is defined as the length of the `hop_payload` plus the varint encoding of the length and the length of that HMAC. Thus if the payload length is `l` then the `shift_size` is `1 + l + 32` for `l < 253`, otherwise `3 + l + 32` due to the varint encoding of `l`. + - `shift_size` is defined as the length of the `hop_payload` plus the bigsize encoding of the length and the length of that HMAC. Thus if the payload length is `l` then the `shift_size` is `1 + l + 32` for `l < 253`, otherwise `3 + l + 32` due to the bigsize encoding of `l`. - The `hop_payload` field is right-shifted by `shift_size` bytes, discarding the last `shift_size` bytes that exceed its 1300-byte size. - - The varint-serialized length, serialized `hop_payload` and `hmac` are copied into the following `shift_size` bytes. + - The bigsize-serialized length, serialized `hop_payload` and `hmac` are copied into the following `shift_size` bytes. - The _rho_-key is used to generate 1300 bytes of pseudo-random byte stream which is then applied, with `XOR`, to the `hop_payloads` field. - If this is the last hop, i.e. the first iteration, then the tail of the @@ -662,7 +662,7 @@ The routing information is then deobfuscated, and the information about the next hop is extracted. To do so, the processing node copies the `hop_payloads` field, appends 1300 `0x00`-bytes, generates `2*1300` pseudo-random bytes (using the _rho_-key), and applies the result, using `XOR`, to the copy of the `hop_payloads`. -The first few bytes correspond to the varint-encoded length `l` of the `hop_payload`, followed by `l` bytes of the resulting routing information become the `hop_payload`, and the 32 byte HMAC. +The first few bytes correspond to the bigsize-encoded length `l` of the `hop_payload`, followed by `l` bytes of the resulting routing information become the `hop_payload`, and the 32 byte HMAC. The next 1300 bytes are the `hop_payloads` for the outgoing packet. A special `hmac` value of 32 `0x00`-bytes indicates that the currently processing hop is the intended recipient and that the packet should not be forwarded. @@ -1000,7 +1000,7 @@ The CLTV expiry in the HTLC is too far in the future. 1. type: PERM|22 (`invalid_onion_payload`) 2. data: - * [`varint`:`type`] + * [`bigsize`:`type`] * [`u16`:`offset`] The decrypted onion per-hop payload was not understood by the processing node diff --git a/07-routing-gossip.md b/07-routing-gossip.md index 4808e1d09..a89f80ca7 100644 --- a/07-routing-gossip.md +++ b/07-routing-gossip.md @@ -602,10 +602,10 @@ Nodes can signal that they support extended gossip queries with the `gossip_quer 2. types: 1. type: 1 (`query_flags`) 2. data: - * [`u8`:`encoding_type`] + * [`byte`:`encoding_type`] * [`...*byte`:`encoded_query_flags`] -`encoded_query_flags` is an array of bitfields, one varint per bitfield, one bitfield for each `short_channel_id`. Bits have the following meaning: +`encoded_query_flags` is an array of bitfields, one bigsize per bitfield, one bitfield for each `short_channel_id`. Bits have the following meaning: | Bit Position | Meaning | | ------------- | ---------------------------------------- | @@ -642,7 +642,7 @@ The sender: - SHOULD NOT send this if the channel referred to is not an unspent output. - MAY include an optional `query_flags`. If so: - MUST set `encoding_type`, as for `encoded_short_ids`. - - Each query flag is a minimally-encoded varint. + - Each query flag is a minimally-encoded bigsize. - MUST encode one query flag per `short_channel_id`. The receiver: @@ -663,7 +663,7 @@ The receiver: - MUST follow with any `node_announcement`s for each `channel_announcement` - otherwise: - We define `query_flag` for the Nth `short_channel_id` in - `encoded_short_ids` to be the Nth varint of the decoded + `encoded_short_ids` to be the Nth bigsize of the decoded `encoded_query_flags`. - if bit 0 of `query_flag` is set: - MUST reply with a `channel_announcement` @@ -707,9 +707,9 @@ timeouts. It also causes a natural ratelimiting of queries. 2. types: 1. type: 1 (`query_option`) 2. data: - * [`varint`:`query_option_flags`] + * [`bigsize`:`query_option_flags`] -`query_option_flags` is a bitfield represented as a minimally-encoded varint. Bits have the following meaning: +`query_option_flags` is a bitfield represented as a minimally-encoded bigsize. Bits have the following meaning: | Bit Position | Meaning | | ------------- | ----------------------- | @@ -732,7 +732,7 @@ Though it is possible, it would not be very useful to ask for checksums without 2. types: 1. type: 1 (`timestamps_tlv`) 2. data: - * [`u8`:`encoding_type`] + * [`byte`:`encoding_type`] * [`...*byte`:`encoded_timestamps`] 1. type: 3 (`checksums_tlv`) 2. data: