Complete the Fundamental Types. (lightning#778)

* Rename all the 'varint' to 'bigsize'. Having both is confusing; we chose the name bigsize, so use it explicitly. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> * BOLT 7: use `byte` instead of `u8`. `u8` isn't a type; see BOLT #1 "Fundamental Types". Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> * BOLT 1: promote bigsize to a Fundamental Type. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell · May 25, 2020 · 9e8e29a · 9e8e29a
1 parent 5322c2b
commit 9e8e29a
Show file tree

Hide file tree

Showing 4 changed files with 27 additions and 34 deletions.
diff --git a/.aspell.en.pws b/.aspell.en.pws
@@ -355,7 +355,6 @@ verifier
 verifiers
 EOF
 monotonicity
-varint
 optimizations
 structs
 CompactSize
@@ -367,8 +366,6 @@ namespaces
 tlvs
 fips
 rfc
-varint
-CompactSize
 multipath
 mpp
 tlvs

diff --git a/01-messaging.md b/01-messaging.md
@@ -110,28 +110,23 @@ the backwards-compatible addition of new fields to existing message types.
 
 A `tlv_record` represents a single field, encoded in the form:
 
-* [`varint`: `type`]
-* [`varint`: `length`]
+* [`bigsize`: `type`]
+* [`bigsize`: `length`]
 * [`length`: `value`]
 
-A `varint` is a variable-length, unsigned integer encoding using the
-[BigSize](#appendix-a-bigsize-test-vectors) format, which resembles the bitcoin
-CompactSize encoding but uses big-endian for multi-byte values rather than
-little-endian.
-
 A `tlv_stream` is a series of (possibly zero) `tlv_record`s, represented as the
 concatenation of the encoded `tlv_record`s. When used to extend existing
 messages, a `tlv_stream` is typically placed after all currently defined fields.
 
-The `type` is a varint encoded using the BigSize format. It functions as a
+The `type` is encoded using the BigSize format. It functions as a
 message-specific, 64-bit identifier for the `tlv_record` determining how the
 contents of `value` should be decoded. `type` identifiers below 2^16 are
 reserved for use in this specification. `type` identifiers greater than or equal
 to 2^16 are available for custom records. Any record not defined in this
 specification is considered a custom record. This includes experimental and
 application-specific messages.
 
-The `length` is a varint encoded using the BigSize format signaling the size of
+The `length` is encoded using the BigSize format signaling the size of
 `value` in bytes.
 
 The `value` depends entirely on the `type`, and should be encoded or decoded
@@ -192,7 +187,7 @@ things, enables the following optimizations:
  - variable-size fields can reserve their expected size up front, rather than
    appending elements sequentially and incurring double-and-copy overhead.
 
-The use of a varint for `type` and `length` permits a space savings for small
+The use of a bigsize for `type` and `length` permits a space savings for small
 `type`s or short `value`s. This potentially leaves more space for application
 data over the wire or in an onion payload.
 
@@ -236,6 +231,7 @@ The following convenience types are also defined:
 * `signature`: a 64-byte bitcoin Elliptic Curve signature
 * `point`: a 33-byte Elliptic Curve point (compressed encoding as per [SEC 1 standard](http://www.secg.org/sec1-v2.pdf#subsubsection.2.3.3))
 * `short_channel_id`: an 8 byte value identifying a channel (see [BOLT #7](07-routing-gossip.md#definition-of-short-channel-id))
+* `bigsize`: a variable-length, unsigned integer similar to Bitcoin's CompactSize encoding, but big-endian.  Described in [BigSize](#appendix-a-bigsize-test-vectors).
 
 ## Setup Messages
 
@@ -474,10 +470,10 @@ decoded with BigSize should be checked to ensure they are minimally encoded.
 
 The following is an example of how to execute the BigSize decoding tests.
 ```golang
-func testReadVarInt(t *testing.T, test varIntTest) {
+func testReadBigSize(t *testing.T, test bigSizeTest) {
         var buf [8]byte 
         r := bytes.NewReader(test.Bytes)
-        val, err := tlv.ReadVarInt(r, &buf)
+        val, err := tlv.ReadBigSize(r, &buf)
         if err != nil && err.Error() != test.ExpErr {
                 t.Fatalf("expected decoding error: %v, got: %v",
                         test.ExpErr, err)
@@ -541,19 +537,19 @@ A correct implementation should pass against these test vectors:
         "name": "two byte not canonical",
         "value": 0,
         "bytes": "fd00fc",
-        "exp_error": "decoded varint is not canonical"
+        "exp_error": "decoded bigsize is not canonical"
     },
     {
         "name": "four byte not canonical",
         "value": 0,
         "bytes": "fe0000ffff",
-        "exp_error": "decoded varint is not canonical"
+        "exp_error": "decoded bigsize is not canonical"
     },
     {
         "name": "eight byte not canonical",
         "value": 0,
         "bytes": "ff00000000ffffffff",
-        "exp_error": "decoded varint is not canonical"
+        "exp_error": "decoded bigsize is not canonical"
     },
     {
         "name": "two byte short read",
@@ -604,14 +600,14 @@ A correct implementation should pass against these test vectors:
 
 The following is an example of how to execute the BigSize encoding tests.
 ```golang
-func testWriteVarInt(t *testing.T, test varIntTest) {
+func testWriteBigSize(t *testing.T, test bigSizeTest) {
         var (
                 w   bytes.Buffer
                 buf [8]byte
         )
-        err := tlv.WriteVarInt(&w, test.Value, &buf)
+        err := tlv.WriteBigSize(&w, test.Value, &buf)
         if err != nil {
-                t.Fatalf("unable to encode %d as varint: %v",
+                t.Fatalf("unable to encode %d as bigsize: %v",
                         test.Value, err)
         }
 

diff --git a/04-onion-routing.md b/04-onion-routing.md
@@ -95,7 +95,7 @@ There are a number of conventions adhered to throughout this document:
  - Each hop in the route has a variable length `hop_payload`, or a fixed-size
    legacy `hop_data` payload.
     - The legacy `hop_data` is identified by a single `0x00`-byte prefix
-    - The variable length `hop_payload` is prefixed with a `varint` encoding
+    - The variable length `hop_payload` is prefixed with a `bigsize` encoding
       the length in bytes, excluding the prefix and the trailing HMAC.
 
 # Key Generation
@@ -163,7 +163,7 @@ It is 1300 bytes long and has the following structure:
 
 1. type: `hop_payloads`
 2. data:
-   * [`varint`:`length`]
+   * [`bigsize`:`length`]
    * [`hop_payload_length`:`hop_payload`]
    * [`32*byte`:`hmac`]
    * ...
@@ -523,10 +523,10 @@ For each hop in the route, in reverse order, the sender applies the
 following operations:
 
  - The _rho_-key and _mu_-key are generated using the hop's shared secret.
- - `shift_size` is defined as the length of the `hop_payload` plus the varint encoding of the length and the length of that HMAC. Thus if the payload length is `l` then the `shift_size` is `1 + l + 32` for `l < 253`, otherwise `3 + l + 32` due to the varint encoding of `l`.
+ - `shift_size` is defined as the length of the `hop_payload` plus the bigsize encoding of the length and the length of that HMAC. Thus if the payload length is `l` then the `shift_size` is `1 + l + 32` for `l < 253`, otherwise `3 + l + 32` due to the bigsize encoding of `l`.
  - The `hop_payload` field is right-shifted by `shift_size` bytes, discarding the last `shift_size`
  bytes that exceed its 1300-byte size.
- - The varint-serialized length, serialized `hop_payload` and `hmac` are copied into the following `shift_size` bytes.
+ - The bigsize-serialized length, serialized `hop_payload` and `hmac` are copied into the following `shift_size` bytes.
  - The _rho_-key is used to generate 1300 bytes of pseudo-random byte stream
  which is then applied, with `XOR`, to the `hop_payloads` field.
  - If this is the last hop, i.e. the first iteration, then the tail of the
@@ -662,7 +662,7 @@ The routing information is then deobfuscated, and the information about the
 next hop is extracted.
 To do so, the processing node copies the `hop_payloads` field, appends 1300 `0x00`-bytes,
 generates `2*1300` pseudo-random bytes (using the _rho_-key), and applies the result, using `XOR`, to the copy of the `hop_payloads`.
-The first few bytes correspond to the varint-encoded length `l` of the `hop_payload`, followed by `l` bytes of the resulting routing information become the `hop_payload`, and the 32 byte HMAC.
+The first few bytes correspond to the bigsize-encoded length `l` of the `hop_payload`, followed by `l` bytes of the resulting routing information become the `hop_payload`, and the 32 byte HMAC.
 The next 1300 bytes are the `hop_payloads` for the outgoing packet.
 
 A special `hmac` value of 32 `0x00`-bytes indicates that the currently processing hop is the intended recipient and that the packet should not be forwarded.
@@ -1000,7 +1000,7 @@ The CLTV expiry in the HTLC is too far in the future.
 
 1. type: PERM|22 (`invalid_onion_payload`)
 2. data:
-   * [`varint`:`type`]
+   * [`bigsize`:`type`]
    * [`u16`:`offset`]
 
 The decrypted onion per-hop payload was not understood by the processing node

diff --git a/07-routing-gossip.md b/07-routing-gossip.md
@@ -602,10 +602,10 @@ Nodes can signal that they support extended gossip queries with the `gossip_quer
 2. types:
     1. type: 1 (`query_flags`)
     2. data:
-        * [`u8`:`encoding_type`]
+        * [`byte`:`encoding_type`]
         * [`...*byte`:`encoded_query_flags`]
 
-`encoded_query_flags` is an array of bitfields, one varint per bitfield, one bitfield for each `short_channel_id`. Bits have the following meaning:
+`encoded_query_flags` is an array of bitfields, one bigsize per bitfield, one bitfield for each `short_channel_id`. Bits have the following meaning:
 
 | Bit Position  | Meaning                                  |
 | ------------- | ---------------------------------------- |
@@ -642,7 +642,7 @@ The sender:
   - SHOULD NOT send this if the channel referred to is not an unspent output.
   - MAY include an optional `query_flags`. If so:
     - MUST set `encoding_type`, as for `encoded_short_ids`.
-    - Each query flag is a minimally-encoded varint.
+    - Each query flag is a minimally-encoded bigsize.
     - MUST encode one query flag per `short_channel_id`.
 
 The receiver:
@@ -663,7 +663,7 @@ The receiver:
       - MUST follow with any `node_announcement`s for each `channel_announcement`
     - otherwise:
       - We define `query_flag` for the Nth `short_channel_id` in
-        `encoded_short_ids` to be the Nth varint of the decoded
+        `encoded_short_ids` to be the Nth bigsize of the decoded
         `encoded_query_flags`.
       - if bit 0 of `query_flag` is set:
         - MUST reply with a `channel_announcement`
@@ -707,9 +707,9 @@ timeouts.  It also causes a natural ratelimiting of queries.
 2. types:
     1. type: 1 (`query_option`)
     2. data:
-        * [`varint`:`query_option_flags`]
+        * [`bigsize`:`query_option_flags`]
 
-`query_option_flags` is a bitfield represented as a minimally-encoded varint. Bits have the following meaning:
+`query_option_flags` is a bitfield represented as a minimally-encoded bigsize. Bits have the following meaning:
 
 | Bit Position  | Meaning                 |
 | ------------- | ----------------------- |
@@ -732,7 +732,7 @@ Though it is possible, it would not be very useful to ask for checksums without
 2. types:
     1. type: 1 (`timestamps_tlv`)
     2. data:
-        * [`u8`:`encoding_type`]
+        * [`byte`:`encoding_type`]
         * [`...*byte`:`encoded_timestamps`]
     1. type: 3 (`checksums_tlv`)
     2. data: