v2.115.15

balena-os · Jun 28, 2023 · 916c6f9 · 916c6f9
1 parent 713e100
commit 916c6f9
Show file tree

Hide file tree

Showing 3 changed files with 207 additions and 1 deletion.
diff --git a/.versionbot/CHANGELOG.yml b/.versionbot/CHANGELOG.yml
@@ -1,3 +1,182 @@
+- commits:
+    - subject: Update layers/meta-balena to c2e6ca9d57fd7214fe7c078591741a9c0fa6de73
+      hash: 2eb3465e242adfcdfd956cae945f19026e728256
+      body: Update layers/meta-balena
+      footer:
+        Changelog-entry: Update layers/meta-balena to c2e6ca9d57fd7214fe7c078591741a9c0fa6de73
+        changelog-entry: Update layers/meta-balena to c2e6ca9d57fd7214fe7c078591741a9c0fa6de73
+      author: Self-hosted Renovate Bot
+      nested:
+        - commits:
+            - subject: Update balena-engine to v20.10.37
+              hash: c607d7be4d740802ab5e27b67f5e6714e1407855
+              body: |
+                Update balena-engine from 20.10.36 to 20.10.37
+              footer:
+                Change-type: patch
+                change-type: patch
+              author: Self-hosted Renovate Bot
+              nested:
+                - commits:
+                    - subject: "Bugfix: concatReadSeekCloser.Read() with buffers of any size"
+                      hash: 84471473ce02a0b8820e13105f845b67c19903c9
+                      body: >
+                        Previously, `concatReadSeekCloser.Read()` would
+                        incorrectly return
+
+                        an `io.ErrUnexpectedEOF` if the last read from the second concatenated
+
+                        `Reader` didn't completely fill the passed buffer.
+
+
+                        For instance:
+
+
+                        ```
+                              First Reader    Second Reader
+                            |aaaaaaaaaaaaa|bbbbbbbbbbbbbbbbbbb|      <--- concatReadSeekCloser
+                            |--- previously read ---|--- buffer ---| <--- last read
+                                                              |xxxx| <--- "excess" buffer
+                        ```
+
+
+                        In this example, we have a `concatReadSeekCloser` that concatenates two
+
+                        `Reader`s (`aaa...` and `bbb...`). The last `Read()` used a buffer
+
+                        larger than the yet-to-be-read portion of the `bbb...`. So, it would
+
+                        incorrectly return an `io.ErrUnexpectedEOF`.
+
+
+                        This commit makes sure that last `Read()` returns all the remaining data
+
+                        without an error. It also adds various test cases for
+
+                        `concatReadSeekCloser.Read()`, many of which would fail before this
+
+                        correction.
+
+
+                        Interestingly, this bug was silently affecting us. Not in a fatal way,
+
+                        but causing deltas to be larger than necessary. Indeed, running
+
+                        `TestDeltaSize` after this commit shows that some test cases are
+
+                        producing deltas smaller than what we expected before. For posterity,
+
+                        see all the details below.
+
+
+                        We use `concatReadSeekCloser`s to concatenate all layers of the basis
+
+                        image when creating the "signature" of the basis image. In this process,
+
+                        the `concatReadSeekCloser`s are wrapped around by a buffered reader with
+
+                        a buffer of 65kB.
+
+
+                        If, in any read, part of this 65kB buffer was beyond the second
+
+                        concatenated reader, it would result in an `io.ErrUnexpectedEOF`. This
+
+                        would not cause the whole process to fail, but would prematurely end the
+
+                        signature generation: some of the final blocks in the basis image would
+
+                        not be added to the signature. Therefore, if those blocks appeared in
+
+                        the target image, they'd result in (larger) LITERAL, instead of
+
+                        (smaller) COPY operations.
+
+
+                        For illustration, here's the delta generated for the `delta-006-008`
+
+                        test case. First before this commit:
+
+
+                        ```
+                          OP_COPY_N1_N2 0 512
+                          OP_COPY_N2_N2 1536 1024
+                          OP_COPY_N2_N2 3584 1024
+                          OP_COPY_N2_N2 5632 1536
+                          OP_COPY_N2_N2 8192 1536
+                          OP_COPY_N2_N2 10752 4096
+                          OP_COPY_N2_N2 15872 8704
+                          OP_COPY_N2_N2 25600 10752
+                          OP_COPY_N2_N2 37376 10752
+                          OP_COPY_N2_N4 49152 131584
+                          OP_COPY_N4_N4 181760 150528
+                          OP_COPY_N4_N4 333312 500736
+                          OP_COPY_N4_N4 835072 1000960
+                          OP_COPY_N4_N4 1837056 1000960
+                          OP_COPY_N4_N4 2839040 1027584    # Here: a COPY op...
+                          OP_LITERAL_N2 21504              # ...followed by a 21 kB LITERAL.
+                          OP_COPY_N4_N2 2838528 512
+                          OP_COPY_N4_N2 2838528 512
+                                 OP_END
+                        ```
+
+
+                        And after this commit:
+
+
+                        ```
+                         OP_COPY_N1_N2 0 512
+                          OP_COPY_N2_N2 1536 1024
+                          OP_COPY_N2_N2 3584 1024
+                          OP_COPY_N2_N2 5632 1536
+                          OP_COPY_N2_N2 8192 1536
+                          OP_COPY_N2_N2 10752 4096
+                          OP_COPY_N2_N2 15872 8704
+                          OP_COPY_N2_N2 25600 10752
+                          OP_COPY_N2_N2 37376 10752
+                          OP_COPY_N2_N4 49152 131584
+                          OP_COPY_N4_N4 181760 150528
+                          OP_COPY_N4_N4 333312 500736
+                          OP_COPY_N4_N4 835072 1000960
+                          OP_COPY_N4_N4 1837056 1000960
+                          OP_COPY_N4_N4 2839040 1049088    # COPY only, since we detected a longer match
+                          OP_COPY_N4_N2 3888640 512
+                          OP_COPY_N4_N2 3888640 512
+                                 OP_END
+                        ```
+
+
+                        That 21kB LITERAL is the difference in size we saw in the test results.
+                      footer:
+                        Signed-off-by: Leandro Motta Barros <leandro@balena.io>
+                        signed-off-by: Leandro Motta Barros <leandro@balena.io>
+                        Change-type: patch
+                        change-type: patch
+                      author: Leandro Motta Barros
+                      nested: []
+                    - subject: Minor code and comments tweaks
+                      hash: 7dd51428c918980163fbe34511152dc72e7a3372
+                      body: >
+                        Using `defer` for the sake of being more idiomatic (and
+                        maybe slightly
+
+                        more reliable); plus, using the proper doc comment standards.
+                      footer:
+                        Signed-off-by: Leandro Motta Barros <leandro@balena.io>
+                        signed-off-by: Leandro Motta Barros <leandro@balena.io>
+                        Change-type: patch
+                        change-type: patch
+                      author: Leandro Motta Barros
+                      nested: []
+                  version: balena-engine-20.10.37
+                  title: ""
+                  date: 2023-06-26T13:44:22.217Z
+          version: meta-balena-2.115.15
+          title: ""
+          date: 2023-06-28T02:21:32.534Z
+  version: 2.115.15
+  title: ""
+  date: 2023-06-28T04:22:28.589Z
 - commits:
     - subject: Update layers/meta-balena to 515a2063746f0a5a1aea47cd6064673f7e80a716
       hash: cd50d4e7444d5949612a70a9dd0c41c09f312552

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,6 +1,33 @@
 Change log
 -----------
 
+# v2.115.15
+## (2023-06-28)
+
+
+<details>
+<summary> Update layers/meta-balena to c2e6ca9d57fd7214fe7c078591741a9c0fa6de73 [Self-hosted Renovate Bot] </summary>
+
+> ## meta-balena-2.115.15
+> ### (2023-06-28)
+> 
+> 
+> <details>
+> <summary> Update balena-engine to v20.10.37 [Self-hosted Renovate Bot] </summary>
+> 
+>> ### balena-engine-20.10.37
+>> #### (2023-06-26)
+>> 
+>> * Bugfix: concatReadSeekCloser.Read() with buffers of any size [Leandro Motta Barros]
+>> * Minor code and comments tweaks [Leandro Motta Barros]
+>> 
+> 
+> </details>
+> 
+> 
+
+</details>
+
 # v2.115.14
 ## (2023-06-28)
 

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-2.115.14
+2.115.15