Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

httpUpdateSigned don't work if used gzipped firmware #7570

Closed
2 tasks
krc-soft opened this issue Sep 2, 2020 · 6 comments · Fixed by #7577
Closed
2 tasks

httpUpdateSigned don't work if used gzipped firmware #7570

krc-soft opened this issue Sep 2, 2020 · 6 comments · Fixed by #7577
Labels
waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.

Comments

@krc-soft
Copy link

krc-soft commented Sep 2, 2020

Basic Infos

  • [ x] This issue complies with the issue POLICY doc.
  • [ x] I have read the documentation at readthedocs and the issue is not addressed there.
  • I have tested that the issue is present in current master branch (aka latest git).
  • [ x] I have searched the issue tracker for a similar issue.
  • If there is a stack dump, I have decoded it.
  • [ x] I have filled out all fields below.

Platform

  • Hardware: [ESP-12]
  • Core Version: [2.7.4]
  • Development Env: [Arduino IDE]
  • Operating System: [Windows]

Settings in IDE

  • Module: [Nodemcu]
  • Flash Mode: [dio]
  • Flash Size: [4MB (FS:1MB OTA:~1019KB)]
  • lwip Variant: [v2 Lower Memory]
  • Reset Method: [ck|nodemcu]
  • Flash Frequency: [40Mhz]
  • CPU Frequency: [80Mhz]
  • Upload Using: [OTA]

Problem Description

Signed http update don't work correctly if using gzipped firmware. During update no error appears and the signature check worked. But the esp boots to the old firmware and when i reset the esp it hangs and cannot boot anymore.

I used the httpUpdateSigned example build it with Arduino IDE v1.8.11.
After compiling i gzipped the firmware.bin and signed it with the signing.py script according to the documentation.

An unsigned gzipped firmware works and signed unzipped firmware works also.

MCVE Sketch


/*
   httpUpdateSigned.ino - Earle F. Philhower, III
   Released into the Public Domain

   For use while building under Linux or Mac.

   Automatic code signing is not supported on Windows, so this example
   DOES NOT WORK UNDER WINDOWS.

   Shows how to use a public key extracted from your private certificate to
   only allow updates that you have signed to be applied over HTTP.  Remote
   updates will require your private key to sign them, but of course
   **ANYONE WITH PHYSICAL ACCESS CAN UPDATE THE 8266 VIA THE SERIAL PORT**.
*/

#include <Arduino.h>

#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>

#include <ESP8266HTTPClient.h>
#include <ESP8266httpUpdate.h>

#ifndef STASSID
#define STASSID "ssid"
#define STAPSK  "pass"
#endif

ESP8266WiFiMulti WiFiMulti;

#define MANUAL_SIGNING 1

// This example is now configured to use the automated signing support
// present in the Arduino IDE by having a "private.key" and "public.key"
// in the sketch folder.  You can also programmatically enable signing
// using the method shown here.

// This key is taken from the server public certificate in BearSSL examples
// You should make your own private/public key pair and guard the private
// key (never upload it to the 8266).
const char pubkey[] PROGMEM = R"EOF(
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu1Pt7yEk/xI+6cozLj5B
u4xV8gXDXcHS0rSJFfl4wBTk4UXpaJRaLfR1k0juEEa5LBRZaoA0iLj2e6kfCibO
Nx0VVoWmeqN2HBc3zkA1eqCksI0QUudzto4KhKHp0odiZ2zo6c/2Tn1zqD/m3OLo
SjVTbsJmGuwx8RGMBXozpg/uL0hHflihX+HND4Xfw92QXv7SaPBhgvM9xyRxn0/w
3J2nNjtuPuVN5vcQkd8ncMexVfy9AWp+HSA5AT5N8CJ/EeIsdDMY1US28bUePzj1
WIo75bZHKZNFw/iXe2xoPpm74qriMNSlW2craFP2K3KYnI28vJeUU6t9I6LS9zt2
zQIDAQAB
-----END PUBLIC KEY-----
)EOF";
#if MANUAL_SIGNING
BearSSL::PublicKey *signPubKey = nullptr;
BearSSL::HashSHA256 *hash;
BearSSL::SigningVerifier *sign;
#endif

void setup() {

  Serial.begin(115200);
  // Serial.setDebugOutput(true);

  Serial.println();
  Serial.println();
  Serial.println();

  for (uint8_t t = 4; t > 0; t--) {
    Serial.printf("[SETUP] WAIT %d...\n", t);
    Serial.flush();
    delay(1000);
  }

  WiFi.mode(WIFI_STA);
  WiFiMulti.addAP(STASSID, STAPSK);

  #if MANUAL_SIGNING
  signPubKey = new BearSSL::PublicKey(pubkey);
  hash = new BearSSL::HashSHA256();
  sign = new BearSSL::SigningVerifier(signPubKey);
  #endif
}


void loop() {
  // wait for WiFi connection
  if ((WiFiMulti.run() == WL_CONNECTED)) {

    WiFiClient client;

    #if MANUAL_SIGNING
    // Ensure all updates are signed appropriately.  W/o this call, all will be accepted.
    Update.installSignature(hash, sign);
    #endif
    // If the key files are present in the build directory, signing will be
    // enabled using them automatically

    ESPhttpUpdate.setLedPin(LED_BUILTIN, LOW);

    t_httpUpdate_return ret = ESPhttpUpdate.update(client, "http://192.168.0.2/firmware.bin");

    switch (ret) {
      case HTTP_UPDATE_FAILED:
        Serial.printf("HTTP_UPDATE_FAILED Error (%d): %s\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        break;

      case HTTP_UPDATE_NO_UPDATES:
        Serial.println("HTTP_UPDATE_NO_UPDATES");
        break;

      case HTTP_UPDATE_OK:
        Serial.println("HTTP_UPDATE_OK");
        break;
    }
  }
  delay(10000);
}

Debug Messages

[SETUP] WAIT 4...
[SETUP] WAIT 3...
[SETUP] WAIT 2...
[SETUP] WAIT 1...

ets Jan 8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 3584, room 16
tail 0
chksum 0xb0
csum 0xb0
v2843a5ac
@cp:0
ld

[SETUP] WAIT 4...
[SETUP] WAIT 3...
[SETUP] WAIT 2...
[SETUP] WAIT 1...

@earlephilhower
Copy link
Collaborator

Those debug messages don't really give much info. Can you bump up to full debug and re-run?

Sign checks are done in Updater.cpp before a reboot, and it doesn't care/know if the blob is signed or raw bin. It just looks for the proper magic bytes at the end and a signed hash of the bits uploaded (i.e. you should first gzip, then sign the update...the other way won't work). The exact steps you used to make the signed binaries and a demo public/private keypar would be helpful, too.

@earlephilhower earlephilhower added the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label Sep 3, 2020
@krc-soft
Copy link
Author

krc-soft commented Sep 3, 2020

I use the public/private keypar included in the httpUpdateSigned example.

I build it with Arduino IDE v1.8.11 on Windows and use Debian on Windows (WSL) and call

  1. gzip -9 firmware.bin
  2. signing.py --mode sign --privatekey private.key --bin firmware.bin.gz --out firmware.bin.gz.signed
  3. after that i rename firmware.bin.gz.signed to firmware.bin and upload it to my webserver

Debug Messages

SDK:2.2.2-dev(38a443e)/Core:2.7.3-3-g2843a5ac=20703003/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-30-g92add50/BearSSL:5c771be



[SETUP] WAIT 4...
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt

connected with test, channel 1
dhcp client start...
ip:192.168.0.114,mask:255.255.255.0,gw:192.168.0.1
[SETUP] WAIT 3...
[SETUP] WAIT 2...
[SETUP] WAIT 1...
[httpUpdate] Header read fin.
[httpUpdate] Server header:
[httpUpdate]  - code: 200
[httpUpdate]  - len: 304594
[httpUpdate] ESP8266 info:
[httpUpdate]  - free Space: 2801664
[httpUpdate]  - current Sketch Size: 343232
[httpUpdate] runUpdate flash...
sleep disable
[begin] roundedSize:       0x0004B000 (307200)
[begin] updateEndAddress:  0x00300000 (3145728)
[begin] currentSketchSize: 0x00054000 (344064)
[begin] _startAddress:     0x002B5000 (2838528)
[begin] _currentAddress:   0x002B5000 (2838528)
[begin] _size:             0x0004A5D2 (304594)
pm open,type:0 0
[Updater] sigLen: 256
[Updater] Adjusted binsize: 304334
[Updater] Computed Hash: ea 6e 9e 92 a2 0d 1c 9f 45 02 bf 36 de 3d 04 64 d6 fd 83 06 d8 35 94 79 c9 81 cf 09 bb 5f 1e e9
[Updater] Received Signature: 69 39 fe 7a 1c df 7b 1b 4b 11 48 11 e5 72 bb 22 f5 d8 de 8a 78 c0 1a c9 84 f3 11 ff 0a 55 2a 0e 7c bb 5d 58 40 ce e0 08 52 32 66 90 70 31 d9 88 c0 97 4c 21 cb 21 99 b8 85 72 9d 1d a4 26 eb 99 fa 
11 23 d7 80 40 39 b8 ef 47 79 2d 42 ee 77 55 06 47 6e 0a 01 b1 3c d7 c3 8e 20 84 38 4b 02 28 32 ce 72 e6 48 e5 9c fe 39 86 c7 bd 97 37 f3 c9 4a c3 7f f9 d9 eb 6f c0 46 83 a0 c6 f2 09 03 3b 36 4f 96 12 de de a2 46 84 b3 b6 d5 
a5 6d 6a 00 f6 fa 50 5f 3a 2c ce 2d d4 ea de 0f a0 0d ec 1b 78 9a 60 68 c4 af 87 3a 9c fd bf 7e a3 ae bb 2a 3c a0 ab 5e 28 3d 43 fa 9c 36 9e 68 58 36 b6 9d f2 f4 9a e4 2b 5b 29 cb f8 e9 8e 53 3c 75 54 ec 62 75 9c 03 37 48 d4 
90 e3 45 5a 8f a5 39 71 22 2d f9 a0 ec 83 42 32 32 42 4a 3a 87 82 dc f2 32 7a 0b 45 6d dd ec d8 03 d9 41 b7 17 b0 51 62 23
[Updater] Signature matches
Staged: address:0x002B5000, size:0x0004A5D2
[httpUpdate] Update ok
state: 5 -> 0 (0)
rm 0
pm close 7
del if0
usl

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 3584, room 16
tail 0
chksum 0xb0
csum 0xb0
v2843a5ac
@cp:0
ld

@earlephilhower
Copy link
Collaborator

@cp:0 means the firmware was copied over by the bootloader.

Check

if (cmd.action == ACTION_COPY_RAW) {
ets_putc('c'); ets_putc('p'); ets_putc(':');

So, I'm seeing no issue here. The new firmware was copied the it was loaded (ld).

Make sure you're uploading the file you want and not another one (http cache or something weird like that).

@earlephilhower
Copy link
Collaborator

Also, if the original bootloader does not have GZIP support, you could end up having it write gzip'd data instead of opcodes when it copies the new app over. You need to ensure the bootloader has gzip support (i.e. upload one FW using uncompressed mode), and once that's up only then try and use gzip'd bins.

The signing stuff should be orthogonal to any gzip support. Signing is just doing a signed hash compare over a set of bits, it doesn't care what those bits are. Once the signing is validated, it should be exactly the same as unsigned.

@earlephilhower earlephilhower reopened this Sep 3, 2020
@earlephilhower
Copy link
Collaborator

Quick check of Update.cpp shows it might be getting the wrong bin size to copy, meaning the eboot gzip decompressor could abort because the len was invalid.

Try this patch, at line


add

    _size = binSize;

after the free() and report back.

@krc-soft
Copy link
Author

krc-soft commented Sep 3, 2020

It works 👍 thanks for your help!

earlephilhower added a commit to earlephilhower/Arduino that referenced this issue Sep 4, 2020
The last 4 bytes of a GZIP file is the decompressed file length, and
are used in eboot to do sanity checks and know when decompression is
done.

Updater was incorrectly telling eboot to look at
"end-of-bin + sizeof(signing)", and when eboot did so it got an
incorrect value causing either the update to be skipped or for only a
portion of update to be completed.

Fix by adjusting the size back to the end of binary.
Fixes esp8266#7570
earlephilhower added a commit that referenced this issue Sep 4, 2020
The last 4 bytes of a GZIP file is the decompressed file length, and
are used in eboot to do sanity checks and know when decompression is
done.

Updater was incorrectly telling eboot to look at
"end-of-bin + sizeof(signing)", and when eboot did so it got an
incorrect value causing either the update to be skipped or for only a
portion of update to be completed.

Fix by adjusting the size back to the end of binary.
Fixes #7570
krc-soft added a commit to krc-soft/Esp8266Arduino that referenced this issue Mar 7, 2024
The last 4 bytes of a GZIP file is the decompressed file length, and
are used in eboot to do sanity checks and know when decompression is
done.

Updater was incorrectly telling eboot to look at
"end-of-bin + sizeof(signing)", and when eboot did so it got an
incorrect value causing either the update to be skipped or for only a
portion of update to be completed.

Fix by adjusting the size back to the end of binary.
Fixes esp8266#7570
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants