No support for reading client certificate and private key from LittleFS (ESP8266WiFi - WiFiClientSecure/BearSSL)

[itay7564]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: All
• Core Version: 24-Oct-2020
• Development Env: Arduino IDE
• Operating System: All

Problem Description

Currently, when using WiFiClientSecure (BearSSL), certificate stores can be loaded from LittleFS or SD.
But there is no documented way or code to load a client certificate and private key in a similar manner.
(The X509List and PrivateKey do not take files/streams as arguments)

Old issues and examples show that older versions used to have this feature:

• https://hackaday.io/project/12482-garage-door-opener/log/45617-connecting-the-esp8266-with-tls
• Can ESP-12E support MQTT with Client Certificates #3544

Specifically it seems like the old functions loadCertificate() and loadPrivateKey() (which are deprecated) could load files.

My current solution is to copy the certificate and key to a global variable, which wastes several KB's RAM:

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <LittleFS.h>
#include <WiFiClientSecure.h>

#define MAX_PEM_SIZE 4096

char clientKeyStr[MAX_PEM_SIZE];
char clientCertStr[MAX_PEM_SIZE];

void setup() {
  LittleFS.begin();
  Serial.begin(115200);

  //... initialize wifi and time...

  File cert = LittleFS.open("/client-crt.pem", "r"); //can be .der file as well
  File key = LittleFS.open("/client-key.pem", "r");

  //... verify files are opened correctly and not exceed MAX_PEM_SIZE ...

  cert.readBytes(clientCertStr, cert.size()); //copy certificate from file to char array
  key.readBytes(clientKeyStr, key.size()); //same for private key
  X509List clientCert(clientCertStr); 
  PrivateKey clientKey(clientKeyStr);

 //...connect to server...
}

This sketch works well but wastes 4096*2 = 8192 bytes of RAM which is 10% of total RAM.
My assumption is, when using CertStoreBearSSL.h, the certificates are not copied to the RAM for most of the time, but loaded in a different way.
Same thing goes when loading a certificate which is saved in PROGMEM (sketch ROM).
Therefore, it should be possible to use a client certificates and a private key which are stored as .pem or .der in the file system, without copying the whole file content to the RAM, for the whole lifetime of the program.
I tried understanding the code in CertStoreBearSSL.cpp but it's too complicated for me.
Thanks in advance and Best regards!

[earlephilhower]

I think there's a little misunderstanding here. Once you create the X509List there is no need to preserve the undecoded bits.

So your code could become (hacking in editor, please excuse any typos!):

X509List *clientCert;
void setup() {
  LittleFS.begin();
  Serial.begin(115200);

  //... initialize wifi and time...

  char *buff = new char[4096];

  File cert = LittleFS.open("/client-crt.pem", "r"); //can be .der file as well
  cert.readBytes(buff cert.size());
  cert.close();
  clientCert = new X509List(buff); 

  ... repeat for the key ...

  delete buff;

Note that the cert and key need to be long-lived, so they should be stored as a global so connections in your loop() can access it. Once the string is parsed by the constructor, you can free that temp memory.

There's no add'l memory savings possible. The cert/key you're using need to be in RAM when a connection is made (and while it's live, because there can be renegotiation at any time per the SSL protocol) so the X509List and PrivateKey need to live forever.

So, while I think making the Cert/Key constructors create from a Stream is a good addition, it's not going to materially affect things other than removing a little user code (i.e. no RAM difference).

[itay7564]

Thanks for the clarification! this solves the issue for me.