JSON gets corrupted when created from a temporary buffer

[Anton-V-K]

When JSON is read from file (SPIFFS is used) into DynamicJsonBuffer and some values are modified, it gets corrupted.

Sample code for testing:

#include <ArduinoJson.h>
#include <FS.h>

void setup() {

    Serial.begin(115200); // Init console
  
    SPIFFS.begin();
  
    DynamicJsonBuffer jsonBuffer;
    
    JsonObject* root = NULL;
    
    static const char* FILENAME = "Test.json";
/*    
    SPIFFS.remove(FILENAME);
    return;
*/    
    if (File file = SPIFFS.open(FILENAME, "r"))
    {
      if (const size_t size = file.size())
      {
        std::unique_ptr<char[]> buf (new char[size]);
        file.readBytes(buf.get(), size);
        root = &jsonBuffer.parseObject(buf.get()); // BUG - should be: &jsonBuffer.parseObject((const char*)buf.get())
        root->prettyPrintTo(Serial); // DEBUG
        Serial.println("\nLoaded!");
      }
      file.close();
    }
    if (File file = SPIFFS.open(FILENAME, "w"))
    {
      if (!root)
        root = &jsonBuffer.createObject();
        
      (*root)["DEVICE_ID"] = 1;
      (*root)["millis"]    = millis();
      
      root->prettyPrintTo(Serial); // DEBUG
      root->printTo(file);
      file.close();
      Serial.println("\nSaved!");
    }
    
}

void loop() {
  // not used in this example
}

The output when ran 1st time (OK):

{
  "DEVICE_ID": 1,
  "millis": 553
}
Saved!

The output when ran 2nd time (JSON is corrupted):

{
  "DEVICE_ID": 1,
  "millis": 553
}
Loaded!
{
  "\t�": 1,
  "millis": 322,
  "DEVICE_ID": 1
}
Saved!

Expected output when ran 2nd time:

{
  "DEVICE_ID": 1,
  "millis": 553
}
Loaded!
{
  "DEVICE_ID": 1
  "millis": 322,
}
Saved!

Target platform: ESP8266 (WeMos D1 R2)
IDE: Arduino IDE 1.6.13
LIB version: 5.11.2

I've finally managed to find an issue in my code: the content of the buffer for file operation isn't copied into DynamicJsonBuffer, so its memory becomes invalid when execution goes out of scope (I've adjusted the initial code).
I've also found the sample how to parse JSON from SPIFFS, which makes temporary buffer redundant + shorter code.

[bblanchon]

Hi @Anton-V-K

As I understand, you found the solution by yourself. 👍

Since the input of parseObject() was a char*, the parser assumed it to be persistent; and therefore stored a pointer to a temporary buffer.

Now, as you found, you can avoid this problem by just calling parseObject(file).
In that case, the parser knows that the input is volatile and will make the copy in the JsonBuffer.

Is there anything else I can do?

Regards,
Benoit

[Anton-V-K]

Well, I guess the issue may be closed. Though I think the interface of DynamicJsonBuffer is somewhat confusing with overloaded method parseObject - the specifics of handling const/non-const buffer is easily overlooked...

[bblanchon]

Agreed, @Anton-V-K.
Unfortunately, this came organically with the evolution of the library.
Do you have any suggestion for futures versions?