QUESTION: Getting "null"s in return from fucntion returning DynamicJsonDocument

[garbled1]

ESP8266:

My code, simplified: some of the silliness like doc=json_doc; is my attempt to debug.

DynamicJsonDocument read_gnhast_config()
{
DynamicJsonDocument doc(50);
if (thing) {
DynamicJsonDocument json_doc(size);

    Serial.println("opened config file");
    configFile.readBytes(buf.get(), size);
    DeserializationError j_error =
	deserializeJson(json_doc, buf.get());
    if (!j_error) {
	serializeJson(json_doc, Serial);
	Serial.println("\nparsed json");
	doc = json_doc;
	return doc;
         }
 }
 Serial.printf("wrong");
 return(doc); /* which in theory should be a null json doc */

}

void other_function()
{
DynamicJsonDocument gncfg(5000);

gncfg = read_gnhast_config();
serializeJson(gncfg, Serial);

}

When I run this, I get the following on the Serial terminal:

reading gnhast config file
opened config file
{"device":{"28E768270500005E":{"name":"Test DS1820B"}}}
parsed json
{"device":{null:null}}

So it correctly reads the file. It parses the json, and sees the full document, but when it passes it on, it passes it as null:null.

What confuses me here, is that it does pass "device".

[garbled1]

Oh, quick note. Using 6.11.5

[bblanchon]

Hi @garbled1,

The DynamicJsonDocument is probably too small.

See:

• FAQ: https://arduinojson.org/v6/faq/why-is-the-output-incomplete/

Best Regards,
Benoit

[garbled1]

Oh... I totally misunderstood https://arduinojson.org/v6/api/dynamicjsondocument/
Under the Copying subheading there, it seems to indicate that if you do doc = doc2; and doc isn't big enough, it will auto-resize, so I (perhaps incorrectly) assumed that doc = doc_returning_function() would auto-resize.

I'll try this out tonight, I'm sure my problem will magically go away. :) Thank you!

[garbled1]

I think something else is going on here...

I tried setting the allocations of all of my DynamicJsonDocument's to 2048. My actual data is:
{"device":{"28E768270500005E":{"name":"Test DS18B20"}}}
Which according to the handy little calculator should be like 138 bytes or so on ESP

Now my code does this:

opened config file
{"device":{"28E768270500005E":{"name":"Test DS18B20"}}}
parsed json
{"⸮�⸮�ce":{"28E768270500005E":{"name":"Test DS18B20"}}}

The second print of the line is in the function that called the read from file function. So again, it reads it from disk, parses it correctly, and then when I return the value, it's corrupt. Now it's just corrupt differently... Can I not return a DynamicJsonDocument from a function? Do I have to do something else to do so?

My code now looks like this:

DynamicJsonDocument read_gnhast_config()
{
DynamicJsonDocument doc(JSON_DOC_WRITE_SIZE);

if (SPIFFS.exists("/gnhast.json")) {
File configFile = SPIFFS.open("/gnhast.json", "r");

Serial.println("reading gnhast config file");
if (configFile) {
    size_t size = configFile.size();
    // Allocate a buffer to store contents of the file.
    std::unique_ptr<char[]> buf(new char[size]);
    //DynamicJsonDocument json_doc(size);

    Serial.println("opened config file");
    configFile.readBytes(buf.get(), size);
    DeserializationError j_error =
	deserializeJson(doc, buf.get());
    if (!j_error) {
	serializeJson(doc, Serial);
	Serial.println("\nparsed json");
	return doc;
    } else {
	Serial.println("failed to load json config");
    }
} else
    Serial.println("Cannot open gnhast config file for reading");
}
return(doc);

}

int create_devices(int nrofdevs)
{
DynamicJsonDocument gncfg(JSON_DOC_WRITE_SIZE);

gncfg = read_gnhast_config();
serializeJson(gncfg, Serial);

}

[garbled1]

I think my fault.. adding a configFile.close() made all the problems go away. Some kind of memory corruption I guess? Odd...

[ewaldc]

@garbled1 ,
I was having a similar issue, based on a misunderstanding of how V5 works compared to V6. If you read the file into the buffer (buf) and the buffer is writeable then ArduinoJson will modify that buffer and use it as storage for the textual data in the JSON string. In order words, this buffer can not be a local variable and go out of scope, even when using new which allocates the buffer on the heap versus the stack. If you deserialize directly from a read-only variable (e.g. a const String) or a stream (e.g. your SPIFFS file), then ArduinoJson makes a copy of the source JSON data, and it's OK to have that variable (or file) go out of scope (or being closed). You will notice that in the read-only case, your DynamicJsonDocument needs much more space (extra space is more or less the size of the source JSON file/string).
So IMHO, your problem is not resolved by just adding the close().
Try :
deserializeJson(doc, configfile);
just keep into account the extra space needed to store the text/string data from the source file JSON string.
Alternatively e.g. when using classes, store the buffer as long as you need the JsonObject/Document e.g. into the class instance.

PS with doc = doc2; and doc isn't big enough, it will indeed auto-resize doc based on the size of doc2 (at least that's my understanding after looking at the code). However, if some storage container for doc or doc2 goes out of scope, the data will get corrupted (sooner or later).
Ewald

[bblanchon]

@ewaldc is right, configFile.close() is not the right fix.

1. You get garbage because the JsonDocument stores pointers to the temporary buffer buf.
2. JsonDocument stores pointers because deserializeJson() uses the zero-copy mode.
3. deserializeJson() uses the zero-copy mode because the input is a char*.

As Ewald said, you can fix this issue by passing the file directly to deserializeJson().

See also:

• deserializeJson()
• FAQ: Why does the output contain garbage?

[garbled1]

I think this makes more sense to me now. I'm used to doing C on Unix, so things like this come as a surprise to me. Makes alot of sense in a low memory environment. Just.. Caught me off guard.

However, learning that I can just straight up give it the filehandle, that's gold. So much easier. Seems to work fine now switching it over to the file handle, and it's less code.
Thank you!

[bblanchon]

You're welcome @garbled1.
Thank you for using ArduinoJson.
Don't forget to star the library to support the project 😉