Unexpected end of JSON input error

General
1

I have an ESP based device operating fine. It sends two MQTT messages to node red running on a pi zero.

The first message comes in and goes through a JSON node to convert from a string to JSON object.

The second message is sent in the same way, from the same device, using the same function (all worked out with the arduinoJSON assistant) but the second message returns the error after the JSON node

"msg : string[28]"
"Unexpected end of JSON input"

Ive used mqttfx to pick up the message and can see the issue.

The expected message is something like

{
"heaterDemand":false,
"heaterStarts":123456,
"heaterStartsPerDay":12345,
"heaterRunTime":123456.0,
"heaterRunTimePerDay":12346.0
}

but the actual message is

{"heaterDemand":false,"heaterStarts":0,"heaterStartsPerDay":0,"heaterRunTime":0

So the end of the message is being cut off.

I suspect if I increase the buffer object size it should work but the arduinoJSON assistant has worked out the value. So Im a bit stuck.

This is the sending program in case its of interest.

const size_t capacity4 = JSON_OBJECT_SIZE(5);
      DynamicJsonDocument doc4(capacity4);

      doc4["heaterDemand"]        = heaterDemand;
      doc4["heaterStarts"]        = heaterStarts;
      doc4["heaterStartsPerDay"]  = heaterStartsPerDay;
      doc4["heaterRunTime"]       = heaterRunTime;
      doc4["heaterRunTimePerDay"] = heaterRunTimePerDay;

      char buffer4[capacity4];
      serializeJson(doc4, buffer4, sizeof(buffer4));

      sendValues(buffer4, mqtt_publish_topic2);

The send function works as its sending the first message. unless this is some kind of memory issue on the device (wemos D1 mini, or maybe a clone, cant remember)?

I guess I could try to increase the buffer size by the following or something similar, but as I say, this was worked out the same way as the first message which is much larger and works fine.

const size_t capacity4 = JSON_OBJECT_SIZE(5) + 80;

Any direction appreciated. Thanks

2

On the esp device, what are you using for MQTT? Check and see what it's buffer size is.

3

Ive expanding the library limit from 128 to 256 as the other message is much larger. So I know that part of it isnt an issue.

Its the pubsubclient library.

Unless there is some kind of total limit of all objects???

4

Have you tried printing your MQTT msg in your esp code so you can look at what it is?

What do have you have the 'Output' the mqtt-in node configured to?

Do you have a debug node attached to the output of the mqtt-in node to look at all the messages coming in? Could it be a LWT msg coming thru?

  • what version of NR and node.js? (see startup log)
5

A few things that might be worth trying -
You said the first message is ok but not the second. So they come in pairs is it always the second that fails? What if you disable the first so it only seems the second? What if you swap the order? What if you put a gap between them?
Could something you are doing in the device lock it up temporarily before it sends the last character?

6

@zenofmud

the JSON node responds with the error only and no debug, but after picking up the message in mqttfx and I can confirm the debug node does show the same string from the mqtt node. I think my version is 0.9 or something, I have been apprehensive about upgrading it in case it causes any issues.

@Colin

I havent tried any of these things yet. I did notice on a previous project that if it went into deepsleep too quickly after trying to send an mqtt message then it wouldnt send, but as this device isnt going to sleep I dont know whats happening. I will serial output the content of the message and see what I get directly from the device and also try disabling the first message.

Is there a total available memory limit I might be exceeding on the device itself? Thats not something I ever considered as there are also messages coming in but they dont appear to have any issues.

7

Version of what? If you look at the log when NR starts, you will see both the version for Nose-RED and node.js.

8

node - 10.15.3
nodered - 0.19.6

I thought nodered was 0.9 since they started releasing v1 , whoops.

Ive havent had any running issues with this however so I dont really want to change it if I dont have to. I think this issue is on the ESP.

9

What ESP device are you using and what are you using to code on the device?

10

Its a wemos d1 mini or a clone, cant remember, and im using the Arduino IDE.

I will have a look at the previous suggestion of disabling the first message to see if the second works ok, and also to serial output the content to see if the message is complete before it gets sent out

11

You can upgrade and downgrade NR by specifying the version number to use. Version 0.19.6 is pretty old at this point. You could upgrade to a newer version and always fall back to the older version if you ran into an error4.

Of course you would backup your device first.

What platform is NR running on?

12

raspbian stretch on a pi zero.

When you say a node red backup, im exporting the data as a text file after making any changes, but is there anything else I should do? thanks

13

Clone the SD card.

14

There is no need to upgrade Node-RED here - the MQTT nodes have not been broken / changed in a long time. This is about packet fragmentation - either on the sending or receiving side.

When you say you checked with MQTTfx - it's not clear to me if it's showing that the data received is correct or not. Is it truncated there as well ?

15

the output in mqttfx was also cut off. Ive just changed the line setting the buffer size as follows

const size_t capacity4 = JSON_OBJECT_SIZE(5) + 90;

and now the full message is coming through. This is odd as it didnt suggest I would need this in the arduinoJSON assistant, but it did say an extended buffer may be needed in other applications. Like I say I used the same method for all the other messages Ive set up in this same way so this was the odd one out. But at least its working now!

2 Likes
16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.