# Reduce large JSON and publish over http endpoint

**URL:** <https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006>\
**Category:** General\
**Created:** [10 December 2019 20:30 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006 "2019-12-10T20:30:40Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 20:30 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/1 "2019-12-10T20:30:41Z")

</div>

Im having large JSON (darksky api - [https://darksky.net/dev/docs](https://darksky.net/dev/docs)) which is too large to process with esp8266 microcontroller. All i need from that JSON is timestamp and temperature from hourly \> data.

I've tried many ways, but always failed.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 20:35 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/2 "2019-12-10T20:35:23Z")

</div>

Do you have a small sample of the data you have, and an example of what you want to convert it to?

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 21:09 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/3 "2019-12-10T21:09:54Z")

</div>

```auto
{
"latitude": 48.778194,
"longitude": 18.618188,
"timezone": "Europe/Bratislava",
"currently": {
"time": 1575386648,
"summary": "Clear",
"icon": "clear-night",
"precipIntensity": 0,
"precipProbability": 0,
"temperature": 0.36,
"apparentTemperature": -4.29,
"dewPoint": -3.74,
"humidity": 0.74,
"pressure": 1027.9,
"windSpeed": 4.68,
"windGust": 4.68,
"windBearing": 357,
"cloudCover": 0.01,
"uvIndex": 0,
"visibility": 10.52,
"ozone": 275.9
},
"hourly": {
"summary": "Clear throughout the day.",
"icon": "clear-day",
"data": [
{
"time": 1575385200,
"summary": "Clear",
"icon": "clear-night",
"precipIntensity": 0,
"precipProbability": 0,
"temperature": 0.83,
"apparentTemperature": -3.74,
"dewPoint": -3.38,
"humidity": 0.73,
"pressure": 1027.7,
"windSpeed": 4.76,
"windGust": 4.76,
"windBearing": 356,
"cloudCover": 0.03,
"uvIndex": 0,
"visibility": 10.356,
"ozone": 275.8
},

```

. . . this goes on like 48 times (for every hour in next two days)  
and just expecting to get

```auto
[ 
   { 
      "time":1575385200,
      "temperature":0.83
   },
   { 
      "time":1575388800,
      "temperature":-0.39
   }
]

```

. . . all 48 instances

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 21:16 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/4 "2019-12-10T21:16:02Z")

</div>

You can use a Change node with a JSONata expression to do this.

Configure the node to set `msg.payload` to the Expression `$.payload.hourly.data.{"time":time,"temperature":temperature}`

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/9/9a2b1f26ef0a0bda5b9590f65edef8fadc8415dd.png)

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 21:28 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/5 "2019-12-10T21:28:42Z")

</div>

OMG, that's great! Thank You! Now I just need to wrap arround my head about what you're writed there.

Is the propper way to publish it like this please?

```auto
[{"id":"df5b9cf2.99f9e","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"d97a19a8.2ae4b8","type":"http request","z":"df5b9cf2.99f9e","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"...../temperature/darksky.json","tls":"","persist":false,"proxy":"","authType":"","x":290,"y":180,"wires":[["208da9c4.5ce226","819aa565.7cafc8"]]},{"id":"9ee799af.2313f8","type":"inject","z":"df5b9cf2.99f9e","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":120,"y":180,"wires":[["d97a19a8.2ae4b8"]]},{"id":"819aa565.7cafc8","type":"debug","z":"df5b9cf2.99f9e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":790,"y":180,"wires":[]},{"id":"8f6d090d.446f98","type":"http in","z":"df5b9cf2.99f9e","name":"","url":"/darksky","method":"get","upload":false,"swaggerDoc":"","x":110,"y":120,"wires":[["d97a19a8.2ae4b8"]]},{"id":"4292168a.29a608","type":"http response","z":"df5b9cf2.99f9e","name":"","statusCode":"","headers":{},"x":710,"y":120,"wires":[]},{"id":"208da9c4.5ce226","type":"change","z":"df5b9cf2.99f9e","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$.payload.hourly.data.{\"time\":time,\"temperature\":temperature}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":220,"wires":[["819aa565.7cafc8","4292168a.29a608"]]}]

```

It complains about "No response object"

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 22:00 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/6 "2019-12-10T22:00:01Z")

</div>

> [@RoSi](#):
>
> It complains about "No response object"

How are you triggering it when you get that error? If you are using the Inject node, then the HTTP Response node will log that error because the message hasn't originated from an HTTP In node for it to reply to.

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 22:02 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/7 "2019-12-10T22:02:09Z")

</div>

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/e/e952d81ed455e1f42555222ae845dd4bb1baad0d.png)

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 22:04 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/8 "2019-12-10T22:04:42Z")

</div>

I saw your flow when I imported it. This is why I asked when are you seeing the No Response Object error message?

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 22:10 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/9 "2019-12-10T22:10:19Z")

</div>

I see, sorry. In debug window. But I've filtered the IP address... So it won't work for you. I trigger manually. Injecting timestamp.

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/6/626603d4ef2cb08e283ac166db0c3ed0760f288d.png)  
Anyway, I should not really worry, because it works 😄

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 22:16 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/10 "2019-12-10T22:16:11Z")

</div>

> [@RoSi](#):
>
> I trigger manually. Injecting timestamp.

So as I said in my reply, you are getting that error because a message is arriving at the HTTP Response node that did not start at the HTTP In node. That is to be expected if you are manually triggering the flow - there is no HTTP request for the HTTP Response node to reply to.

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 22:25 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/11 "2019-12-10T22:25:00Z")

</div>

I understand now. Thank you very much for your help.

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 22:46 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/12 "2019-12-10T22:46:29Z")

</div>

If I may have one more question please, how do I include some data from original JSON but from currently \> temperature? It always turns to disaster if I try myself...

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 22:50 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/13 "2019-12-10T22:50:53Z")

</div>

> [@RoSi](#):
>
> how do I include some data from original JSON but from currently \> temperature

As with your original question, its much easier if you provide an actual example of what you'd want that to look like.

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [10 December 2019 23:04 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/14 "2019-12-10T23:04:10Z")

</div>

Original JSON as it was previousl and new as follows:

```auto
{ 
   "currently":{ 
      "temperature":0.36
   },
   "hourly":[ 
      { 
         "time":1575385200,
         "temperature":0.83
      },
      { 
         "time":1575388800,
         "temperature":-0.39
      }
   ]
} ......

```

So basically I'd like to include current temperature at the beginning of my new JSON.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [10 December 2019 23:08 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/15 "2019-12-10T23:08:49Z")

</div>

Change the expression to:

```auto
{"currently":$$.payload.currently.{"temperature":temperature}, "hourly":[$$.payload.hourly.data.{"time":time,"temperature":temperature}]}

```

---

<div class="post-metadata">

**Author:** ![RoSi](https://avatars.discourse-cdn.com/v4/letter/r/ba9def/32.png) [@RoSi](https://discourse.nodered.org/u/RoSi)\
**Post date:** [13 December 2019 09:17 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/16 "2019-12-13T09:17:59Z")

</div>

I found out, that even more reduce this JSON will be to do it as an JSON array:

```auto
{ 
   "currently":{ 
      "temperature":1.69
   },
   "hourly":[ 
      { 
         "temperature":[ 
            1.67,
            2.98,
            3.45,
            4.42 ...
         ],
         "precipIntensity":[ 
            0.2968,
            0.4584,
            0.6258,
            0.2321 ...
         ]
      }
   ]
}

```

Will it be possible using this method please?

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [15 December 2019 13:16 UTC](https://discourse.nodered.org/t/reduce-large-json-and-publish-over-http-endpoint/19006/17 "2019-12-15T13:16:43Z")

</div>

I think your JSON can be simplified a bit more as you still have `hourly` as an array that contains just one element - it would be better to have `hourly` as the object with the `temperature`/`precipIntesity` properties under it....

```auto
{
  "currently": {
    "temperature": 0.36
  },
  "hourly": {
    "temperature": [
      0.83,
      0.93,
      1.03
    ],
    "precipIntensity": [
      1,
      2,
      3
    ]
  }
}

```

The following expression generates that:

```auto
{"currently":$$.payload.currently.{"temperature":temperature},"hourly":{"temperature": $$.payload.hourly.data.temperature,"precipIntensity":$$.payload.hourly.data.precipIntensity}}

```

If you really want the output as you propose, with `hourly` being a single-element array, then you can use:

```auto
{"currently":$$.payload.currently.{"temperature":temperature},"hourly":[{"temperature": $$.payload.hourly.data.temperature,"precipIntensity":$$.payload.hourly.data.precipIntensity}]}

```