# Bulk serial port data and sending to a database

**URL:** <https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555>\
**Category:** General\
**Created:** [8 November 2020 16:49 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555 "2020-11-08T16:49:17Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 16:49 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/1 "2020-11-08T16:49:18Z")

</div>

Hello,  
I receive from a serial port a burst of data that obviously always ends with a line break.  
If I send this data to a file it looks like this.

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/a/1/a16d380508c63cd2817cbb115d23eeb5ba9c9785.png)

On the other hand I try to store each value visibly separated by a space in a database and stop when the linebreak character arrives.  
Then start again with the next dataset that will arrive, send this again in the database but obvious to the next record.  
Each value will obviously be written in a specific variable and then I will push these variables in the ad-oc fields of the database.  
Do you have an idea how to proceed?  
I hope I have been as clear as possible 🤔  
Thank you for your help.  
/JL

Here's what comes out raw and bursting from the serial node with the debug node

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/5/45f681f1f89a74ed3ea8c66b2416045a85abf084.png)

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [8 November 2020 17:09 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/2 "2020-11-08T17:09:49Z")

</div>

How have you configured the serial port, have you tried telling it to split on new line?

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:13 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/3 "2020-11-08T17:13:24Z")

</div>

Here is the port setting

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/b/dbcbf532461b6ad628cb25de1db047fdbb0cce76.png)

---

<div class="post-metadata">

**Author:** ![fogmajor](https://avatars.discourse-cdn.com/v4/letter/f/bbce88/32.png) [@fogmajor](https://discourse.nodered.org/u/fogmajor)\
**Post date:** [8 November 2020 17:18 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/4 "2020-11-08T17:18:38Z")

</div>

Hi Jean Luc  
Just a couple of questions. Is the number of the data values and format constant or can it vary? and what type of database are you using?

Regards  
Andrew

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:18 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/5 "2020-11-08T17:18:51Z")

</div>

Okay,  
I changed a bit the entrance to wait 1000 ms and now it's better indeed

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/3/93c74ff4174c90d6798e2fb87a760d7f1a576f5e.png)

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/2/1/21c426ca60ab1e070cf0f9bfa959ed4abfaf18a5.png)

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:22 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/6 "2020-11-08T17:22:24Z")

</div>

> [@fogmajor](#):
>
> Just a couple of questions. Is the number of the data values and format constant or can it vary? and what type of database are you using?

I don't know exactly yet because I just had my first conversation with my hybrid inverter today.  
I have the impression that the data will be the same length for each type of question.  
One question, answer always the same number of fields.  
I will use influxDB and Grafana.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [8 November 2020 17:42 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/7 "2020-11-08T17:42:11Z")

</div>

If each line ends in a newline then would it not be better to split on that? It would be `\n` or `\r`

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:51 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/8 "2020-11-08T17:51:09Z")

</div>

> [@Colin](#):
>
> \r

okay, changed with:  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/e/ee4b30c33342649d95fe5a124d26dd0ef28960c3.png)  
i receive this, it's better than waiting !

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/c/1c95e5b6db881f4bcbda2b9591c90a9ac5476337.png)  
I feel i'm now going to use the String Node to try splitting into an array, like that  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/3/93719a5a9b21eba9abe5cfd2be3dd7535e7c21a1.png)  
But I don't know at the moment how to fill this node to tell it to cut on a separator which is a character which is a space ?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [8 November 2020 17:55 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/9 "2020-11-08T17:55:08Z")

</div>

I would do that by writing some javascript in a Function node. First I would check that the first character is a `(` and discard the message if not as that means something has got out of sync. Then I would (in javascript) split the rest on the space character to give an array, then assign the values to attributes of the payload.

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:56 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/10 "2020-11-08T17:56:27Z")

</div>

Here it's works. Thanks to all.  
Quick response and solution. It's cool!

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/d/4d5f5d3787b99497e45be14b81f741d1be9a2610.png)

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [8 November 2020 17:58 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/11 "2020-11-08T17:58:26Z")

</div>

If you know what each column is then you could use a csv node set to split on a space rather than a comma. That would then assign each value by a named property

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 17:59 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/12 "2020-11-08T17:59:47Z")

</div>

> [@Colin](#):
>
> I would do that by writing some javascript in a Function node. First I would check that the first character is a `(` and discard the message if not as that means something has got out of sync. Then I would (in javascript) split the rest on the space character to give an array, then assign the values to attributes of the payload.

Thank you Colin,  
I think I have my solution with this string knot.

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [8 November 2020 18:00 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/13 "2020-11-08T18:00:17Z")

</div>

I will be able to create beautiful graphs on the photovoltaic production as well as control certain relays and other alarms. Node red & Raspberry is too cool for that.  
But... still... it should be sunny 😉

---

<div class="post-metadata">

**Author:** ![Jean-Luc](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jean-luc/32/89996_2.png) [@Jean-Luc](https://discourse.nodered.org/u/Jean-Luc)\
**Post date:** [11 November 2020 18:12 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/14 "2020-11-11T18:12:34Z")

</div>

All goes well !  
I was able to store the inverter values in my database and start making graphs,  
**BUT** for some unknown reason, sometimes the communication on the serial port **gets blocked.**  
I have the feeling that the serial port is not responding anymore.  
In order to make it work again **I'll reboot the Pi and then it's OK**.  
I saw that it is possible to close and reopen the **ttyUSB0** port in Linux.  
On this one my inverter is seen via the **cp210x** chip.

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/f/df2a5bac33955151be887441b60a01ae6161123d.png)

If you have an idea how to fix this big flaw, it will be cool. 💫

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [10 January 2021 18:12 UTC](https://discourse.nodered.org/t/bulk-serial-port-data-and-sending-to-a-database/35555/15 "2021-01-10T18:12:36Z")

</div>

Closed on 10 January 2021 18:12 UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.