# Csv data into sqlite

**URL:** https://discourse.nodered.org/t/csv-data-into-sqlite/60405
**Category:** General
**Tags:** database
**Created:** [27 March 2022 14:21 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405 "2022-03-27T14:21:58Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [27 March 2022 14:21 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/1 "2022-03-27T14:21:58Z")

</div>

Hi.

I have a device that sends me data in a csv format. How can i move/import the csv data into sqlite. I have set up the database and tables and can read the csv data but cant get it into sql. I dont want to do it manually it must be an automated process every time a file arrives in the directory.

Can someone kindly help.

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [27 March 2022 14:41 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/2 "2022-03-27T14:41:24Z")

</div>

Assuming that you are trying to write the individual values in the csv to fields in the database then have a look at the csv node. You should be able to use that to convert the csv to a javascript object, then you just need to build the SQL query to insert it into the database using the sqlite node.

**[Edit]** Or maybe just use the string directly in a query built, for example, using a Template node (not ui\_template). What does the csv look like and what SQL query are you trying to build?  
Though it is generally safer to use parameters in the query in order to avoid SQL injection.

---

<div class="post-metadata">

### Author: ![jbudd](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jbudd](https://discourse.nodered.org/u/jbudd)
#### Post date: [27 March 2022 17:10 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/3 "2022-03-27T17:10:44Z")

</div>

> [@SplitSecComp](#):
>
> I dont want to do it manually it must be an automated process every time a file arrives in the directory.

A Watch node will trigger a message whenever a new file is created in the watched directory.

> [@Colin](#):
>
> Though it is generally safer to use parameters in the query in order to avoid SQL injection.

Sorry, totally off topic but why is that?  
I _think_ an SQL injection might be something like `" 'Smith'; SELECT secret from secrets"`  
Can you give examples using / not using parameters and how an attempted injection would be affected?  
Thanks 🙂

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [27 March 2022 20:52 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/4 "2022-03-27T20:52:18Z")

</div>

A more critical embedded command might be a DROP TABLE command. When you use parameters the code that expands the parameters checks for SQL injection and stops it. In fact in sqlite I think they are called Prepared Queries. It tells you how to do that in the node's help text.

An example of injection in node-red might be a query generated in a Template node like this.  
`SELECT * from sometable WHERE name = '{{msg.payload}}';`  
If msg.payload came from user input in some way then a malicious user might provide a name like `fred'; DROP TABLE sometable;`  
So the query now reads  
`SELECT * from sometable WHERE name = 'fred'; DROP TABLE sometable; ';`

Using a prepared query prevents the DROP TABLE statement from getting through.

---

<div class="post-metadata">

### Author: ![jbudd](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jbudd](https://discourse.nodered.org/u/jbudd)
#### Post date: [27 March 2022 22:35 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/5 "2022-03-27T22:35:59Z")

</div>

Sorry for the topic hijack @SplitSecComp

OK thanks @Colin !  
So this will drop the table - unsafe

 ![Untitled 3](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/6/e638599a01e5640cde5eff29287eedfa0960cbd5.jpeg)

This will not - safe

 ![Untitled 4](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/d/1d7a6f66b494521f8c61ec7a1433f3f19d78de45.jpeg)

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 06:47 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/6 "2022-03-28T06:47:12Z")

</div>

This is what the csv file look like

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/4/d431c4369a4ad8abc0164260826f5038d5e5c415.png)

This is my database structure

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/c/3/c3c6b9cc3a49ad887bc79f30a83a166cfcc44091.png)  
This is my flow  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/2/5/25a9edbe3c6d0826fbae096fac22d80ddc3182c7.png)  
This is my function node that i am trying to write to the db  
var sql = "";  
var d = new Date();  
var epoch = d.getTime();  
var outputs = ;

sql = "INSERT INTO gpstrack (Time) " +  
"VALUES ("+msg.payload.Time+")";  
outputs.push({topic:sql});

return [outputs];

But cant get i to work 🙂

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 06:49 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/7 "2022-03-28T06:49:29Z")

</div>

I get these errors

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/d/ed6a918139334442d15639b79ac14d5f4ab62fb2.png)

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [28 March 2022 08:01 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/8 "2022-03-28T08:01:13Z")

</div>

The one thing you have not shown us is what is in the message going into your function node. Specifically what is in msg.payload.Time. It should be an integer, if it is a Date object then you should use `msg.payload.Time.getTime()`.

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:06 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/9 "2022-03-28T08:06:25Z")

</div>

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/2/c/2c402f2a088e6aa92b26842018f6d82822f43777.png)

---

<div class="post-metadata">

### Author: ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [28 March 2022 08:08 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/10 "2022-03-28T08:08:15Z")

</div>

You are using standard SQL queries and the date is not surrounded by quotes - that's why it fails. As explained earlier in the thread, you should really use prepared statements to avoid worrying about things like quotations and sequel injection attacks.

Sorry, I've just looked at your table design. You want time stamp not a string time. You will have to convert the time column in your data to an epoch before generating the sql

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:08 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/11 "2022-03-28T08:08:55Z")

</div>

This is what comes out of the Get CSV file node  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/e/4e62abf4c86a5abc4abfeb54479dc87b3a4b6a0d.png)

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:17 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/13 "2022-03-28T08:17:55Z")

</div>

i am new to sql please show me an example what you mean with "prepared staements" 🙂 🙂

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [28 March 2022 08:22 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/14 "2022-03-28T08:22:06Z")

</div>

That does not show us what is in the message going into the function. Feed it into a debug node and check that Time is an integer.

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [28 March 2022 08:23 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/15 "2022-03-28T08:23:17Z")

</div>

I suggest you get what you have going first, then change the sql to use prepared statements. It shows how to do that in the help text for the sqlite node.

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:33 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/16 "2022-03-28T08:33:15Z")

</div>

Here what comes out of the debug node

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/8/488ce8fab0d32aa64202fe6349bffbaf549212f7.png)

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:34 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/17 "2022-03-28T08:34:07Z")

</div>

expanded view

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/1/11cc81891097651572cfc2713f1e073f484314a1.png)

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:34 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/18 "2022-03-28T08:34:56Z")

</div>

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/e/eebf4dfc6fa0a45f71876d006473e7fc116c38db.png)

---

<div class="post-metadata">

### Author: ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [28 March 2022 08:38 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/19 "2022-03-28T08:38:03Z")

</div>

You have not selected First Row contains column names in the csv node, and the Time value is a string. If you want to have it as an integer then you will have to convert it. Alternatively change the column type to string.

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:49 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/20 "2022-03-28T08:49:05Z")

</div>

now i get this error

Error: SQLITE\_ERROR: near ":35": syntax error

---

<div class="post-metadata">

### Author: ![SplitSecComp](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/splitseccomp/32/39802_2.png) [@SplitSecComp](https://discourse.nodered.org/u/SplitSecComp)
#### Post date: [28 March 2022 08:52 UTC](https://discourse.nodered.org/t/csv-data-into-sqlite/60405/21 "2022-03-28T08:52:28Z")

</div>

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/5/e56ed44e75c101248a051e65fbfeb0c35560d36b.png)

[Next page](https://discourse.nodered.org/t/csv-data-into-sqlite/60405.md?page=2)
