How big is too big for a flow?

General
1

I have a project that has grown quite large and I tried to make a backup by exporting each flow to a text file that I could import from backup if needed. But it won't. The text file of my main flow is 62Kb and it won't import to another RPI. (I was just testing the backup).

Should I just take the time to break it into more, smaller flows? Is there a better way to save my flows?

2

Exactly how are you writing to file and importing it again?

3

Urm, that isn't big. Even my test flow is 157k. I'm away from the house at the moment so I can't check the live flows.

4

To back up my entire NR program (19 tabs, around 400 nodes) I simply copy all the files in /home/pi/.node-red/ (certainly the code file {flows_"xxxxx".json} where "xxxxx" is your system name (IE: "raspberry" maybe)) to a back-up directory (actually on my NAS, well and my PC too). To capture any persistent variables you store to file (IE: context.global.xxxx) also copy the "/context/" directory and its' files.

To restore simply copy all those files back to where you copied them from (into "/home/pi/.node-red/" directory) and restart NR.

st

5

I've always exported the flow to the clipboard then saved it in a text file (and later import from clipboard). It works on all smaller flows, which is why I am wondering if there is a size limit that I have overrun.

6

How would you simply back up individual flows?

7

It what way did it not work? Please describe it in more detail so we have some idea of what you're hitting.

8

On the big flow, nothing imported. Not a single wire or node. The smaller flows imported OK, however.

9

My 2 live flows are currently 621k (on a Pi2 with lots of other things running) and 387k (Pi3 also with lots running).

10

915kb here and it runs smoothly.

11

Yes, I do the same. Worked all the times when I tried to restore. Don't export, just copy.
In addition, in all my "production" Pi's, I have a small mini sd card reader with an additional sd card. For smaller changes I copy the .node-red folder, after larger modifications, I run the sd card copy utility, gives a full working backup. Works perfect, I have tested to swap cards, just works as expected

(My largest flow is around 450k, running in a Pi 3)

12

Hey Steve.

Yeah, "been there" myself.

As has been mentioned it is better to not get too worried. It can be done.

I have a little script I use:
(Below)

You may need to edit some parts for paths, etc.

But then if you want to get anything back, you simple copy from the folder to your node-red folder.

Hope this helps.

#!/bin/bash
# ---------------------------------------
# Simple backup script v1.0
# ---------------------------------------

# Variables
myDate=`date "+%Y-%m-%d.%H.%M.%S"`
#backupFolderName="Backup_$myDate"
backupFolderName="$myDate"
backupSource="/home/pi/.node-red"
backupDest="/home/pi/Backups/NR"
backupFilter="*.j*"
backupExclude="lost\+found"

# Tell the user what we're working with
echo "The myDate variable contains: $myDate"
echo "A backup of $backupSource/$backupFilter will be made and stored in $backupDest/$backupFolderName"

# Begin backup
rsync -avz --progress $backupSource/$backupFilter --exclude=$backupExclude $backupDest/$backupFolderName

# We're done.
echo "Done!"
#return $?
13

Can you be more explicit what you mean?

Do you mean:

  1. open the `Import Flows' dialog
  2. paste in your flow
  3. click import
  4. nothing happens

If so, can you check the browser's javascript developer console for any error messages?

14

This is getting interesting.
By the way, this is a very low priority- the flows did import after all- but in new flows. And I think I know where I went wrong and assumed that NR wasn't importing the flows.

This project is over and not a high priority, I just wanted to save the flows for possible reuse next year. (NR was controlling an Easter-themed game and included RFID card readers, Alexa [for the voice], electromagnets and other challenges you would see in an Escape Room.)

My steps to import the saved flows were to create a new flow (Flow 3) then import from clipboard. clicked on "Current Flow" then Import::

NR%20import%20steps
NR import steps.jpg763×509 45.3 KB

When I clicked on Import, I saw something new that I did not see before because I probably wasn't watching the screen:
NR%20import%20error

And this was the result:
nr%20import%20step2
So you can see why I thought nothing imported.

After a few more attempts, I decided to try something else and clicked on the right-scroll of the flows tab, and I discovered that the flows did import after all. In new flows.:
nr%20import%20step3

So, in conclusion...
Did the flow import into a new flow tab because I originally copied every node on a named flow?

15

Thanks for the script. I have another RPI that is running my personal wiki that I use to keep notes. (Sort of like a lab notebook, but with my memory, I can't keep track of what I did last month). I back up the Wiki database manually once in a while.
I mount a thumb drive when the RPI starts up. All my database backups go there and not on the source SD card.
But your script looks better and I will put it on my RPI with NodeRed.

Can I run a shell script from NR Dashboard? I would love to put a button labeled "Back-up" there.

16

think .... exec node....
but you are getting into self-licking lollipop territory here... as your flow now has to include the flow to back itself up... and you will probably want to use that on the next project also... etc... this is one of those things that's probably easier to run outside of Node-RED whenever you want.

17

I hadn't heard that term in a while, but I grok.