with your simple use case... If I inject enough single images eventually it does start spitting out images - so it's buffering things internally (even though -flush_packets is set.) - which indeed may make sense for video in order to try to keep the frame rate constant, but for single frame at a time. You then need to send the end() to push out the final few frames...
I still think it is the system buffering it, not ffmpeg. Also I believe that an mp4 file must be closed before it is valid mp4, I think additional data is written at the start (length and so on) when the file is closed. Therefore you can't do this with a long running process, you will have to close ffmpeg it in order for it to close the file. So I think you would be better to tell ffmpeg to write directly to the file. That assumes I am correct in my thinking.
Hey guys (@dceejay, @Colin),
After wasting some evenings setting up a debugger session for FFmeg, I finally managed to debug FFmpeg (when spawned from my Node-RED flow). And it appears that the images are not cached in NodeJs but in FFmeg.
This is what happens under the hood:
- When running FFmpeg it first reads all the input files.
- Since we are dealing with an input stream (instead of physical input files) it tries to find stream information. For example for an MJpeg stream it needs to know how to read the stream. This not really what we want, since we are dealing with individual images instead of a stream (unless we convert our images to an mpjeg stream ...).
- To get the stream information, it reads as much data as required (i.e. an endless loop that is only interrupted under certain situations: see below).
- So it reads the next image (in data chunks of 32768 bytes) from the input stream. Here the debugger will wait until Node-RED sends an image. As soon as I send an image in Node-RED, FFmpeg will immediately continue here!!
- For EOF signal, the loop will be interrupted.
- And otherwise - since the data chunk doesn't contain the required stream info - the data chunk will be added to a buffer here. And then the endless loop repeats the whole process by waiting for a next image ...
That is the root cause. But now I should find a solution/workaround somehow 
PS. I tried to avoid that FFmpeg searches for stream info - see IF statement at (2) - by adding an extra "find_stream_info" argument:
-f image2pipe -find_stream_info false -i pipe:0 -vf scale=320:240 -f image2pipe pipe:1 -loglevel debug
Then indeed I don't arrive in the endless loop, but the FFmpeg just stops reading the stream...
1 Like
Hi BartButenaers, Can I do the same thing you mentioned in your first suggestion using a raspberry pi camera instead of a IP camera ?
Hello @nassima100,
This had become a rather long thread, but I guess you mean this one (????):
If that is your question, there is no difference if the images (running through your Node-RED flow) are coming from an IP camera or a Raspicam or whatever. But I haven't managed to get that done, especially since the timing of the images in the video files (e.g. mp4) should be correctly. Otherwise the playback rate afterwards would be wrong ...
1 Like
In januari I was completely stuck, so I contacted Kevin Godell. Got immediate help from Kevin, so would like to thank him for his time!!!!!! After some research, he came up with following ffmpeg command:
-f mjpeg -i pipe:0 -vf scale=320:240 -f image2pipe pipe:1
When you use this command in Node-RED's daemon node, it is now possible to pass images via memory (stdin stream pipe:0) to ffmpeg (since we (ab)use an mjpeg stream to be able to process an infinite series of images). And the resized image is returned back to Node-RED as an output message (stdout stream pipe:1):
[{"id":"122804f3.f8b7eb","type":"daemon","z":"50df184.124f2e8","name":"Resize image","command":"/home/pi/FFmpeg/ffmpeg","args":"-f mjpeg -i pipe:0 -vf scale=320:240 -f image2pipe pipe:1","autorun":true,"cr":false,"redo":true,"op":"buffer","closer":"SIGTERM","x":560,"y":320,"wires":[["98ce6e9f.81789"],[],[]]},{"id":"631c9e62.c5bf1","type":"inject","z":"50df184.124f2e8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":340,"wires":[["afda2c1f.ec754"]]},{"id":"dfde7d7e.a8b0b","type":"image","z":"50df184.124f2e8","name":"","width":200,"x":980,"y":300,"wires":[]},{"id":"afda2c1f.ec754","type":"http request","z":"50df184.124f2e8","name":"","method":"GET","ret":"bin","paytoqs":false,"url":"https://static.independent.co.uk/s3fs-public/thumbnails/image/2017/09/12/11/naturo-monkey-selfie.jpg?w968","tls":"","proxy":"","authType":"basic","x":340,"y":340,"wires":[["122804f3.f8b7eb","c886e69d.47aa98"]]},{"id":"93ccc4c3.6f45c8","type":"image","z":"50df184.124f2e8","name":"","width":200,"x":720,"y":400,"wires":[]},{"id":"c886e69d.47aa98","type":"image-info","z":"50df184.124f2e8","name":"","x":530,"y":400,"wires":[["93ccc4c3.6f45c8"]]},{"id":"98ce6e9f.81789","type":"image-info","z":"50df184.124f2e8","name":"","x":790,"y":300,"wires":[["dfde7d7e.a8b0b"]]},{"id":"853bae8d.c380d","type":"comment","z":"50df184.124f2e8","name":"Image processing without spawn but with delay","info":"-f mjpeg -i pipe:0 -vf scale=320:240 -f image2pipe pipe:1","x":660,"y":260,"wires":[]}]
Advantages:
- All data is passed via memory, and no temporary files (or disc IO) is needed.
- A single daemon ffmpeg needs to be spawned that processes all images. I.e. it is not necessary to spawn a new ffmpeg process for every image, like e.g. the ffmpeg-stream package.
Disadvantage: there is a single image delay 
When you insert an image nothing happens. As soon as you inject the second image, the first image will arrive on the output. And so on ... Not 100% sure what the problem is, but after the first image is inserted I get only this output (on stderr and adding -loglevel debug):
[mjpeg @ 0x7fffcd526700] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
This is similar to the issue I had before: since we (ab)use an mjpeg stream, ffmeg wants to collect information about that stream. And I think that causes the delay. But I'm not sure...
Unless somebody else has some usefull tips, I will have to stop with this mechanism 
Would be a pitty, since we are almost there. And it would have been an awesome addition for Node-RED... P.S. I'm not referring to the image resizing, because that is only a simple test that you can also achieve with node-red-contrib-image-tools in pure Javascript ...
Alright thanks sir.
Hey Bart.
Did you ever find a solution to get rid of the "single image delay" problem?
Morning Kevin,
Nice to hear from you again!
Unfortunately not. And I had some other issues with the pipes (e.g. creating mp4 files seemed not to be possible). So at the end I decided to go for a file-based approach, since that is the standard way to use it. And for systems that mostly don't have a hard drive ( like a Raspberry PI 3), I have been experimenting with in-memory filesystems. Those allow you to achieve better results as the pipe-approach (i.e. all ffmpeg commands are possible). You only need enough ram...
I have summarized it into another discussion.
Kind regards,
Bart
That is strange. The other discussion was "automatically closed 60 days after the last reply" and this much older one was still open.
FYI, piping mp4 is possible as long as you pass the correct -movflags. I connect to my rtsp ip cams and use ffmpeg to mux them to mp4 fragments and pass them to media source extension in the browser. I also encode some jpegs to send to browser. No files are ever written to disk.
1 Like
Awhile ago a change to the forum was made so that threads marked solved whould automatically be closed after 60 days. Ones 'not' marked 'solved' still remain open.
1 Like
Hi Kevin, that sounds very interesting! Would it be possible to share some examples of such ffmpeg command lines? I mean from rtsp -> mp4
Best regards, Walter
2 Likes
Interesting about the mp4. I had read somewhere that it was not possible...
And with the jpeg encoding you also have a 1 image delay?
There are some options when it comes to piping an mp4. You can take an RTSP input stream and mux it (without re-encoding) out to mp4. The key to having a pipe-able mp4 will be the -movflags. Some combinations of -movflags include, but not limited to, +frag_keyframe or +empty_moov or +dash or any combination including atleast 1 of those. Without the movflags, ffmpeg will error with muxer does not support non seekable output.
Some actual examples:
ffmpeg -loglevel quiet -progress pipe:1 -hwaccel auto -probesize 32 -analyzeduration 0 -reorder_queue_size 0 -rtsp_transport tcp -i rtsp://192.168.1.5:554/user=admin_password=pass_channel=1_stream=0.sdp -an -c:v copy -f mp4 -movflags +dash+negative_cts_offsets -metadata title="ip 192.168.1.4" pipe:3 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4
ffmpeg -loglevel info -progress pipe:3 -hwaccel auto -analyzeduration 10000000 -probesize 1048576 -re -f hls -i http://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/TearsOfSteel.m3u8 -c:a aac -c:v copy -f mp4 -movflags +dash+negative_cts_offsets pipe:1 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4
You may notice the pipe:3, pipe:4 etc. With nodejs, we are able to create pipes beyond stdin, stdout, stderr.
I dont have a jpeg delay because the jpegs are being encoded from an RTSP source, and thus have a constant input pushing through. From what I remember, you were trying to push in jpegs and have them output in a different size. There must be some magical ffmpeg flags that we have not yet discovered. Perhaps adding -fflags +nobuffer+flush_packets -flags low_delay
Kevin, thank you very much for this!
Kevin, I will try your proposal next weekend...
Hi Kevin, I am trying to test your 2nd command line to display the "TearsOfSteal" out to MP4 video in NR Dashboard (as i usually do with my IPcam RTSP stream) :
ffmpeg -loglevel info -progress pipe:3 -hwaccel auto -analyzeduration 10000000 -probesize 1048576 -re -f hls -i http://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/TearsOfSteel.m3u8 -c:a aac -c:v copy -f mp4 -movflags +dash+negative_cts_offsets pipe:1 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4
This is my test flow :
[{"id":"8c6b37c4.a5aef8","type":"inject","z":"fe2c7bc6.bd49c8","name":"Start stream","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"str","x":110,"y":4910,"wires":[["dc8a0ca9.117e9"]]},{"id":"2de27e17.be3c12","type":"inject","z":"fe2c7bc6.bd49c8","name":"Stop all streams","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"false","payloadType":"str","x":120,"y":4950,"wires":[["dc8a0ca9.117e9"]]},{"id":"40f893c3.a21d1c","type":"image-info","z":"fe2c7bc6.bd49c8","name":"","x":500,"y":4950,"wires":[["21cccabd.e70df6"]]},{"id":"15f19ef2.c099d1","type":"base64","z":"fe2c7bc6.bd49c8","name":"b64","action":"str","property":"payload","x":630,"y":4850,"wires":[["8b9cf7bd.a00318"]]},{"id":"8b9cf7bd.a00318","type":"ui_template","z":"fe2c7bc6.bd49c8","group":"51c96dc0.e62d64","name":"dash oeuf","order":1,"width":6,"height":4,"format":"<img width=\"16\" height=\"360\" src=\"data:image/jpg;base64,{{msg.payload}}\"/>","storeOutMessages":true,"fwdInMessages":true,"templateScope":"local","x":760,"y":4850,"wires":[[]]},{"id":"361bb3b.15d904c","type":"ui_switch","z":"fe2c7bc6.bd49c8","name":"on/off stream","label":"","tooltip":"","group":"51c96dc0.e62d64","order":4,"width":1,"height":1,"passthru":true,"decouple":"false","topic":"","style":"","onvalue":"true","onvalueType":"str","onicon":"","oncolor":"","offvalue":"false","offvalueType":"str","officon":"","offcolor":"","x":110,"y":4930,"wires":[["dc8a0ca9.117e9"]]},{"id":"dc8a0ca9.117e9","type":"switch","z":"fe2c7bc6.bd49c8","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"true","vt":"str"},{"t":"eq","v":"false","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":270,"y":4890,"wires":[["2ca9755e.222aaa"],["a3c294f8.aed6b8"]]},{"id":"a3c294f8.aed6b8","type":"function","z":"fe2c7bc6.bd49c8","name":"stop","func":"msg= [\n {\n kill:'SIGTERM',\n payload : 'SIGTERM'\n }\n \n \n ]; // set a new payload & the counter\nreturn msg;","outputs":1,"noerr":0,"x":270,"y":4930,"wires":[["2ca9755e.222aaa"]]},{"id":"21cccabd.e70df6","type":"msg-speed","z":"fe2c7bc6.bd49c8","name":"","frequency":"sec","estimation":true,"ignore":false,"x":590,"y":4950,"wires":[["794946e8.e0b618","2cbb32f3.899c1e"],[]]},{"id":"2ca9755e.222aaa","type":"exec","z":"fe2c7bc6.bd49c8","command":"ffmpeg -loglevel info -progress pipe:3 -hwaccel auto -analyzeduration 10000000 -probesize 1048576 -re -f hls -i \"http://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/TearsOfSteel.m3u8\" -c:a aac -c:v copy -f mp4 -movflags +dash+negative_cts_offsets pipe:1 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4","addpay":false,"append":"","useSpawn":"true","timer":"","oldrc":false,"name":"Decode RTSP stream","x":440,"y":4890,"wires":[["40f893c3.a21d1c","15f19ef2.c099d1"],["67ffbec1.566cf"],[]],"info":"oeuf: MARCHE PAS \nffmpeg -rtsp_transport tcp -i \"rtsp://admin:271273@192.168.1.6:554/onvif1\" -f image2pipe pipe:1\n\nmisecu : MARCHE OK\nffmpeg -rtsp_transport tcp -i \"rtsp://192.168.1.7:554/user=admin_password=271273_channel=0_stream=1.sdp?real_stream\" -filter:v fps=fps=5 -f image2pipe pipe:1\n\nORIGINAL -> MP4:\n------------------\nffmpeg -loglevel quiet -progress pipe:1 -hwaccel auto -probesize 32 -analyzeduration 0 -reorder_queue_size 0 -rtsp_transport tcp -i rtsp://192.168.1.5:554/user=admin_password=pass_channel=1_stream=0.sdp -an -c:v copy -f mp4 -movflags +dash+negative_cts_offsets -metadata title=\"ip 192.168.1.4\" pipe:3 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4\n ou\nffmpeg -loglevel info -progress pipe:3 -hwaccel auto -analyzeduration 10000000 -probesize 1048576 -re -f hls -i http://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/TearsOfSteel.m3u8 -c:a aac -c:v copy -f mp4 -movflags +dash+negative_cts_offsets pipe:1 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4\n\n\nmisecu -> MP4 : MARCHE PAS \nffmpeg -loglevel quiet -progress pipe:1 -hwaccel auto -probesize 32 -analyzeduration 0 -reorder_queue_size 0 -rtsp_transport tcp -i \"rtsp://192.168.1.7:554/user=admin_password=vEsQc8t4_channel=0_stream=1.sdp?real_stream\" -an -c:v copy -f mp4 -movflags +dash+negative_cts_offsets -metadata title=\"ip 192.168.1.4\" pipe:3 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4\n\n\nffmpeg -loglevel quiet -progress pipe:1 -hwaccel auto -probesize 32 -analyzeduration 0 -reorder_queue_size 0 -rtsp_transport tcp -i rtsp://192.168.1.7:554/user=admin_password=vEsQc8t4_channel=0_stream=1.sdp?real_stream -an -c:v copy -f mp4 -movflags +dash+negative_cts_offsets -metadata title=\"ip 192.168.1.4\" pipe:3 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4\n\nffmpeg -loglevel quiet -progress pipe:1 -hwaccel auto -probesize 32 -analyzeduration 0 -reorder_queue_size 0 -rtsp_transport tcp -i \"rtsp://192.168.1.7:554/user=admin_password=vEsQc8t4_channel=0_stream=1.sdp?real_stream\" -an -c:v copy -f mp4 -movflags +dash+negative_cts_offsets -metadata title=\"ip 192.168.1.54\" pipe:3 -c mjpeg -q 10 -r 7 -vf scale=trunc(iw*0.75/2)*2:-2 -f image2pipe pipe:4\n\n\n\n\n\n"},{"id":"794946e8.e0b618","type":"ui_valuetrail","z":"fe2c7bc6.bd49c8","group":"51c96dc0.e62d64","order":3,"width":3,"height":1,"name":"","label":"","blur":false,"minmax":false,"showvalue":false,"decimals":"0","colorLine":"#ff9900","colorFromTheme":true,"stroke":2,"property":"payload","pointcount":"24","x":800,"y":4950,"wires":[]},{"id":"2cbb32f3.899c1e","type":"ui_text","z":"fe2c7bc6.bd49c8","group":"51c96dc0.e62d64","order":5,"width":1,"height":1,"name":"fps","label":"{{msg.payload}}fps","format":"","layout":"row-center","x":710,"y":4950,"wires":[]},{"id":"67ffbec1.566cf","type":"debug","z":"fe2c7bc6.bd49c8","name":"complet","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":690,"y":4890,"wires":[]},{"id":"51c96dc0.e62d64","type":"ui_group","z":"","name":"camOeuf","tab":"250de740.57ee78","order":9,"disp":true,"width":"6","collapse":true},{"id":"250de740.57ee78","type":"ui_tab","z":"","name":"Dashboard","icon":"dashboard","order":1,"disabled":false,"hidden":false}]
but without success .
Did you see something wrong ?
Best regards
Chris
That command as-is will probably not work. It was just an example of making a streaming mp4 using the movflags. The various data is output to the pipe:N's and have to be handled by some script to receive the raw data.
In response to the yellow highlighted code, I always see the deprecated pixel format warning and it seems to have no negative side affect. Instead of using +dash+negative_cts_offsets, try just using +frag_keyframe. You can get rid of -vf scale=trunc(iw*0.75/2)*2:-2 or change it to -vf scale='trunc(iw*0.75/2)*2:-2' The quotes needed may be different in the context in which it is called.
As of right now, I don't know enough about node-red to understand how you handle the data flow. I read through the docs but i was unable to understand how to get started. Perhaps you can educate me?
If you tell me what you are trying to accomplish, I can probably suggest some appropriate ffmpeg command line to use.
1 Like
Many of us use IP cameras which stream rtsp. To display these flows on the Node Red Dashboard, thanks to (@BartButenaers) Bart and his hard work on this use, to create this flow:
the central element is the "DECODE RTSP STREAM" node which will execute any command like the one I am currently using (thanks Bart again):
ffmpeg -rtsp_transport tcp -i "rtsp://192.168.1.7:554/user=xxxxxxx_password=xxxxxx_channel=0_stream=1.sdp?real_stream" -filter:v fps=fps=5 -f image2pipe pipe:1
we start or stop the rtsp flow thanks to the "buttons" on the left "
The result is a succession of images which will be encoded in Base 64 to be displayed in the dashboard as a video.
It's satisfactory, nothing more. There is a delay of several seconds between the launch of the command and the display of the flow. The image is pixelated compared to the original (with VLC player) in 720p,
and does not allow full HD 1080p under penalty of "buffer saturated" or something like that.
Therefore, a new idea is welcome.
so i tested your order (which can't work like you said)
ok i will
PM me if you want (so as not to pollute this topic) ?
1 Like
