Send a selfie picture from a mobile device to node-red server

Dashboard
1

Hi all

I'm quite new to node-red, looking for a solution for the following task to be done, the basic idea would be:

  1. There is a node-red server with a dashboard running on a linux system at home.
  2. Any persons (with access) can send photos from their mobile device out of the dashboard to the server.
  3. The server places all files in a dedicated folder.
  4. Another process on a Raspberry Pi fetches these files from the server and places them on a screen as a slide show. Only the latest (for example 20 images) are displayed (First in, First out).

I'm grateful for suggestions.
The images should be sent via node-red dashboard as other events can also be triggered via dashboard by the users. However, it is not mandatory that the slide show on the raspberry pi be executed by node-red.

BR
Wym

1 Like
2

Hi Wym,
We had a rather similar question recently. Based on that question I started experimenting with my dashboard's camera and I developed a UI node to capture camera images from my dashboard's camera.

That (experimental) UI-node worked, but I had a number of open issues (which is why I haven't shared it yet with the community). My main problem was that I had to tell my browser a number of things:

  • Which resolution I want, from a range of available solutions.
  • Which camera I want to use, from the available camera's (front, back, ...)
  • Which format I want, from a list of available formats (jpeg, png, ...)
  • ...

A user needs to specify all these settings in the node's config screen (which is running at the server side). But suppose 2 dashboards are connected to the server. One phone has 1 camera, and the other phone has 2 camera's:

image
image870Ă—487 176 KB

In the config screen I don't know which clients are connected, so I cannot determine what they offer. At the moment you open the config screen, perhaps even no clients are connected at the moment ... So I don't know what options to show in the config screen, because I don't know the capabilities of the client :woozy_face:

If anybody knows how to solve this problem, please let me know so I can publish a beta version of this experimental node ...

Bart

2 Likes
3

Somehow I'm thinking in the wrong direction.

  • The client-side of this node does most of the work:
    1. It captures images from the camera, based on the user specified settings (front camera, resolution, ...).
    2. It sends those images to the server-side part, i.e. toward the Node-RED flow.
    3. Optionally it can display those images in the dashboard, if required by the user.
  • The server-side of this node has to do very little:
    1. It has to tell to the client (based on the settings in the config screen) whether it has to capture/send images.
    2. It has to create an output message, for all the images it has received from the client.

But somewhere in this scenario the user should be able to specify the camera capture settings (camera, resolution, ...). But if I can't put it on the (server-side) config screen, where should the user do it? It seems to me that the user needs to configure these settings inside the dashboard, instead of in the config screen? I could of course add some widgets to my client-side part, where the user can do the setup (e.g. a dropdown with 'front', 'back' ...).

Anybody has better ideas?

4

Thank you for sharing your ideas!

As mentioned by you, a Java Script must be executed on the client side. This will probably not be possible for security reasons on a mobile device (perhaps Android but for sure not on an apple device). Maybe I'm wrong here, but using the camera from a browser really brings security problems.

Probably one would have to send the photo over another platform like e.g. Twitter or Facebook to the Node-Red server. But that's not user-friendly, I'd like it if someone could trigger such an event directly out of the dashboard.

At the moment the adjustment of the image size and other properties is not so important to me. I'm just trying to understand how this could be done in principle. But this will be clarified in a next step.

I will study your proposal from
Selfie with JS
However, this is an app which has to be installed by the user. In addition, the user must allow the appropriate rights (access to the camera). I find this too complex and complicated for an inexperienced user. On the other hand, an access from the dashboard UI will probably be difficult (or even impossible)...

5

You could easily use something like Telegram with a bot to achieve this. It would be cross-platform too.

@BartButenaers - you might look at using browser storage to keep the browser settings. Could be a cookie or one of the other local storage options.

6

Oh, that is no problem. As soon as I open my dashboard, my browser (both on windows and android) shows a popup : he asks you whether the dashboard is allowed to access your camera. If you 'allow' it, I can capture images without problem.

I would stick to node-red, without making it too complex...

I understand, but if we don't make it cutomizable we will never get a decent solution that everybody in this community can benefit from. But if you need a solution very rapidly, you will need to go that way...

I don't think that is correct?

Damn, no experience with that...
But aside from the storage, I'm still confused in which way the user can access and change the settings...

7

What about dropbox (or similar), it will sync automatically between mobile devices, cloud and node-red. You don't have to open ports to you node-red server that could be a security issue.

Just my 2 cents.

8

@BartButenaers:
Now I'm really interested in your solution. If I understood that correctly, you have written a node which takes a picture from the client side and sends it to the node-red server. Of course without the desired settings.

Basically I agree with all your objections. However, I lack the knowledge about access to the camera resource and JavaScript at all. I come from the C/C++ world. I am really happy when you say that this is not a problem.

I would be very interested in this part of your node. Is there the possibility to have a look at it? I will probably not be a support to you at the beginning. But you never know...

@edje11:
Yes, that would be a good solution. That could be interesting for fetching the files from somewhere.

9

It isn't hard. You either use a cookie or, best for modern browsers, localstorage.

There is a clear API for using localstorage which you should be able to use via a Dashboard template node. Basically, localstorage gives you a key/value store associated with your URL so you can stuff things in it and retrieve them when you like. It is automatically persisted to disk so you don't have to worry about it. It is exactly what you need for persisting browser settings.

That would require everyone using the node to also use Dropbox. That is not a good user experience. Not everyone wants to sign up to cloud services, not everyone wants to or even is permitted to connect their Node-RED or browser to Dropbox.

Perhaps not, but the Node-RED server has to have the Dropbox credentials and a connection to Dropbox both of which could also be security issues for some. Not to mention that many corporate/enterprise firewalls block access to Dropbox and similar services.

2 Likes
10

Hey @wym,
I have refurbished my experimental node a little bit, to make the minimum functionality working. Here is a short demo as a proof of concept, so you can see it is all possible inside Node-RED (without using external tools):

  • You (once) get a popup from your browser, to warn you that the Node-RED dashboard wants to have access to your camera.
  • In the right screen you see the dashboard. I added an option (in the config screen) to hide or show the camera footage in the dashboard.
  • In the left screen you see the flow editor. Every time you click on the 'take picture' inject button, a camera snapshot image will be transferred to the Node-RED flow.
  • A message (containing the image) will be send on the output of my recorder node, and displayed as a preview via the node-red-contrib-image-output node.

Looks like this:

webcam_record
webcam_record857Ă—321 2.81 MB

As you can see in the flow, I also have added start/stop streaming. In those cases the camera images will be send to the Node-RED flow at full speed. However the node-red-contrib-image-output node cannot keep track of this large data flow, so it will eventually die a painfull dead ...

But it is currently really minimum functionality. You cannot specify any settings at all (resolution, ...)!

Will need to do some adjustments before publishing a first demo version on Github ...

P.S. I see now that the video contains some flashes. Perhaps my node cleanup code needs to be adjusted, e.g. that a previous stream is not closed correctly so two video streams are running almost simultaneously ...

2 Likes
11

Note that in the demo I press the inject button to take a snapshot. However if multiple clients are connected, a snapshot will be made on every client. Therefore a button "make snapshot" should be added to the dashboard, which sends a message to the server side flow. And that message should be used as input for the recorder node to ask only that specific client to create a snapshot.

So instead of me adding my own snapshot button inside my client side widget, if we do it that way all logic is nicely inside the flow AND you can choose whatever button you want...

So I need to do something similar like the current notification popups in the dashboard, which you can show only on specific clients.

But now I'm off to work ...

12

This topic interests me also a lot.

I am trying to setup something similar for a party where people can take pictures with their camera and share them somehow so that they become almost immediately projected in a slide show.

The current idea is that they have to email the pictures they take with their camera to a specific email address and within node-red we have a flow reading those emails and extracting the photo attachments from it.

Taking pictures and sharing them in an email can be done with a few clicks, most of the work is getting the email address entered but it would be nice to have a more easy solution.

So I am definitely interested in a dashboard node that allows to take a picture with your phone camera (ideally with option to choose the camera), view the picture taken and (if ok) send the picture to node-red.

And in my use case it should support multiple people doing this at the same time (as this is likely to happen).

13

@BartButenaers
This looks very nice. Since the Christmas holidays are coming now and I would like to take a closer look, it would be great if you could share the node. I would keep it confidential and return any modifications directly to you.
It is your task to release the node in a (beta) version.

BR
wym

14

Hey guys,
The dashboard's Notification-node has a checkbox (on its config screen) to allow a request to be send to one or all clients:

<div class="form-row" id="node-toast-sendtoall">
        <label style="width:auto" for="node-input-sendall"><i class="fa fa-arrow-right"></i> Send to all browser sessions. </label>
        <input type="checkbox" checked id="node-input-sendall" style="display:inline-block; width:auto; vertical-align:baseline;">
</div>

To make my node as generic as possible, I have added the same option to the recorder node last night. This way it works like this e.g. for the "get_snapshot" command:

  • When "unchecked" then the request will be send to only one client. To determine which client needs to be accessed, the client needs to trigger the action! So you add a standard button (with label "upload selfie") on the dashboard.

    1. As soon as the user presses the button, a message arrives in the flow (which contains among others a socketid field).
    2. That message is wired to the input of my node, so I can send the "get_snapshot" command only to that specific client (based on the socketid field from the input message).
    3. My client side widget captures a snapshot image from the webcam, and sends it to the server side flow.
    4. The snapshot image is send as an output message on my node.

  • When "checked" I send the "get_snapshot" command, so (I assume) all clients will return a snapshot image.

However in times of GDPR and privacy violations all over the place, it feels somehow very wrong to allow this kind of stuff. Because now Jan could easily start filming people on the party on their own smartphone, and they don't even realize it (since they simply have allowed the dashboard to to access their webcam at the start of the party, e.g. to create a 'single' snapshot image). So I would like to remove the checkbox again ...

Any thoughts on this one?

15

Based on the following interesting link:

I have created following flow

[{"id":"bc2c036f.64fdd","type":"tab","label":"camera","disabled":false,"info":""},{"id":"8034150c.af4418","type":"http in","z":"bc2c036f.64fdd","name":"","url":"/camera","method":"get","upload":false,"swaggerDoc":"","x":120,"y":100,"wires":[["47244028.f6766"]]},{"id":"d30eb1de.5a647","type":"http response","z":"bc2c036f.64fdd","name":"","statusCode":"","headers":{},"x":810,"y":100,"wires":[]},{"id":"47244028.f6766","type":"template","z":"bc2c036f.64fdd","name":"take/select photo and upload","field":"payload","fieldType":"msg","format":"html","syntax":"plain","template":"<!DOCTYPE html>\n<html>\n<head>\n    <title>Take or select photo</title>\n    <script type=\"text/javascript\">\n    // The  code on this page comes from: \n    //     https://www.codepool.biz/take-a-photo-and-upload-it-on-mobile-phones-with-html5.html\n      function fileSelected() {\n        var count = document.getElementById('fileToUpload').files.length;\n              document.getElementById('details').innerHTML = \"\";\n              for (var index = 0; index < count; index ++)\n              {\n                     var file = document.getElementById('fileToUpload').files[index];\n                     var fileSize = 0;\n                     if (file.size > 1024 * 1024)\n                            fileSize = (Math.round(file.size * 100 / (1024 * 1024)) / 100).toString() + 'MB';\n                     else\n                            fileSize = (Math.round(file.size * 100 / 1024) / 100).toString() + 'KB';\n                     document.getElementById('details').innerHTML += 'Name: ' + file.name + '<br>Size: ' + fileSize + '<br>Type: ' + file.type;\n                     document.getElementById('details').innerHTML += '<p>';\n              }\n      }\n      function uploadFile() {\n        var fd = new FormData();\n              var count = document.getElementById('fileToUpload').files.length;\n              for (var index = 0; index < count; index ++)\n              {\n                     var file = document.getElementById('fileToUpload').files[index];\n                     fd.append(file.name, file);\n              }\n        var xhr = new XMLHttpRequest();\n        xhr.upload.addEventListener(\"progress\", uploadProgress, false);\n        xhr.addEventListener(\"load\", uploadComplete, false);\n        xhr.addEventListener(\"error\", uploadFailed, false);\n        xhr.addEventListener(\"abort\", uploadCanceled, false);\n        xhr.open(\"POST\", \"upload_file\");\n        xhr.send(fd);\n      }\n\n      function uploadProgress(evt) {\n        if (evt.lengthComputable) {\n          var percentComplete = Math.round(evt.loaded * 100 / evt.total);\n          document.getElementById('progress').innerHTML = percentComplete.toString() + '%';\n        }\n        else {\n          document.getElementById('progress').innerHTML = 'unable to compute';\n        }\n      }\n\n      function uploadComplete(evt) {\n        /* This event is raised when the server sends back a response */\n        alert(evt.target.responseText);\n      }\n\n      function uploadFailed(evt) {\n        alert(\"There was an error attempting to upload the file.\");\n      }\n\n      function uploadCanceled(evt) {\n        alert(\"The upload has been canceled by the user or the browser dropped the connection.\");\n      }\n\n    </script>\n</head>\n\n<body>\n  <form id=\"form1\" enctype=\"multipart/form-data\" method=\"post\" action=\"upload_file\">\n    <div>\n      <label for=\"fileToUpload\">Take or select photo: </label><br />\n      <input type=\"file\" name=\"fileToUpload\" id=\"fileToUpload\" onchange=\"fileSelected();\" accept=\"image/*\" capture=\"camera\" />\n    </div>\n    <div id=\"details\"></div>\n    <div>\n      <input type=\"button\" onclick=\"uploadFile()\" value=\"Upload\" />\n    </div>\n    <div id=\"progress\"></div>\n  </form>\n</body>\n</html>","output":"str","x":520,"y":100,"wires":[["d30eb1de.5a647"]]},{"id":"5e233b94.b4dd84","type":"http in","z":"bc2c036f.64fdd","name":"","url":"upload_file","method":"post","upload":true,"swaggerDoc":"","x":140,"y":200,"wires":[["be60433e.90ec1","a6291ff2.65a8e"]]},{"id":"ad665769.14da28","type":"http response","z":"bc2c036f.64fdd","name":"","statusCode":"","headers":{},"x":850,"y":200,"wires":[]},{"id":"8c21a2ff.3d4d7","type":"template","z":"bc2c036f.64fdd","name":"response text","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"Thank you, we have successfully received \"{{payload.filename}}\" (size = {{payload.filesizeKB}}KB )","output":"str","x":680,"y":200,"wires":[["ad665769.14da28"]]},{"id":"be60433e.90ec1","type":"debug","z":"bc2c036f.64fdd","name":"upload_file [post]","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":310,"y":160,"wires":[]},{"id":"a6291ff2.65a8e","type":"change","z":"bc2c036f.64fdd","name":"set filename, filesizeKB","rules":[{"t":"set","p":"payload.filename","pt":"msg","to":"req.files[0].fieldname","tot":"msg"},{"t":"set","p":"payload.filesize","pt":"msg","to":"req.files[0].size","tot":"msg"},{"t":"set","p":"payload.filesizeKB","pt":"msg","to":"$round(payload.filesize/1024)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":200,"wires":[["8c21a2ff.3d4d7"]]},{"id":"27009030.02da4","type":"comment","z":"bc2c036f.64fdd","name":"Documentation","info":"This node is based on https://www.codepool.biz/take-a-photo-and-upload-it-on-mobile-phones-with-html5.html\n\n# Usage\n\n *  On your mobile phone navigate to `<your Node-RED-editor-url>/camera`\n *  Click on `Choose File` button to take a photo.\n *  Click on `Upload` button to send the photo to Node-RED","x":100,"y":40,"wires":[]}]

This flow creates a simple webpage at <your Node-RED-editor-url>/camera which allows you to take a photo and send it to node-red. The URL also works in the browser of your laptop but in that case you won't be able to take a photo, but instead you can select a photo from your hard disk.

I think that this flow also addresses the security issues raised in previous post. So Node-RED doesn't get access to the camera. Only photos that the mobile phone user has explicitly uploaded by pushing the upload button will be send to Node-RED.

2 Likes
16

Hey Jan,
Thanks for the feedback! That is indeed another approach: take the picture with your smartphone itself, and then upload it to the server with user intervention. That might indeed be sufficient for some users, so good that you have shared your flow!

But my personal favor is always to have it competely integrated inside Node-RED.
This is what I achieved yesterday evening:

image
image771Ă—240 80.2 KB

  1. This was my from my first setup, where the server side triggers the snapshot. Since I don't know the client here (i.e. don't know the socketid), I send the request to ALL the clients. So I get a snapshot image from all the dashboards, which is bad for privacy (since nobody knows I am watching their camera). TODO: Bad practice, which I will remove!!!
  2. I show the camera images live in the dashboard (directly from the webcam so no network traffic involved). TODO: Will have to make this adjustable: that you can choose whether you want to show it or not...
  3. User presses a (standard Node-RED dashboard) button to trigger the selfie. But you can choose any other widget you like to trigger it ...
  4. The server side button component receives the click event. It will send an output message containing the socketid of the client where the button has been clicked!
  5. I set msg.payload="take_snapshot" to let my node know what it has to do
  6. My node sends a command to my client side to get a snapshot. I send it only to the dashboard where the button has been clicked (based on the socketid)!!!
  7. My client side gets a snapshot and returns it to the server side of my node.
  8. An output message will be send by my node, containing the snapshot image.

That all seems to be working fine already. To be continued ...

2 Likes
17

Thanks for sharing your design so clearly.

For me one additional requirement is that the people on the party and the Node-RED application are not connected to the same LAN:

  • Node-RED application is running on a raspberry pi at my home

  • people at the party are only connected to the internet using their mobile data.

So somehow the communication between mobile phone and Node-RED needs to happen securely over the internet.

One solution for that is using balena to run Node-RED at my home on a raspberry pi and then enable the public device url. This works but for 3MB photos it can take a while before the photo arrives at Node-RED.

I am just wondering if there are other solutions for this.

18

Well that is a minimum requirement of this node also. To be able to use media devices via Javascript in the browser, Chrome requires a secure connection. And believe me: no ssl = no images or sound! I spend last week a few hours figuring out why my widget didn't work...

Not sure what you mean. You need an SSL connection for the dashboard, as I explained above. This can very easily be setup. Then you arive safely on your Raspberry that has a username/password that everybody on the party needs to know. And you server a very minmalistic dashboard where people can take selfies and send them to your server. Only disadvantage I see, is that everybody needs to specifiy that they trust your self-signed certificate... But perhaps it is better to create a new discussion on Discourse, because I would like to keep this one focussed on media capturing. Otherwise users that are interested in this topic, will get lost in all kind of non-selfie stuff...

19

Thanks, I wasn't aware of the ssl constraint with Chrome. I have tested on safari which hasn't this constraint.

The word internet is important. Node-RED is running on a raspberry pi at my home which is connected to the internet and the party is at another place where the people have a mobile phone. So in that setup, a way needs to be foreseen over the internet to securely access my raspberry pi Node-RED flows.

I think that the "setup" link refers to the wrong page.

Agreed.

20

Set up a Telegram group and get everyone to add pictures to that. You can create a bot that is joined to the group and that will archive the pictures for you.