Send a selfie picture from a mobile device to node-red server

Thank you for sharing your ideas!

As mentioned by you, a Java Script must be executed on the client side. This will probably not be possible for security reasons on a mobile device (perhaps Android but for sure not on an apple device). Maybe I'm wrong here, but using the camera from a browser really brings security problems.

Probably one would have to send the photo over another platform like e.g. Twitter or Facebook to the Node-Red server. But that's not user-friendly, I'd like it if someone could trigger such an event directly out of the dashboard.

At the moment the adjustment of the image size and other properties is not so important to me. I'm just trying to understand how this could be done in principle. But this will be clarified in a next step.

I will study your proposal from
Selfie with JS
However, this is an app which has to be installed by the user. In addition, the user must allow the appropriate rights (access to the camera). I find this too complex and complicated for an inexperienced user. On the other hand, an access from the dashboard UI will probably be difficult (or even impossible)...

You could easily use something like Telegram with a bot to achieve this. It would be cross-platform too.

@BartButenaers - you might look at using browser storage to keep the browser settings. Could be a cookie or one of the other local storage options.

Oh, that is no problem. As soon as I open my dashboard, my browser (both on windows and android) shows a popup : he asks you whether the dashboard is allowed to access your camera. If you 'allow' it, I can capture images without problem.

I would stick to node-red, without making it too complex...

I understand, but if we don't make it cutomizable we will never get a decent solution that everybody in this community can benefit from. But if you need a solution very rapidly, you will need to go that way...

I don't think that is correct?

Damn, no experience with that...
But aside from the storage, I'm still confused in which way the user can access and change the settings...

What about dropbox (or similar), it will sync automatically between mobile devices, cloud and node-red. You don't have to open ports to you node-red server that could be a security issue.

Just my 2 cents.

@BartButenaers:
Now I'm really interested in your solution. If I understood that correctly, you have written a node which takes a picture from the client side and sends it to the node-red server. Of course without the desired settings.

Basically I agree with all your objections. However, I lack the knowledge about access to the camera resource and JavaScript at all. I come from the C/C++ world. I am really happy when you say that this is not a problem.

I would be very interested in this part of your node. Is there the possibility to have a look at it? I will probably not be a support to you at the beginning. But you never know...

@edje11:
Yes, that would be a good solution. That could be interesting for fetching the files from somewhere.

It isn't hard. You either use a cookie or, best for modern browsers, localstorage.

There is a clear API for using localstorage which you should be able to use via a Dashboard template node. Basically, localstorage gives you a key/value store associated with your URL so you can stuff things in it and retrieve them when you like. It is automatically persisted to disk so you don't have to worry about it. It is exactly what you need for persisting browser settings.

That would require everyone using the node to also use Dropbox. That is not a good user experience. Not everyone wants to sign up to cloud services, not everyone wants to or even is permitted to connect their Node-RED or browser to Dropbox.

Perhaps not, but the Node-RED server has to have the Dropbox credentials and a connection to Dropbox both of which could also be security issues for some. Not to mention that many corporate/enterprise firewalls block access to Dropbox and similar services.

2 Likes

Hey @wym,
I have refurbished my experimental node a little bit, to make the minimum functionality working. Here is a short demo as a proof of concept, so you can see it is all possible inside Node-RED (without using external tools):

  • You (once) get a popup from your browser, to warn you that the Node-RED dashboard wants to have access to your camera.
  • In the right screen you see the dashboard. I added an option (in the config screen) to hide or show the camera footage in the dashboard.
  • In the left screen you see the flow editor. Every time you click on the 'take picture' inject button, a camera snapshot image will be transferred to the Node-RED flow.
  • A message (containing the image) will be send on the output of my recorder node, and displayed as a preview via the node-red-contrib-image-output node.

Looks like this:

As you can see in the flow, I also have added start/stop streaming. In those cases the camera images will be send to the Node-RED flow at full speed. However the node-red-contrib-image-output node cannot keep track of this large data flow, so it will eventually die a painfull dead ...

But it is currently really minimum functionality. You cannot specify any settings at all (resolution, ...)!

Will need to do some adjustments before publishing a first demo version on Github ...

P.S. I see now that the video contains some flashes. Perhaps my node cleanup code needs to be adjusted, e.g. that a previous stream is not closed correctly so two video streams are running almost simultaneously ...

2 Likes

Note that in the demo I press the inject button to take a snapshot. However if multiple clients are connected, a snapshot will be made on every client. Therefore a button "make snapshot" should be added to the dashboard, which sends a message to the server side flow. And that message should be used as input for the recorder node to ask only that specific client to create a snapshot.

So instead of me adding my own snapshot button inside my client side widget, if we do it that way all logic is nicely inside the flow AND you can choose whatever button you want...

So I need to do something similar like the current notification popups in the dashboard, which you can show only on specific clients.

But now I'm off to work ...

This topic interests me also a lot.

I am trying to setup something similar for a party where people can take pictures with their camera and share them somehow so that they become almost immediately projected in a slide show.

The current idea is that they have to email the pictures they take with their camera to a specific email address and within node-red we have a flow reading those emails and extracting the photo attachments from it.

Taking pictures and sharing them in an email can be done with a few clicks, most of the work is getting the email address entered but it would be nice to have a more easy solution.

So I am definitely interested in a dashboard node that allows to take a picture with your phone camera (ideally with option to choose the camera), view the picture taken and (if ok) send the picture to node-red.

And in my use case it should support multiple people doing this at the same time (as this is likely to happen).

@BartButenaers
This looks very nice. Since the Christmas holidays are coming now and I would like to take a closer look, it would be great if you could share the node. I would keep it confidential and return any modifications directly to you.
It is your task to release the node in a (beta) version.

BR
wym

Hey guys,
The dashboard's Notification-node has a checkbox (on its config screen) to allow a request to be send to one or all clients:

<div class="form-row" id="node-toast-sendtoall">
        <label style="width:auto" for="node-input-sendall"><i class="fa fa-arrow-right"></i> Send to all browser sessions. </label>
        <input type="checkbox" checked id="node-input-sendall" style="display:inline-block; width:auto; vertical-align:baseline;">
</div>

To make my node as generic as possible, I have added the same option to the recorder node last night. This way it works like this e.g. for the "get_snapshot" command:

  • When "unchecked" then the request will be send to only one client. To determine which client needs to be accessed, the client needs to trigger the action! So you add a standard button (with label "upload selfie") on the dashboard.

    1. As soon as the user presses the button, a message arrives in the flow (which contains among others a socketid field).
    2. That message is wired to the input of my node, so I can send the "get_snapshot" command only to that specific client (based on the socketid field from the input message).
    3. My client side widget captures a snapshot image from the webcam, and sends it to the server side flow.
    4. The snapshot image is send as an output message on my node.
  • When "checked" I send the "get_snapshot" command, so (I assume) all clients will return a snapshot image.

However in times of GDPR and privacy violations all over the place, it feels somehow very wrong to allow this kind of stuff. Because now Jan could easily start filming people on the party on their own smartphone, and they don't even realize it (since they simply have allowed the dashboard to to access their webcam at the start of the party, e.g. to create a 'single' snapshot image). So I would like to remove the checkbox again ...

Any thoughts on this one?

Based on the following interesting link:

I have created following flow

[{"id":"bc2c036f.64fdd","type":"tab","label":"camera","disabled":false,"info":""},{"id":"8034150c.af4418","type":"http in","z":"bc2c036f.64fdd","name":"","url":"/camera","method":"get","upload":false,"swaggerDoc":"","x":120,"y":100,"wires":[["47244028.f6766"]]},{"id":"d30eb1de.5a647","type":"http response","z":"bc2c036f.64fdd","name":"","statusCode":"","headers":{},"x":810,"y":100,"wires":[]},{"id":"47244028.f6766","type":"template","z":"bc2c036f.64fdd","name":"take/select photo and upload","field":"payload","fieldType":"msg","format":"html","syntax":"plain","template":"<!DOCTYPE html>\n<html>\n<head>\n    <title>Take or select photo</title>\n    <script type=\"text/javascript\">\n    // The  code on this page comes from: \n    //     https://www.codepool.biz/take-a-photo-and-upload-it-on-mobile-phones-with-html5.html\n      function fileSelected() {\n        var count = document.getElementById('fileToUpload').files.length;\n              document.getElementById('details').innerHTML = \"\";\n              for (var index = 0; index < count; index ++)\n              {\n                     var file = document.getElementById('fileToUpload').files[index];\n                     var fileSize = 0;\n                     if (file.size > 1024 * 1024)\n                            fileSize = (Math.round(file.size * 100 / (1024 * 1024)) / 100).toString() + 'MB';\n                     else\n                            fileSize = (Math.round(file.size * 100 / 1024) / 100).toString() + 'KB';\n                     document.getElementById('details').innerHTML += 'Name: ' + file.name + '<br>Size: ' + fileSize + '<br>Type: ' + file.type;\n                     document.getElementById('details').innerHTML += '<p>';\n              }\n      }\n      function uploadFile() {\n        var fd = new FormData();\n              var count = document.getElementById('fileToUpload').files.length;\n              for (var index = 0; index < count; index ++)\n              {\n                     var file = document.getElementById('fileToUpload').files[index];\n                     fd.append(file.name, file);\n              }\n        var xhr = new XMLHttpRequest();\n        xhr.upload.addEventListener(\"progress\", uploadProgress, false);\n        xhr.addEventListener(\"load\", uploadComplete, false);\n        xhr.addEventListener(\"error\", uploadFailed, false);\n        xhr.addEventListener(\"abort\", uploadCanceled, false);\n        xhr.open(\"POST\", \"upload_file\");\n        xhr.send(fd);\n      }\n\n      function uploadProgress(evt) {\n        if (evt.lengthComputable) {\n          var percentComplete = Math.round(evt.loaded * 100 / evt.total);\n          document.getElementById('progress').innerHTML = percentComplete.toString() + '%';\n        }\n        else {\n          document.getElementById('progress').innerHTML = 'unable to compute';\n        }\n      }\n\n      function uploadComplete(evt) {\n        /* This event is raised when the server sends back a response */\n        alert(evt.target.responseText);\n      }\n\n      function uploadFailed(evt) {\n        alert(\"There was an error attempting to upload the file.\");\n      }\n\n      function uploadCanceled(evt) {\n        alert(\"The upload has been canceled by the user or the browser dropped the connection.\");\n      }\n\n    </script>\n</head>\n\n<body>\n  <form id=\"form1\" enctype=\"multipart/form-data\" method=\"post\" action=\"upload_file\">\n    <div>\n      <label for=\"fileToUpload\">Take or select photo: </label><br />\n      <input type=\"file\" name=\"fileToUpload\" id=\"fileToUpload\" onchange=\"fileSelected();\" accept=\"image/*\" capture=\"camera\" />\n    </div>\n    <div id=\"details\"></div>\n    <div>\n      <input type=\"button\" onclick=\"uploadFile()\" value=\"Upload\" />\n    </div>\n    <div id=\"progress\"></div>\n  </form>\n</body>\n</html>","output":"str","x":520,"y":100,"wires":[["d30eb1de.5a647"]]},{"id":"5e233b94.b4dd84","type":"http in","z":"bc2c036f.64fdd","name":"","url":"upload_file","method":"post","upload":true,"swaggerDoc":"","x":140,"y":200,"wires":[["be60433e.90ec1","a6291ff2.65a8e"]]},{"id":"ad665769.14da28","type":"http response","z":"bc2c036f.64fdd","name":"","statusCode":"","headers":{},"x":850,"y":200,"wires":[]},{"id":"8c21a2ff.3d4d7","type":"template","z":"bc2c036f.64fdd","name":"response text","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"Thank you, we have successfully received \"{{payload.filename}}\" (size = {{payload.filesizeKB}}KB )","output":"str","x":680,"y":200,"wires":[["ad665769.14da28"]]},{"id":"be60433e.90ec1","type":"debug","z":"bc2c036f.64fdd","name":"upload_file [post]","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":310,"y":160,"wires":[]},{"id":"a6291ff2.65a8e","type":"change","z":"bc2c036f.64fdd","name":"set filename, filesizeKB","rules":[{"t":"set","p":"payload.filename","pt":"msg","to":"req.files[0].fieldname","tot":"msg"},{"t":"set","p":"payload.filesize","pt":"msg","to":"req.files[0].size","tot":"msg"},{"t":"set","p":"payload.filesizeKB","pt":"msg","to":"$round(payload.filesize/1024)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":200,"wires":[["8c21a2ff.3d4d7"]]},{"id":"27009030.02da4","type":"comment","z":"bc2c036f.64fdd","name":"Documentation","info":"This node is based on https://www.codepool.biz/take-a-photo-and-upload-it-on-mobile-phones-with-html5.html\n\n# Usage\n\n *  On your mobile phone navigate to `<your Node-RED-editor-url>/camera`\n *  Click on `Choose File` button to take a photo.\n *  Click on `Upload` button to send the photo to Node-RED","x":100,"y":40,"wires":[]}]

This flow creates a simple webpage at <your Node-RED-editor-url>/camera which allows you to take a photo and send it to node-red. The URL also works in the browser of your laptop but in that case you won't be able to take a photo, but instead you can select a photo from your hard disk.

I think that this flow also addresses the security issues raised in previous post. So Node-RED doesn't get access to the camera. Only photos that the mobile phone user has explicitly uploaded by pushing the upload button will be send to Node-RED.

1 Like

Hey Jan,
Thanks for the feedback! That is indeed another approach: take the picture with your smartphone itself, and then upload it to the server with user intervention. That might indeed be sufficient for some users, so good that you have shared your flow!

But my personal favor is always to have it competely integrated inside Node-RED.
This is what I achieved yesterday evening:

  1. This was my from my first setup, where the server side triggers the snapshot. Since I don't know the client here (i.e. don't know the socketid), I send the request to ALL the clients. So I get a snapshot image from all the dashboards, which is bad for privacy (since nobody knows I am watching their camera). TODO: Bad practice, which I will remove!!!
  2. I show the camera images live in the dashboard (directly from the webcam so no network traffic involved). TODO: Will have to make this adjustable: that you can choose whether you want to show it or not...
  3. User presses a (standard Node-RED dashboard) button to trigger the selfie. But you can choose any other widget you like to trigger it ...
  4. The server side button component receives the click event. It will send an output message containing the socketid of the client where the button has been clicked!
  5. I set msg.payload="take_snapshot" to let my node know what it has to do
  6. My node sends a command to my client side to get a snapshot. I send it only to the dashboard where the button has been clicked (based on the socketid)!!!
  7. My client side gets a snapshot and returns it to the server side of my node.
  8. An output message will be send by my node, containing the snapshot image.

That all seems to be working fine already. To be continued ...

2 Likes

Thanks for sharing your design so clearly.

For me one additional requirement is that the people on the party and the Node-RED application are not connected to the same LAN:

  • Node-RED application is running on a raspberry pi at my home

  • people at the party are only connected to the internet using their mobile data.

So somehow the communication between mobile phone and Node-RED needs to happen securely over the internet.

One solution for that is using balena to run Node-RED at my home on a raspberry pi and then enable the public device url. This works but for 3MB photos it can take a while before the photo arrives at Node-RED.

I am just wondering if there are other solutions for this.

Well that is a minimum requirement of this node also. To be able to use media devices via Javascript in the browser, Chrome requires a secure connection. And believe me: no ssl = no images or sound! I spend last week a few hours figuring out why my widget didn't work...

Not sure what you mean. You need an SSL connection for the dashboard, as I explained above. This can very easily be setup. Then you arive safely on your Raspberry that has a username/password that everybody on the party needs to know. And you server a very minmalistic dashboard where people can take selfies and send them to your server. Only disadvantage I see, is that everybody needs to specifiy that they trust your self-signed certificate... But perhaps it is better to create a new discussion on Discourse, because I would like to keep this one focussed on media capturing. Otherwise users that are interested in this topic, will get lost in all kind of non-selfie stuff...

Thanks, I wasn't aware of the ssl constraint with Chrome. I have tested on safari which hasn't this constraint.

The word internet is important. Node-RED is running on a raspberry pi at my home which is connected to the internet and the party is at another place where the people have a mobile phone. So in that setup, a way needs to be foreseen over the internet to securely access my raspberry pi Node-RED flows.

I think that the "setup" link refers to the wrong page.

Agreed.

Set up a Telegram group and get everyone to add pictures to that. You can create a bot that is joined to the group and that will archive the pictures for you.

I guess this means that all people have to install the Telegram app on their mobile phone. I am looking for a solution where they don't need to install any specific apps on their mobile phone.

I have created new discussion for this: see How to communicate with Node-RED server on the edge using a mobile phone via the internet?

Yes, it is true that people would have to install an app. On the other hand, it takes care of all of the security for you and also takes care of the phone interaction and the camera.

The problem with a web interface is that you have to securely configure your Node-RED server for the Internet and then think about who might have access to the web site you are creating. How do you make sure that party-goers images are safe and only shared to other party attendees without things leaking everywhere.

Don't forget that you are effectively committing everyone at the party to having their images shared - even if they don't take any, others may take photo's of them.

You, as the owner of the server, are also taking on a commitment to both keep the data safe and to respond to the wishes of anyone at the party.

Will you password-protect access to the site? In which case, users have to be given the site name, enter that into their phones and the password which they also have to enter.

Using Telegram could actually be easier for people in that case.