I will see if it is possible ...
I continue here not to pollute the topic of Bart .
I changed the size of max-old-space from 256 to 512, Normally I consume 70% then I go up to 88% during a detection (normal), it goes back down to 86%, goes up 
3h later : 
... In short , I continue to devour RAM without any other action.
@dceejay : Is there a way to winterize tensoflow until a next request? or empty the RAM or something like that?
Not sure. No idea.
It takes much more specific measurements to find out resources eater. And the tools to collect the data should be known and valid. If graphical representation of resources of measured system is part of resources usage the outcome loses quite of meaning. No matter the presenter node is lightweight.
There is not too much solution then. If I deactivate the tfjs-coco-ssd node, the RAM consumption becomes normal ...
Hi all,
So I've been wanting to play with this for some time and finally got around to it.
However, I got the following error when node-red starts (or I re-deploy the TF node - i.e. when it initialises)
(node:20868) UnhandledPromiseRejectionWarning: TypeError: response.arrayBuffer is not a function
at c:\Users\Stephen\.node-red\node_modules\@tensorflow\tfjs-core\src\io\weights_loader.ts:60:61
at Array.map (<anonymous>)
at Object.<anonymous> (c:\Users\Stephen\.node-red\node_modules\@tensorflow\tfjs-core\src\io\weights_loader.ts:60:36)
at step (c:\Users\Stephen\.node-red\node_modules\@tensorflow\tfjs-core\dist\io\weights_loader.js:48:23)
at Object.next (c:\Users\Stephen\.node-red\node_modules\@tensorflow\tfjs-core\dist\io\weights_loader.js:29:53)
at fulfilled (c:\Users\Stephen\.node-red\node_modules\@tensorflow\tfjs-core\dist\io\weights_loader.js:20:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
warning.js:32
(node:20868) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
warning.js:32
(node:20868) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
NOTE: This is a brand new box, everything latest & new (so everything is latest versions and there was nothing in global node_modules before starting - might be a clue?)
After a bit looking around - i found this unlikely looking answer but it actually worked.
@dceejay I'm unsure why others dont get this issue (Is everyone on linux & I'm the only one on windows?)
Windows 10 1909
node -v v12.16.1
npm -v 6.13.4
node-red v1.0.4
Anyhow, as a test, I added the 2 lines suggested to your node in file tfts.js
function TensorFlowCoCo(n) {
var fs = require('fs');
var express = require("express");
var compression = require("compression");
/* suggestion from https://github.com/tensorflow/tfjs/issues/2029 */
const nodeFetch = require('node-fetch'); // <<--- ADD
global.fetch = nodeFetch; // <<--- ADD
/* ************************************************************** */
var tf = require('@tensorflow/tfjs-node');
var cocoSsd = require('@tensorflow-models/coco-ssd');
...
...
Any clues as to why this was happening?
If it fixes it. Happy to accept a PR
Dave, happy to offer Pr but...
It was more the implications of setting global.fetch I dont understand.
I imagine on yours and others systems this doesnt seem to be an issue & I am confused as to why more than anything.
Will have a look tomorrow if I get a chance
1 Like
If managed to "break" a machine here to have that problem (not sure how... 
- and indeed that does fix it. So I'm going to patch it like that...
1 Like
I have during a couple of hours been trying to understand how the code below works. I want to make add the possible to "grab" the completed image with boxes & texts and send it out of the ui_template node. To present it using for instance the "image preview" node or for other usage.
The completed image is nicely presented in the web browser. I have had thoughts in direction to the html canvas element but I do not know, I'm not enough good in html and javascript, just have to realize
Any expert around with some ideas?
<head>
<style>
:root {
--boxcolor: {{msg.boxcolor}};
--boxstroke: {{msg.boxstroke}};
--textcolor: {{msg.textcolor}};
--textfontsize: {{msg.textfontsize}};
--textstroke: {{msg.textstroke}};
}
.imag {
width: 100%;
height: 100%;
}
#svgimage text {
font-family: Arial;
font-size: var(--textfontsize, 18px);
fill : var(--textcolor, yellow);
paint-order: stroke;
stroke: black;/*#ffffff;*//*yellow;*/
stroke-width: var(--textstroke, 3px);/*3px;*/
font-weight: 600;
}
rect {
fill: blue;
fill-opacity: 0;
stroke: var(--boxcolor, yellow);
stroke-width: var(--boxstroke, 1);
}
#svgimage {
/*background-color: #cccccc; Used if the image is unavailable */
background-repeat: no-repeat;
background-size: cover;
}
</style>
</head>
<body>
<svg preserveAspectRatio="xMidYMid meet" id="svgimage" style="width:100%" viewBox="0 0 {{msg.shape[1]}} {{msg.shape[0]}}">
<image class="imag" href="data:image/jpg;base64,{{msg.image}}"/>
</svg>
<!--
<svg preserveAspectRatio="xMidYMid meet" id="svgimage" style="width:100%" viewBox="0 0 {{msg.shape[1]}} {{msg.shape[0]}}">
<image class="imag" href="data:image/jpg;base64,{{msg.pictureBuffer}}"/>
</svg>
-->
<div>{{msg.payload}}</div>
<script>
(function (scope) {
scope.$watch('msg', function (msg) {
if (msg && msg.detect) {
var svg = d3.select("#svgimage");
var box = svg.selectAll("rect").data(msg.detect)
.attr("x", function (d) { return d.bbox[0]; })
.attr("y", function (d) { return d.bbox[1]; })
.attr("width", function (d) { return d.bbox[2]; })
.attr("height", function (d) { return d.bbox[3]; });
box.enter()
.append("rect")
.attr("x", function (d) { return d.bbox[0]; })
.attr("y", function (d) { return d.bbox[1]; })
.attr("width", function (d) { return d.bbox[2]; })
.attr("height", function (d) { return d.bbox[3]; });
box.exit().remove();
var text = svg.selectAll("text").data(msg.detect)
.text(function (d) { return d.class; })
.attr("x", function (d) { return d.bbox[0]; })
.attr("y", function (d) { return 10 + d.bbox[1]; });
text.enter()
.append("text")
.text(function (d) { return d.class; })
.attr("x", function (d) { return d.bbox[0]; })
.attr("y", function (d) { return 10 + d.bbox[1]; });
text.exit().remove();
}
});
})(scope);
</script>
</body>
Not an expert but can tell you that html canvas is indeed the way to go. The Canvas API method .toDataURL will do the trick. I have it partially working for this use case but need to polish the flow.
Hey Walter,
Do I understand correctly that you currently have an svg, and you want to create an image of it? If so, could this be perhaps of any help?
Hello Bart, nice to hear from you!
Exactly, that is the goal, I was diving into the canvas stuff today, many hours available due to the situation, home office you know, but it was too advanced for me, I did understand I had to go that way but how,,,
I have also seen that @Andrei has started looking at this, so that will be exciting to see
My best regards from Sweden,
Walter
1 Like
Hi Bart, nice reference. Having a look on the original stack overflow post there is this note
Note: The only drawback of the method is that it cannot draw images embedded in the
svg. (see demo)
The use case we are handling hits the above constraint since we are drawing shapes in the top of a bitmap image. 
1 Like
I will post the ongoing development, even if there are things missing.
The flow has two parts.
The first one will take as input two pieces of information: the path for the image (from your static file) and an object with the structure that is output by the contrib node tjfs-coco-ssd. It will draw the image in the left dashboard group/widget. Note that I am using a static/fixed object for the testing since I do not have the contrib-tfjs-coco-ssd installed in my computer.
The second flow will respond to the click on the "grab" button. It will find the html canvas in the page and will send to the runtime the image formatted as base64 string. You can store , process or do whatever you want with the image. I added an ui_template to the flow to test this output (so the image will be sent back to the browser).
[{"id":"57a33646.04f798","type":"tab","label":"Flow 4","disabled":false,"info":""},{"id":"f93b8e50.f7681","type":"ui_template","z":"57a33646.04f798","group":"8de750ad.da7f8","name":"","order":4,"width":"1","height":"1","format":" <script>\n (function(scope) {\n scope.$watch('msg', function(msg) {\n if (msg) {\n // Do something when msg arrives\n let dataURL = document.querySelector(\"#canvasImage\").toDataURL('image/jpeg');\n scope.send({payload: dataURL});\n }\n });\n })(scope);\n </script>\n","storeOutMessages":false,"fwdInMessages":true,"templateScope":"local","x":400,"y":260,"wires":[["26649ecb.bcc552"]]},{"id":"579c5bf2.b1ccc4","type":"inject","z":"57a33646.04f798","name":"imag 1","topic":"","payload":"/nri/cars-01-1280-720.jpg","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":"0.5","x":170,"y":120,"wires":[["c2191b60.ab25c8"]]},{"id":"5bad55ea.ba00bc","type":"ui_template","z":"57a33646.04f798","group":"b82c95e7.53a488","name":"","order":2,"width":"7","height":"6","format":"","storeOutMessages":false,"fwdInMessages":true,"templateScope":"local","x":780,"y":320,"wires":[["5a7184ad.26a7ec"]]},{"id":"5a7184ad.26a7ec","type":"debug","z":"57a33646.04f798","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":910,"y":320,"wires":[]},{"id":"82399508.484fd8","type":"template","z":"57a33646.04f798","name":"","field":"template","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"<img width=\"60\" height=\"60\" alt=\"storepic\" src={{{payload}}} />","output":"str","x":640,"y":320,"wires":[["5bad55ea.ba00bc"]]},{"id":"a1012cf2.f1a18","type":"ui_template","z":"57a33646.04f798","group":"df16a965.bce0b8","name":"","order":4,"width":"7","height":"6","format":"<canvas id=\"canvasImage\" width=\"368\" height=\"312\" style = \"border: 3px solid\">\n Your browser does not support the canvas element.\n</canvas>\n\n<script>\n (function(scope) {\n scope.$watch('msg', function (msg) {\n if (msg) {\n // Do something when msg arrives\n function squares(canvasid, squarelist, myimag) {\n let canvas = document.querySelector(canvasid);\n let ctx = canvas.getContext('2d');\n ctx.scale(1, 1);\n let image = new Image;\n image.src = myimag;\n image.onload = function () {\n ctx.drawImage(image, 0, 0, canvas.width, canvas.height);\n //ctx.drawImage(image, 0, 0, 1280, 720);\n for (const square of squarelist) {\n ctx.lineWidth = 3;\n ctx.strokeStyle = 'white';\n ctx.strokeRect(square.bbox[0], square.bbox[1], square.bbox[2], square.bbox[3]);\n //ctx.fillText(square.class,square.bbox[0],square.bbox[1]);\n ctx.font = \"italic 12px Calibri\";\n ctx.fillStyle = 'yellow';\n let width = ctx.measureText(square.class).width;\n ctx.fillRect(square.bbox[0], square.bbox[1] - 12, width, 12);\n ctx.fillStyle = '#000';\n ctx.fillText(square.class, square.bbox[0], square.bbox[1]);\n }\n };\n }\n\n let canvasid1 = \"#canvasImage\";\n let detect = msg.detect;\n let imag1 = msg.image;\n squares(canvasid1, detect, imag1);\n }\n });\n})(scope);\n</script>","storeOutMessages":false,"fwdInMessages":true,"templateScope":"local","x":580,"y":120,"wires":[[]]},{"id":"175e0869.b5e248","type":"ui_button","z":"57a33646.04f798","name":"Grab","group":"b82c95e7.53a488","order":2,"width":"7","height":"1","passthru":false,"label":"Grab","tooltip":"","color":"","bgcolor":"","icon":"","payload":"grab","payloadType":"str","topic":"","x":270,"y":260,"wires":[["f93b8e50.f7681"]]},{"id":"544b23c4.d46dbc","type":"inject","z":"57a33646.04f798","name":"","topic":"","payload":"grab","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":260,"wires":[["175e0869.b5e248"]]},{"id":"26649ecb.bcc552","type":"switch","z":"57a33646.04f798","name":"data:image","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"data:image","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":490,"y":320,"wires":[["82399508.484fd8"]]},{"id":"c2191b60.ab25c8","type":"change","z":"57a33646.04f798","name":"","rules":[{"t":"set","p":"detect","pt":"msg","to":"[{\"bbox\":[5,20,60,50],\"class\":\"people\",\"score\":0.657528936},{\"bbox\":[190,185,55,35],\"class\":\"plate\",\"score\":0.657528936},{\"bbox\":[300,200,16,16],\"class\":\"virus\",\"score\":0.657528936},{\"bbox\":[340,130,30,20],\"class\":\"plate\",\"score\":0.657528936}]","tot":"json"},{"t":"move","p":"payload","pt":"msg","to":"image","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":400,"y":120,"wires":[["a1012cf2.f1a18"]]},{"id":"dac4f67d.973eb8","type":"inject","z":"57a33646.04f798","name":"imag 2","topic":"","payload":"/nri/cars-02-1000-750.jpg","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":"0.5","x":170,"y":180,"wires":[["c2191b60.ab25c8"]]},{"id":"8de750ad.da7f8","type":"ui_group","z":"","name":"D3","tab":"f766b90.18abc48","order":1,"disp":false,"width":"7","collapse":false},{"id":"b82c95e7.53a488","type":"ui_group","z":"","name":"PIC","tab":"f766b90.18abc48","order":3,"disp":false,"width":"7","collapse":false},{"id":"df16a965.bce0b8","type":"ui_group","z":"","name":"SVG","tab":"f766b90.18abc48","order":2,"disp":false,"width":"7","collapse":false},{"id":"f766b90.18abc48","type":"ui_tab","z":"","name":"Libraries","icon":"fa-bank","order":6}]
2 Likes
Nice work, @Andrei , i try it
Very nice @Andrei, you are for sure our great expert!
1 Like
Hi Walter,
I was playing with this the other day & needed the image results to contain the drawn bounding box
also, I needed it at server side / headless (to work without browser / canvas etc)
So using node-red-contrib-image-tools, I simply drew the rects onto the bitmap
Demo...
If its on any use to anyone, let me know & I'll tidy it up for posting.
3 Likes
