Clarifai in Node Red

General
1

Hi folks,

I'm new to Node Red but I'm loving it so far. 1 week in and I'm already building flows that I never imagined I'd be able to achieve with my skill levels. It really is an amazing tool!

My question is around the integration of Clarifai in my flows. I would like to do some object recognition but since I've got Node Red running on a Pi3, I thought it would be faster and more efficient to use a free cloud solution. Clarifai seem to have one of the best free tiers as it allows 1000 hits a month and allows you to train your own engine.

My problem is that there isn't a Node Red node for Clarifai which I can see in the in the install tab (it seems like there was one 5 years ago, but I guess that has been removed).

There are some CURL examples on the Clarifai site to use an HTTP PUT and then gather the results, but I'm not smart enough to convert it to the Node Red way of doing it like Steve-Mcl recommends here

I would be really grateful if someone could give me some advice or pointers as to how I can send a photo to the Clarifai API from within Node Red and then get the JASON file back so I cam deal with the response.

Thanks muchly!

2

Here is the curl command converted to a http request. Untested as i do not have a Clarifai account.

[{"id":"79b1979c.9de228","type":"function","z":"9b3f9f31.c45298","name":"","func":"msg.method = \"POST\";\nmsg.headers[\"Authorization\"] = 'Key YOUR_API_KEY';\nmsg.headers[\"Content-Type\"] = \"application/json\";\nmsg.payload.image = {\n    \"url\": \"https://samples.clarifai.com/metro-north.jpg\"\n}\nmsg.url =\"https://api.clarifai.com/v2/models/{THE_MODEL_ID}/versions/{THE_MODEL_VERSION_ID}/outputs\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":310,"y":1320,"wires":[["233bb9c0.96ff4e"]]},{"id":"233bb9c0.96ff4e","type":"http request","z":"9b3f9f31.c45298","name":"","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":500,"y":1320,"wires":[["29dbcf2d.703528"]]},{"id":"29dbcf2d.703528","type":"debug","z":"9b3f9f31.c45298","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":660,"y":1340,"wires":[]},{"id":"f7625296.59877","type":"inject","z":"9b3f9f31.c45298","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":1320,"wires":[["79b1979c.9de228"]]}]
3

Hi,

Thank you so much for your quick and very detailed reply.

Unfortunately, I am getting a strange error in the Function node when I have added MY_API_KEY which says "TypeError: Cannot set property 'Authorization' of undefined".

Do you have any idea why it would say that and how I can resolve it?

Thanks again for your time and expertise!

4

Sorry i never set the headers to an object. Use this function node.

[{"id":"79b1979c.9de228","type":"function","z":"9b3f9f31.c45298","name":"","func":"msg.headers = {};\nmsg.payload ={};\nmsg.method = \"POST\";\nmsg.headers[\"Authorization\"] = 'Key YOUR_API_KEY';\nmsg.headers[\"Content-Type\"] = \"application/json\";\nmsg.payload.image = {\n    \"url\": \"https://samples.clarifai.com/metro-north.jpg\"\n}\nmsg.url =\"https://api.clarifai.com/v2/models/{THE_MODEL_ID}/versions/{THE_MODEL_VERSION_ID}/outputs\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":310,"y":1320,"wires":[["29dbcf2d.703528"]]}]
1 Like
5

That's super, thanks!

It seems to be posting to the website now and I'm getting a response. The response is "Malformed or invalid request", but I guess that's something picky about how the site needs the request formatted.

I'll have a fiddle with it to see if I can sort it and post here if I manage to get it fixed.

Thanks again for your fast and helpful replies. I really appreciate it!

6

Me again... I think I figured it out.

Instead of

msg.payload.image = {
    "url": "https://samples.clarifai.com/metro-north.jpg"
}

I had to put

msg.payload = {
      "inputs": [
        {
          "data": {
            "image": {
              "url": "https://samples.clarifai.com/metro-north.jpg"
            }
          }
        }
      ]
    }

Seems to be giving the correct response now.

Thanks again for all your help.

7

Hi,

Sorry to bother you again, but after the success with the URL lookup, I have been trying to get the base64 upload to work and I'm baffled.

The CURL code looks like this :
// Larger Files (Greater than 195 KB)

curl -X POST \
  -H "Authorization: Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- https://api.clarifai.com/v2/models/{model-id}/outputs << FILEIN
  {
    "inputs": [
      {
        "data": {
          "image": {
            "base64": "$(base64 /home/user/image.png)"
          }
        }
      }
    ]
  }
FILEIN

I've tried replacing the URL line with the base64 line in the newly working node, but that doesn't seem to work. I've checked that my file is in the right place and it correctly encoded. I really haven't a clue how to get this one working.

Have you got any thoughts?

Thanks.

8

This section will need to be a base64 encode string
Try something like this to convert image, you will need node-red-node-base64 (node) - Node-RED

[{"id":"c9bb246f.fb3f28","type":"inject","z":"9b3f9f31.c45298","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":90,"y":1880,"wires":[["3a9b5695.9ecaa2"]]},{"id":"3a9b5695.9ecaa2","type":"file in","z":"9b3f9f31.c45298","name":"","filename":"/home/user/image.png","format":"","chunk":false,"sendError":false,"encoding":"none","x":330,"y":1880,"wires":[["b5918e5e.607be"]]},{"id":"b5918e5e.607be","type":"base64","z":"9b3f9f31.c45298","name":"","action":"","property":"payload","x":480,"y":1880,"wires":[["3bbefd5a.6e1a22"]]},{"id":"3bbefd5a.6e1a22","type":"debug","z":"9b3f9f31.c45298","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":650,"y":1880,"wires":[]}]

[edit @LetsDoIt added example using nodes]

9

Hi,

Sorry, I already figured that bit out and the file is converted to base64 already so I don't think that is the issue.

Do you have any other thoughts? I'm not sure if it has something to do with the FILEIN command in the curl code, but even after Googling I can't see how this works differently.

Thanks!

10

Have you added the base64 text from the file into the message?

e.g.

//get base64 text from payload before setting up msg for http request
var b64 = msg.payload; 
//prepare msg for http request
msg.headers = {};
msg.payload = {};
msg.method = "POST";
msg.headers["Authorization"] = 'Key YOUR_API_KEY';
msg.headers["Content-Type"] = "application/json";
//add b64 text to payload object
msg.payload = {
    "inputs": [
        {
            "data": {
                "image": {
                    "base64": b64
                }
            }
        }
    ]
}
//set the URL
msg.url ="FILL ME IN";
return msg;

demo

[{"id":"c9bb246f.fb3f28","type":"inject","z":"553814a2.1248ec","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":720,"y":2200,"wires":[["3a9b5695.9ecaa2"]]},{"id":"3a9b5695.9ecaa2","type":"file in","z":"553814a2.1248ec","name":"","filename":"/home/user/image.png","format":"","chunk":false,"sendError":false,"encoding":"none","x":920,"y":2200,"wires":[["b5918e5e.607be"]]},{"id":"b5918e5e.607be","type":"base64","z":"553814a2.1248ec","name":"","action":"","property":"payload","x":1110,"y":2200,"wires":[["79b1979c.9de228"]]},{"id":"3bbefd5a.6e1a22","type":"debug","z":"553814a2.1248ec","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1690,"y":2220,"wires":[]},{"id":"79b1979c.9de228","type":"function","z":"553814a2.1248ec","name":"","func":"var b64 = msg.payload;\n\nmsg.headers = {};\nmsg.payload = {};\nmsg.method = \"POST\";\nmsg.headers[\"Authorization\"] = 'Key YOUR_API_KEY';\nmsg.headers[\"Content-Type\"] = \"application/json\";\n\nmsg.payload = {\n    \"inputs\": [\n        {\n            \"data\": {\n                \"image\": {\n                    \"base64\": b64\n                }\n            }\n        }\n    ]\n}\n\nmsg.url =\"FILL ME IN\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1280,"y":2220,"wires":[["61541f58.647f5"]]},{"id":"61541f58.647f5","type":"function","z":"553814a2.1248ec","name":"http request node here","func":"\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1480,"y":2220,"wires":[["3bbefd5a.6e1a22"]]},{"id":"7ec877b9.f74888","type":"file in","z":"553814a2.1248ec","name":"","filename":"/home/user/base64_image.txt","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":950,"y":2240,"wires":[["79b1979c.9de228"]]},{"id":"dfba35ab.0ac328","type":"inject","z":"553814a2.1248ec","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":720,"y":2240,"wires":[["7ec877b9.f74888"]]}]
2 Likes
11

Wow, that worked a treat!! :smiley:

Thank you both for your fast responses and your detailed replies. My flow is looking fantastic now. I'm taking live snapshots from a web camera and Clarifai is identifying the subject.

My son and I are building a system which will identify the birds on our bird feeder and will show a counter and photos of each of the different types. He will be delighted that you have both got it working for us!

Thanks again.

12

Glad it is stored.

For the community, would you be kind enough to share the final code that uses Clarifai service?
Perhaps a little how to?

13

As i said you will have to replace the $(base64 /home/user/image.png) with the base64 encoded string.

[edit I see Steve gave you the same answer]

1 Like
14

Sure, here is the code I used with help from you and E1Cid.

I have a trigger connected the incoming USB camera stream which sends the buffer into a base64 conversion. That goes into the code which adds it into the right place and is passed into the HTML request node. I have a debug for the whole object and the specific variable which tells you the best guess at what is in the photo.

I really hope someone else will find this useful.


[
    {
        "id": "a8dbc0c2.d3946",
        "type": "usbcamera",
        "z": "197403af.63076c",
        "filemode": "0",
        "filename": "capture.jpg",
        "filedefpath": "1",
        "filepath": "/home/pi/Pictures",
        "fileformat": "jpeg",
        "resolution": "2",
        "name": "",
        "x": 150,
        "y": 140,
        "wires": [
            [
                "2c052f51.71dc2",
                "90abc5f1.29d9d8"
            ]
        ]
    },
    {
        "id": "a9c4b431.b8e6d8",
        "type": "inject",
        "z": "197403af.63076c",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 120,
        "y": 40,
        "wires": [
            [
                "a8dbc0c2.d3946"
            ]
        ]
    },
    {
        "id": "2c052f51.71dc2",
        "type": "image viewer",
        "z": "197403af.63076c",
        "name": "",
        "width": 160,
        "data": "payload",
        "dataType": "msg",
        "x": 90,
        "y": 240,
        "wires": [
            []
        ]
    },
    {
        "id": "233bb9c0.96ff4e",
        "type": "http request",
        "z": "197403af.63076c",
        "name": "",
        "method": "use",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "x": 690,
        "y": 140,
        "wires": [
            [
                "29dbcf2d.703528",
                "dc9b4b4b.53e8d8"
            ]
        ]
    },
    {
        "id": "29dbcf2d.703528",
        "type": "debug",
        "z": "197403af.63076c",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload.outputs[0].data.concepts[0].name",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 780,
        "y": 240,
        "wires": []
    },
    {
        "id": "79b1979c.9de228",
        "type": "function",
        "z": "197403af.63076c",
        "name": "Compose Call to Clarifai",
        "func": "// insert base64 image into payload\nvar b64 = msg.payload; \nmsg.headers = {};\nmsg.payload ={};\nmsg.inputs={};\nmsg.method = \"POST\";\n// Add your own key\nmsg.headers[\"Authorization\"] = 'Key YOUR_KEY';\nmsg.headers[\"Content-Type\"] = \"application/json\";\nmsg.payload = {\n    \"inputs\": [\n      {\n        \"data\": {\n          \"image\": {\n            \"base64\": b64\n          }\n        }\n      }        \n    ]\n  };\n// This model number is the general model - change this for other models\nmsg.url =\"https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs\";\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 470,
        "y": 140,
        "wires": [
            [
                "233bb9c0.96ff4e"
            ]
        ]
    },
    {
        "id": "dc9b4b4b.53e8d8",
        "type": "debug",
        "z": "197403af.63076c",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 920,
        "y": 140,
        "wires": []
    },
    {
        "id": "90abc5f1.29d9d8",
        "type": "base64",
        "z": "197403af.63076c",
        "name": "Convert Picture to Base64",
        "action": "str",
        "property": "payload",
        "x": 370,
        "y": 280,
        "wires": [
            [
                "79b1979c.9de228"
            ]
        ]
    }
]
1 Like
15

I see no photos of birds identified !
(go on - share some results :slight_smile:

16

Ha ha, indeed! We are still at the development stage.

The plan is to have a Pi3 running NodeRed with the Clarifai flow (plus other flows) which will be running as a family dashboard. This will show the bird photos, counters, and other info like weather, twitter feeds, etc.

We will have a PiZero set up with a camera and a motion sensor which will be aimed at the bird feeder. The motion sensor will trigger the camera to take a photo, and feed it through to the Pi3 for processing.

I promise I'll be back with an update when it's all working! :smiley:

1 Like
17

As a little additional information, I have done some work doing the process above to allow Node Red to request information from https://imagga.com/ instead of Clarifai. They both do a very similar thing, but I seem to be getting better results from Imagga but it doesn't seem to allow you to train the free engine like Clarifai does. Anyway, I hope someone finds this useful too.

Here is the content of the function:

// insert base64 image from payload
var image = msg.payload; 
msg.headers = {};
msg.payload ={};
msg.method = "POST";
// Add your own key
msg.headers["Authorization"] = 'Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';
msg.headers["accept"] = "application/json";
msg.headers["content-type"] ="application/x-www-form-urlencoded"
msg.payload = {
    "image_base64": image
  };
// Standard tagging engine
msg.url ="https://api.imagga.com/v2/tags";
return msg;

Pass in a base64 image and you get back a JSON file with the content.

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.