Unnoticed 404 error when using node-red as api server

General
1

Hi There!

I was wondering whether there is some way to get notified of 404 errors from the Node-Red http-in node? Or better said, the webserver underlying the http-in node.

Scenario: I'm using Node-Red as an API server to a database, defining the logic and flows in Node-Red. I connect to the Node-Red server using a python (Flask) server that also provides an additional API for a JS frontend. What happened was that I was getting a Json parser error on the frontend because the python server was accessing a page (API path) that didn't exist on the Node-Red instance (basically a typo in my python code).

I got the json parse error on the frontend because the python code was trying to parse the 404 page from Node-Red as json. It took a while to track down the error since Node-Red was logging any errors for sending out a 404 page.

Are there Node-Red html logs as alternative? What happens when Node-Red gets a 404 in a http-in request?

Cheers!

2

The http-in node works in conjunction with the http-response node.
The http-response node can produce the http error codes (you can have several connected).
You can capture them with a switch node.

3

But that's the thing, Node-Red instance responded before it came to any of my flows - there was no http in for the requested path, so the express server returned 404, none of my flows ever saw the request ...

I was talking to ChatGPT about this and it suggested defining a global http in to catch all in coming requests and then having a function node to act as a router. But then I can't use the http in nodes that I defined for each path separately ...

4

Not sure if am following your issue, the key is:

The http-in node works in conjunction with the http-response node.

It does this by keeping various request/response properties in the msg, if these are removed somewhere in your flow (ie, overwritten by producing a complete new msg), it will produce errors, because it cannot respond to a request that it is not aware of.

5

Imagine an http in node with path "/api/fubar", when I make request from the outside to "/api/fubartypo" the http in node does not receive the request instead it falls down into nirvana, i.e. there is no http in node to handle the request. So the express server of Node-Red responds with 404. Where is that 404 response get logged?

So none of my debug nodes nor flows ever see the request, so there is no way to log it in any of my flows.

Sorry for not being clear about the situation :upside_down_face:

6

Got it, you want to capture a non-existing path. I think this ties in with the core of node-red, which uses express for serving the pages and I am not sure if this is being logged somewhere.

7

You can set named parameters on http in path
/api/:route
You and then use msg.res.params.route to make sure the path is fubar and no typo. Then you can handle your own 404 error
e.g.

[{"id":"42175c8109b17081","type":"http in","z":"da8a6ef0b3c9a5c8","name":"","url":"/api/:route","method":"get","upload":false,"swaggerDoc":"","x":180,"y":3120,"wires":[["c868c0b35316a2ae","415daddfa2f09e43"]]},{"id":"c868c0b35316a2ae","type":"debug","z":"da8a6ef0b3c9a5c8","name":"debug 225","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":370,"y":3180,"wires":[]},{"id":"415daddfa2f09e43","type":"switch","z":"da8a6ef0b3c9a5c8","name":"","property":"req.params.route","propertyType":"msg","rules":[{"t":"eq","v":"fubar","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":330,"y":3100,"wires":[["581010b2ca8a3c33"],["c417072c88d71557"]]},{"id":"c417072c88d71557","type":"change","z":"da8a6ef0b3c9a5c8","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"error:  non route","tot":"str"},{"t":"set","p":"statusCode","pt":"msg","to":"404","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":3140,"wires":[["a6f30d5708e1a412"]]},{"id":"581010b2ca8a3c33","type":"change","z":"da8a6ef0b3c9a5c8","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"correct","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":540,"y":3060,"wires":[["a6f30d5708e1a412"]]},{"id":"a6f30d5708e1a412","type":"http response","z":"da8a6ef0b3c9a5c8","name":"","statusCode":"","headers":{},"x":710,"y":3120,"wires":[]},{"id":"5515489234ec93df","type":"inject","z":"da8a6ef0b3c9a5c8","name":"fubar","props":[{"p":"url","v":"http://192.168.1.10:1880/api/fubar/","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":150,"y":2940,"wires":[["912704151f038109"]]},{"id":"25036fd9debabd99","type":"inject","z":"da8a6ef0b3c9a5c8","name":"typo","props":[{"p":"url","v":"http://192.168.1.10:1880/api/fubartypo/","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":150,"y":3000,"wires":[["912704151f038109"]]},{"id":"912704151f038109","type":"http request","z":"da8a6ef0b3c9a5c8","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":350,"y":2980,"wires":[["b1a470fe0ca38d66"]]},{"id":"b1a470fe0ca38d66","type":"debug","z":"da8a6ef0b3c9a5c8","name":"debug 224","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":530,"y":2980,"wires":[]}]
8

Yep, exactly. I was thinking about this and one could argue that it's outside of Node-Red scope since having an Node-Red behind a load-balancer serving many applications, one would check the http logs of the load-balancer.

On the other hand, having a "handle all http requests with path prefix: /api"-node and then doing the routing oneself would also solve such things as authentication of all API requests, i.e., a global pre-request hook... Perhaps a router node similar to the switch node :upside_down_face:

Thanks for the help :+1:

9

Cool! That would do it, particular since the switch makes for a clear overview of supported routes :+1:

Thanks for that :slight_smile:

10

Ah, unfortunately that does not work since any path with another forward-slash, e.g. /api/fubar/snafu, won't be picked up by that by the single :route variable in the http in.

Solution that I did was to add a second http in with /api/:route/:route2 ... etc

11

and what if /blabla was called, you wouldn't capture it.

12

That's true, but you can that too by defining a http in with /:path as url ... my complete flow is now:

[
    {
        "id": "40ea5f2aea6592ae",
        "type": "tab",
        "label": "API Router",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "42175c8109b17081",
        "type": "http in",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "url": "/api/:path1",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 104,
        "y": 743,
        "wires": [
            [
                "415daddfa2f09e43",
                "15c54866d00797d4"
            ]
        ]
    },
    {
        "id": "415daddfa2f09e43",
        "type": "switch",
        "z": "40ea5f2aea6592ae",
        "name": "path1 handler",
        "property": "req.params.path1",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "accounts",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "users",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "account",
                "vt": "str"
            },
            {
                "t": "else"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 4,
        "x": 353,
        "y": 720,
        "wires": [
            [
                "d6a91b22db931dbd"
            ],
            [
                "b03c04d3dbd79e1a"
            ],
            [
                "ac85faa6b8f463ac"
            ],
            [
                "5f9969817f9c403d"
            ]
        ]
    },
    {
        "id": "a6f30d5708e1a412",
        "type": "http response",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "statusCode": "",
        "headers": {},
        "x": 924,
        "y": 1194,
        "wires": []
    },
    {
        "id": "650565a2d8f644a7",
        "type": "link out",
        "z": "40ea5f2aea6592ae",
        "name": "link out 5",
        "mode": "link",
        "links": [
            "cdfe0a2804636784"
        ],
        "x": 905.5,
        "y": 711,
        "wires": []
    },
    {
        "id": "fba142a7b0634259",
        "type": "link out",
        "z": "40ea5f2aea6592ae",
        "name": "link out 6",
        "mode": "link",
        "links": [
            "33d7ea129f06d293"
        ],
        "x": 732.5,
        "y": 536,
        "wires": []
    },
    {
        "id": "168b0844f9b22469",
        "type": "http in",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "url": "/api/:path1/:path2",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 113,
        "y": 687,
        "wires": [
            [
                "415daddfa2f09e43",
                "15c54866d00797d4"
            ]
        ]
    },
    {
        "id": "f41d7d3566d08f7e",
        "type": "http in",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "url": "/api/:path1/:path2/:path3",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 125,
        "y": 633,
        "wires": [
            [
                "415daddfa2f09e43",
                "15c54866d00797d4"
            ]
        ]
    },
    {
        "id": "d6a91b22db931dbd",
        "type": "switch",
        "z": "40ea5f2aea6592ae",
        "name": "accounts handler",
        "property": "req.params.path2",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "all",
                "vt": "str"
            },
            {
                "t": "else"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 575,
        "y": 548,
        "wires": [
            [
                "fba142a7b0634259"
            ],
            []
        ]
    },
    {
        "id": "ac85faa6b8f463ac",
        "type": "switch",
        "z": "40ea5f2aea6592ae",
        "name": "account handler",
        "property": "req.params.path2",
        "propertyType": "msg",
        "rules": [
            {
                "t": "regex",
                "v": "^[a-z0-9-]+$",
                "vt": "str",
                "case": true
            },
            {
                "t": "else"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 568,
        "y": 720,
        "wires": [
            [
                "dda684b3e27cb284"
            ],
            [
                "5f9969817f9c403d"
            ]
        ]
    },
    {
        "id": "dda684b3e27cb284",
        "type": "function",
        "z": "40ea5f2aea6592ae",
        "name": "set uuid parameter",
        "func": "msg.req.params[\"uuid\"] = msg.req.params.path2;\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 766.5,
        "y": 712,
        "wires": [
            [
                "650565a2d8f644a7"
            ]
        ]
    },
    {
        "id": "5f9969817f9c403d",
        "type": "function",
        "z": "40ea5f2aea6592ae",
        "name": "not found",
        "func": "var pa = msg.req.params;\n\nmsg.statusCode = 404;\nmsg.payload = JSON.stringify({\n    status: \"not_found\",\n    msg: \"page not found\",\n    path: msg.req.originalUrl\n})\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 726.5,
        "y": 1161,
        "wires": [
            [
                "a6f30d5708e1a412"
            ]
        ]
    },
    {
        "id": "7ba18c461581ae70",
        "type": "http in",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "url": "/api",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 96,
        "y": 796,
        "wires": [
            [
                "5f9969817f9c403d",
                "15c54866d00797d4"
            ]
        ]
    },
    {
        "id": "b03c04d3dbd79e1a",
        "type": "switch",
        "z": "40ea5f2aea6592ae",
        "name": "users handler",
        "property": "req.params.path2",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "all",
                "vt": "str"
            },
            {
                "t": "regex",
                "v": "^[a-z0-9-]+$",
                "vt": "str",
                "case": true
            },
            {
                "t": "else"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 3,
        "x": 570,
        "y": 628,
        "wires": [
            [
                "7ade38efb2383248"
            ],
            [
                "341c2535113bf35e"
            ],
            [
                "5f9969817f9c403d"
            ]
        ]
    },
    {
        "id": "7ade38efb2383248",
        "type": "link out",
        "z": "40ea5f2aea6592ae",
        "name": "link out 7",
        "mode": "link",
        "links": [
            "48669bbc4153ba14"
        ],
        "x": 743.5,
        "y": 598,
        "wires": []
    },
    {
        "id": "341c2535113bf35e",
        "type": "function",
        "z": "40ea5f2aea6592ae",
        "name": "not implemented",
        "func": "var pa = msg.req.params;\n\nmsg.statusCode = 501;\nmsg.payload = JSON.stringify({\n    status: \"not_implemented\",\n    msg: \"TODO functionality\",\n    path: msg.req.originalUrl\n})\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 778,
        "y": 1085,
        "wires": [
            [
                "a6f30d5708e1a412"
            ]
        ]
    },
    {
        "id": "4c5494a48c4005b7",
        "type": "http in",
        "z": "40ea5f2aea6592ae",
        "name": "",
        "url": "/:path1",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 96,
        "y": 877,
        "wires": [
            [
                "15c54866d00797d4",
                "5f9969817f9c403d"
            ]
        ]
    },
    {
        "id": "15c54866d00797d4",
        "type": "debug",
        "z": "40ea5f2aea6592ae",
        "name": "debug 23",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 341.5,
        "y": 1220,
        "wires": []
    }
]
13

True but you can get at that too by defining a http in with /:path.

My flow is now:

[{"id":"40ea5f2aea6592ae","type":"tab","label":"APIRouter","disabled":false,"info":"","env":[]},{"id":"42175c8109b17081","type":"httpin","z":"40ea5f2aea6592ae","name":"","url":"/api/:path1","method":"get","upload":false,"swaggerDoc":"","x":104,"y":743,"wires":[["415daddfa2f09e43","15c54866d00797d4"]]},{"id":"415daddfa2f09e43","type":"switch","z":"40ea5f2aea6592ae","name":"path1handler","property":"req.params.path1","propertyType":"msg","rules":[{"t":"eq","v":"accounts","vt":"str"},{"t":"eq","v":"users","vt":"str"},{"t":"eq","v":"account","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":4,"x":353,"y":720,"wires":[["d6a91b22db931dbd"],["b03c04d3dbd79e1a"],["ac85faa6b8f463ac"],["5f9969817f9c403d"]]},{"id":"a6f30d5708e1a412","type":"httpresponse","z":"40ea5f2aea6592ae","name":"","statusCode":"","headers":{},"x":924,"y":1194,"wires":[]},{"id":"650565a2d8f644a7","type":"linkout","z":"40ea5f2aea6592ae","name":"linkout5","mode":"link","links":["cdfe0a2804636784"],"x":905.5,"y":711,"wires":[]},{"id":"fba142a7b0634259","type":"linkout","z":"40ea5f2aea6592ae","name":"linkout6","mode":"link","links":["33d7ea129f06d293"],"x":732.5,"y":536,"wires":[]},{"id":"168b0844f9b22469","type":"httpin","z":"40ea5f2aea6592ae","name":"","url":"/api/:path1/:path2","method":"get","upload":false,"swaggerDoc":"","x":113,"y":687,"wires":[["415daddfa2f09e43","15c54866d00797d4"]]},{"id":"f41d7d3566d08f7e","type":"httpin","z":"40ea5f2aea6592ae","name":"","url":"/api/:path1/:path2/:path3","method":"get","upload":false,"swaggerDoc":"","x":125,"y":633,"wires":[["415daddfa2f09e43","15c54866d00797d4"]]},{"id":"d6a91b22db931dbd","type":"switch","z":"40ea5f2aea6592ae","name":"accountshandler","property":"req.params.path2","propertyType":"msg","rules":[{"t":"eq","v":"all","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":575,"y":548,"wires":[["fba142a7b0634259"],[]]},{"id":"ac85faa6b8f463ac","type":"switch","z":"40ea5f2aea6592ae","name":"accounthandler","property":"req.params.path2","propertyType":"msg","rules":[{"t":"regex","v":"^[a-z0-9-]+$","vt":"str","case":true},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":568,"y":720,"wires":[["dda684b3e27cb284"],["5f9969817f9c403d"]]},{"id":"dda684b3e27cb284","type":"function","z":"40ea5f2aea6592ae","name":"setuuidparameter","func":"msg.req.params[\"uuid\"]=msg.req.params.path2;\n\nreturnmsg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":766.5,"y":712,"wires":[["650565a2d8f644a7"]]},{"id":"5f9969817f9c403d","type":"function","z":"40ea5f2aea6592ae","name":"notfound","func":"varpa=msg.req.params;\n\nmsg.statusCode=404;\nmsg.payload=JSON.stringify({\nstatus:\"not_found\",\nmsg:\"pagenotfound\",\npath:msg.req.originalUrl\n})\nreturnmsg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":726.5,"y":1161,"wires":[["a6f30d5708e1a412"]]},{"id":"7ba18c461581ae70","type":"httpin","z":"40ea5f2aea6592ae","name":"","url":"/api","method":"get","upload":false,"swaggerDoc":"","x":96,"y":796,"wires":[["5f9969817f9c403d","15c54866d00797d4"]]},{"id":"b03c04d3dbd79e1a","type":"switch","z":"40ea5f2aea6592ae","name":"usershandler","property":"req.params.path2","propertyType":"msg","rules":[{"t":"eq","v":"all","vt":"str"},{"t":"regex","v":"^[a-z0-9-]+$","vt":"str","case":true},{"t":"else"}],"checkall":"true","repair":false,"outputs":3,"x":570,"y":628,"wires":[["7ade38efb2383248"],["341c2535113bf35e"],["5f9969817f9c403d"]]},{"id":"7ade38efb2383248","type":"linkout","z":"40ea5f2aea6592ae","name":"linkout7","mode":"link","links":["48669bbc4153ba14"],"x":743.5,"y":598,"wires":[]},{"id":"341c2535113bf35e","type":"function","z":"40ea5f2aea6592ae","name":"notimplemented","func":"varpa=msg.req.params;\n\nmsg.statusCode=501;\nmsg.payload=JSON.stringify({\nstatus:\"not_implemented\",\nmsg:\"TODOfunctionality\",\npath:msg.req.originalUrl\n})\nreturnmsg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":778,"y":1085,"wires":[["a6f30d5708e1a412"]]},{"id":"4c5494a48c4005b7","type":"httpin","z":"40ea5f2aea6592ae","name":"","url":"/:path1","method":"get","upload":false,"swaggerDoc":"","x":96,"y":877,"wires":[["15c54866d00797d4","5f9969817f9c403d"]]},{"id":"15c54866d00797d4","type":"debug","z":"40ea5f2aea6592ae","name":"debug23","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":341.5,"y":1220,"wires":[]}]
14

Bit late to this but for future reference - also don't forget that Node-RED offers an express middleware function (or an array of middleware functions). These can be used to capture errors and additional checks.

15

Thanks for the heads up :slight_smile:

Can that be setup as part of the flow.json? I would like to have everything in one place, i.e., using the frontend.

What I like about this solution is that you see the router in a tab, makes it easy to see inconsistency in the API definition and also endpoints that aren't yet defined (they return a status code of 501 not implemented).

16

No, it is defined in settings.js - but that is in the same place as your flows/credentials, etc. So backup, etc is the same.

And it absolutely shouldn't be accessible to the front-end since it gives fundamental access to the inner workings of ExpressJS so would be a serious security issue. But in any case, changing the middleware requires Node-RED to be restarted.

Probably not an answer to your issue but may be useful to others.

17

Definitely! For my purposes (designing an API) it's ideal to have the router as flow.

A bit of OT but this is my first project with Node-Red and have to say it absolutely brilliant :+1: It makes programming became an art form - if the flow doesn't look right, the code must be broken :slight_smile:

Hats off to all the developers that make Node-Red so great!

3 Likes
18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.