I work on the APIGOTO team, and APIGOTO is independent of OpenAI, Anthropic and Google. It's an OpenAI-compatible API gateway. The flow below works the same against OpenAI itself or any compatible endpoint.
I wanted LLM calls in my flows (status summaries, classifying incoming messages) that don't hang or loop forever when an upstream has a bad minute. I also wanted to switch model families without rewiring anything. Here's the setup I settled on.
Node: @inductiv/node-red-openai-api (Manage palette → Install). It has one OpenAI API node, plus a Service Host config node that holds the endpoint and key.
Service Host config
- API Base:
https://api.china-aiapi.com/v1(the default ishttps://api.openai.com/v1. Any compatible base URL goes here.) - API Key: set the type to
envand the value toAPIGOTO_API_KEY, then define that variable for the Node-RED process. That keeps the key out of your flow exports.credalso works if you prefer to store it in the credentials file. - Leave Auth Header at its default.
OpenAI API node
Method: create chat completion. It reads the request from msg.payload and writes the response back to msg.payload, so the model is just a field:
{ "model": "MODEL_ID", "messages": [{ "role": "user", "content": "..." }] }
Use a model ID from your APIGOTO dashboard (or your provider's list).
Bounded retry with a fallback model
The node reports errors with node.error(err, msg), so a scoped catch node gets the original message back. My flow:
build request(function) puts togethermsg.payloadfrommsg.promptand picks the model from a list based onmsg.attempt.- On error,
retry once, then give upincrementsmsg.attempt. On the first failure it waits 3 s and rebuilds the request usingMODEL_ID_FALLBACK. On the second failure it sends the message to a "gave up" output, where you can notify or log.
Why only one retry in the flow: retries stack up. The openai SDK bundled in the node already retries some errors itself (the SDK default is 2). The gateway also fails over between upstreams automatically, with at most 3 retries. Add a big retry loop in Node-RED on top of that and one inject can turn into a dozen upstream calls and a very long wait. One retry at the flow level, on a different model, is what actually helps.
Flow (Import → paste, then select your Service Host in the chat completion node):
[{"id": "e8871aa2cd338a86", "type": "tab", "label": "LLM via OpenAI-compatible gateway", "disabled": false, "info": ""}, {"id": "611325c17b2479a1", "type": "inject", "z": "e8871aa2cd338a86", "name": "ask", "props": [{"p": "prompt", "v": "Write a one-line status summary for: kitchen 21C, door closed, washer done.", "vt": "str"}], "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "topic": "", "x": 110, "y": 100, "wires": [["32897622a6e9a795"]]}, {"id": "32897622a6e9a795", "type": "function", "z": "e8871aa2cd338a86", "name": "build request", "func": "// first attempt uses the primary model\nmsg.attempt = msg.attempt || 0;\nconst models = [\"MODEL_ID\", \"MODEL_ID_FALLBACK\"];\nmsg.payload = {\n model: models[Math.min(msg.attempt, models.length - 1)],\n messages: [{ role: \"user\", content: msg.prompt }],\n max_tokens: 200\n};\nreturn msg;", "outputs": 1, "x": 290, "y": 100, "wires": [["7c1f5a2aff0d207e"]]}, {"id": "7c1f5a2aff0d207e", "type": "OpenAI API", "z": "e8871aa2cd338a86", "name": "chat completion", "property": "payload", "propertyType": "msg", "service": "", "method": "createChatCompletion", "x": 480, "y": 100, "wires": [["fe8e4e5d6566c26e"]]}, {"id": "fe8e4e5d6566c26e", "type": "change", "z": "e8871aa2cd338a86", "name": "answer + model", "rules": [{"t": "set", "p": "answer", "pt": "msg", "to": "payload.choices[0].message.content", "tot": "msg"}, {"t": "set", "p": "answered_by", "pt": "msg", "to": "payload.model", "tot": "msg"}], "x": 680, "y": 100, "wires": [["a16dfc5c7844b03d"]]}, {"id": "a16dfc5c7844b03d", "type": "debug", "z": "e8871aa2cd338a86", "name": "answer", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "answer", "targetType": "msg", "x": 860, "y": 100, "wires": []}, {"id": "0ca1ed8a59794221", "type": "catch", "z": "e8871aa2cd338a86", "name": "LLM errors", "scope": ["7c1f5a2aff0d207e"], "uncaught": false, "x": 120, "y": 200, "wires": [["22f39daafa2cc623"]]}, {"id": "22f39daafa2cc623", "type": "function", "z": "e8871aa2cd338a86", "name": "retry once, then give up", "func": "// The gateway already fails over between upstreams (max 3 retries)\n// and the SDK inside the node retries some errors too.\n// So retry once here, switching to the fallback model, then stop.\nmsg.attempt = (msg.attempt || 0) + 1;\nif (msg.attempt <= 1) {\n node.warn(`attempt ${msg.attempt} failed: ${msg.error && msg.error.message}`);\n return [msg, null];\n}\nreturn [null, msg];", "outputs": 2, "x": 320, "y": 200, "wires": [["e4498bb40df8dd11"], ["62a87544bdc92962"]]}, {"id": "e4498bb40df8dd11", "type": "delay", "z": "e8871aa2cd338a86", "name": "wait 3s", "pauseType": "delay", "timeout": "3", "timeoutUnits": "seconds", "rate": "1", "nbRateUnits": "1", "rateUnits": "second", "randomFirst": "1", "randomLast": "5", "randomUnits": "seconds", "drop": false, "allowrate": false, "outputs": 1, "x": 540, "y": 180, "wires": [["32897622a6e9a795"]]}, {"id": "62a87544bdc92962", "type": "debug", "z": "e8871aa2cd338a86", "name": "gave up", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "error", "targetType": "msg", "x": 540, "y": 240, "wires": []}]
The answer debug shows msg.answer. msg.answered_by holds the model field from the response, which is handy for seeing when the fallback kicked in.
Why I route through a gateway
APIGOTO uses one base URL for Claude, GPT and Gemini model families, so MODEL_ID and MODEL_ID_FALLBACK can come from different vendors with the same key. Each request shows up in a per-request log, and I check that when a flow does something odd. It's pay-as-you-go, BYOK is supported, and the docs are in English and Chinese: www.china-aiapi.com
How do others handle LLM errors in flows? Catch + delay like this, or something with a status node and a queue?
