# LLM calls in Node-RED through an OpenAI-compatible gateway, with a bounded retry + fallback model (flow included)

**URL:** <https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953>\
**Category:** Share Your Projects\
**Tags:** llm\
**Created:** [20 October 2026 20:19 UTC](https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953 "2026-10-20T20:19:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![apigoto](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@apigoto](https://discourse.nodered.org/u/apigoto)\
**Post date:** [20 October 2026 20:19 UTC](https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953/1 "2026-10-20T20:19:53Z")

</div>

I work on the APIGOTO team, and APIGOTO is independent of OpenAI, Anthropic and Google. It's an OpenAI-compatible API gateway. The flow below works the same against OpenAI itself or any compatible endpoint.

I wanted LLM calls in my flows (status summaries, classifying incoming messages) that don't hang or loop forever when an upstream has a bad minute. I also wanted to switch model families without rewiring anything. Here's the setup I settled on.

**Node:** `@inductiv/node-red-openai-api` (Manage palette → Install). It has one `OpenAI API` node, plus a `Service Host` config node that holds the endpoint and key.

**Service Host config**

- **API Base:** `https://api.china-aiapi.com/v1` (the default is `https://api.openai.com/v1`. Any compatible base URL goes here.)
- **API Key:** set the type to `env` and the value to `APIGOTO_API_KEY`, then define that variable for the Node-RED process. That keeps the key out of your flow exports. `cred` also works if you prefer to store it in the credentials file.
- Leave **Auth Header** at its default.

**OpenAI API node**

Method: **create chat completion**. It reads the request from `msg.payload` and writes the response back to `msg.payload`, so the model is just a field:

```json
{ "model": "MODEL_ID", "messages": [{ "role": "user", "content": "..." }] }

```

Use a model ID from your APIGOTO dashboard (or your provider's list).

**Bounded retry with a fallback model**

The node reports errors with `node.error(err, msg)`, so a scoped **catch** node gets the original message back. My flow:

1. `build request` (function) puts together `msg.payload` from `msg.prompt` and picks the model from a list based on `msg.attempt`.
2. On error, `retry once, then give up` increments `msg.attempt`. On the first failure it waits 3 s and rebuilds the request using `MODEL_ID_FALLBACK`. On the second failure it sends the message to a "gave up" output, where you can notify or log.

Why only one retry in the flow: retries stack up. The openai SDK bundled in the node already retries some errors itself (the SDK default is 2). The gateway also fails over between upstreams automatically, with at most 3 retries. Add a big retry loop in Node-RED on top of that and one inject can turn into a dozen upstream calls and a very long wait. One retry at the flow level, on a _different_ model, is what actually helps.

Flow (Import → paste, then select your Service Host in the `chat completion` node):

```json
[{"id": "e8871aa2cd338a86", "type": "tab", "label": "LLM via OpenAI-compatible gateway", "disabled": false, "info": ""}, {"id": "611325c17b2479a1", "type": "inject", "z": "e8871aa2cd338a86", "name": "ask", "props": [{"p": "prompt", "v": "Write a one-line status summary for: kitchen 21C, door closed, washer done.", "vt": "str"}], "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "topic": "", "x": 110, "y": 100, "wires": [["32897622a6e9a795"]]}, {"id": "32897622a6e9a795", "type": "function", "z": "e8871aa2cd338a86", "name": "build request", "func": "// first attempt uses the primary model\nmsg.attempt = msg.attempt || 0;\nconst models = [\"MODEL_ID\", \"MODEL_ID_FALLBACK\"];\nmsg.payload = {\n model: models[Math.min(msg.attempt, models.length - 1)],\n messages: [{ role: \"user\", content: msg.prompt }],\n max_tokens: 200\n};\nreturn msg;", "outputs": 1, "x": 290, "y": 100, "wires": [["7c1f5a2aff0d207e"]]}, {"id": "7c1f5a2aff0d207e", "type": "OpenAI API", "z": "e8871aa2cd338a86", "name": "chat completion", "property": "payload", "propertyType": "msg", "service": "", "method": "createChatCompletion", "x": 480, "y": 100, "wires": [["fe8e4e5d6566c26e"]]}, {"id": "fe8e4e5d6566c26e", "type": "change", "z": "e8871aa2cd338a86", "name": "answer + model", "rules": [{"t": "set", "p": "answer", "pt": "msg", "to": "payload.choices[0].message.content", "tot": "msg"}, {"t": "set", "p": "answered_by", "pt": "msg", "to": "payload.model", "tot": "msg"}], "x": 680, "y": 100, "wires": [["a16dfc5c7844b03d"]]}, {"id": "a16dfc5c7844b03d", "type": "debug", "z": "e8871aa2cd338a86", "name": "answer", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "answer", "targetType": "msg", "x": 860, "y": 100, "wires": []}, {"id": "0ca1ed8a59794221", "type": "catch", "z": "e8871aa2cd338a86", "name": "LLM errors", "scope": ["7c1f5a2aff0d207e"], "uncaught": false, "x": 120, "y": 200, "wires": [["22f39daafa2cc623"]]}, {"id": "22f39daafa2cc623", "type": "function", "z": "e8871aa2cd338a86", "name": "retry once, then give up", "func": "// The gateway already fails over between upstreams (max 3 retries)\n// and the SDK inside the node retries some errors too.\n// So retry once here, switching to the fallback model, then stop.\nmsg.attempt = (msg.attempt || 0) + 1;\nif (msg.attempt <= 1) {\n node.warn(`attempt ${msg.attempt} failed: ${msg.error && msg.error.message}`);\n return [msg, null];\n}\nreturn [null, msg];", "outputs": 2, "x": 320, "y": 200, "wires": [["e4498bb40df8dd11"], ["62a87544bdc92962"]]}, {"id": "e4498bb40df8dd11", "type": "delay", "z": "e8871aa2cd338a86", "name": "wait 3s", "pauseType": "delay", "timeout": "3", "timeoutUnits": "seconds", "rate": "1", "nbRateUnits": "1", "rateUnits": "second", "randomFirst": "1", "randomLast": "5", "randomUnits": "seconds", "drop": false, "allowrate": false, "outputs": 1, "x": 540, "y": 180, "wires": [["32897622a6e9a795"]]}, {"id": "62a87544bdc92962", "type": "debug", "z": "e8871aa2cd338a86", "name": "gave up", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "error", "targetType": "msg", "x": 540, "y": 240, "wires": []}]

```

The `answer` debug shows `msg.answer`. `msg.answered_by` holds the `model` field from the response, which is handy for seeing when the fallback kicked in.

**Why I route through a gateway**

APIGOTO uses one base URL for Claude, GPT and Gemini model families, so `MODEL_ID` and `MODEL_ID_FALLBACK` can come from different vendors with the same key. Each request shows up in a per-request log, and I check that when a flow does something odd. It's pay-as-you-go, BYOK is supported, and the docs are in English and Chinese: [www.china-aiapi.com](https://www.china-aiapi.com/grokbot)

How do others handle LLM errors in flows? Catch + delay like this, or something with a status node and a queue?

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [20 October 2026 20:24 UTC](https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953/2 "2026-10-20T20:24:08Z")

</div>

> [@apigoto](#):
>
> [www.china-aiapi.com](http://www.china-aiapi.com)

Forum members may wish to note that I can't find any 3rd-party information regarding this site. Buyer discretion is advised.

---

<div class="post-metadata">

**Author:** ![apigoto](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@apigoto](https://discourse.nodered.org/u/apigoto)\
**Post date:** [21 October 2026 02:36 UTC](https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953/3 "2026-10-21T02:36:32Z")

</div>

Fair point, thanks for flagging it. We're a young service, so there isn't much independent coverage yet, and caution is reasonable. To be clear about who we are: I work on the APIGOTO team; we're independent of OpenAI, Anthropic and Google, and billing is pay-as-you-go, so nobody needs to commit money up front to try it. Request logs are only stored if you switch logging on in the account settings.

Nothing in the flow depends on us, though: set API Base in the Service Host to `https://api.openai.com/v1` (or any other OpenAI-compatible endpoint, including a local one) and the retry + fallback logic works unchanged. Happy to answer questions here, or at [postmaster@apigoto.com](mailto:postmaster@apigoto.com).

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [21 October 2026 09:06 UTC](https://discourse.nodered.org/t/llm-calls-in-node-red-through-an-openai-compatible-gateway-with-a-bounded-retry-fallback-model-flow-included/101953/4 "2026-10-21T09:06:13Z")

</div>