Missing DNS caching in Node-RED (or NodeJS?)

Hi fellows,
I'm using Node-RED for some years now and I couldn't be happier with it.
But today I stumbled over the statistics of DNS requests from Node-RED in Pi-hole. There are tons of DNS queries. It seems like there is no DNS caching available at all and each and every http(s) request or influx query cause a DNS lookup.

I found this thread which is closed since some time: InfluxDB node does DNS query in on every message

For me missing DNS caching applies to all nodes. Especially http and influx is causing tons of requests. After finding this -> https://httptoolkit.com/blog/configuring-nodejs-dns/ it seems like an issue in NodeJS.
To me the solution doesn't look like it could easily be applied centrally in Node-RED.

I'd be happy to hear some suggestions how to solve that issue.

Just tried "npm install dnscache-ng" but DNS is still requested for the same domain over and over again.

If it happens anywhere, I'd expect DNS caching to happen at the OS level, not the application level.

Hmm, just looked up the Node.js documentation and found something that I hadn't realised. DNS | Node.js v18.14.0 Documentation (nodejs.org)

So it actually depends on how you code your lookups as to whether you get the benefit of the OS's handling (caching) but at the possible issue of excessive external thread usage, or avoid the external thread but have to deal with your own caching.

Are you using Docker? That seems to mess up DNS caching anyway from that 1st link.

It seems anyway that the resolution of these issues comes down to the individual libraries knowing that there is an issue and coding around it accordingly. For InfluxDB, you'd probably need to raise an issue against the underlying library.

In other cases in Node-RED, each node would need to be checked to see what libraries it is using and following up with each. Do you have a list of probably culprits?

So many problems :smiley:
First issue: TTL for my internal domains was 0 by default in PiHole. After setting local-ttl in PiHole at least this is fixed.

Second issue: Yes I'm using docker. When pinging my internal domain three times from the container PiHole shows three requests. Guess that's what needs to be fixed next.

Third issue: Like you said. Depending on the way each node is doing the lookup changes could be required. Still have some hope that when the docker issue is fixed somehow the node package "dnscache-ng" could solve this. That would avoid updating many different nodes.

I can tell for sure that the following node's requests are currently not cached (but doesn't mean anything with the docker issue in mind):

  • influx
  • http request
  • node-red-contrib-ical-events : ical-upcoming

The only solution I found is installing another docker container running a caching DNS and replace the DNS server setting in all my docker containers. For sure it makes sense that docker ist providing DNS service for the containers because otherwise services in a stack could not be resolved by their name. Strange though that it doesn't cache.

Learned a lot about docker and node yesterday :grin:

95% of my dns requests are coming from the influx node. For my small Home environment I decided to ditch the https influx connection (terminated on Caddy reverse proxy) via domain and simply use http with the IP of the influx container. Shouldn't be a security issue as both are running as services on the same stack (docker compose) and no one else has access to the system.

1 Like

Does it really matter that you are getting additional DNS requests?

It was like 100k requests per day. I just don't like the idea of of having so many useless dns requests each day.
Beside that maybe others will be hit harder by this topic and have massive performance issues in bigger Installations.

Have you enabled the DNS cache in pihole, so they don't go out to the internet?

The influx domain is a local dns entry in Pihole and due to static IP I've set TTL to 86400.
Caching on Pihole is working fine.

Why is that any better than the resolution or caching in pihole?

Thats exactly why I didn't want to do it. But it could possibly save some latency or network traffic depending on where Pihole is running.

100k/day is a bit over 1/second, which is going to have absolutely no measurable effect on your network. Mine is running at 30k/day and on a Pi4 it shows mostly 0.0% of 1 core CPU, and occasionally pops up to 0.3%, so again no measurable effect on anything.

Got your point. Call me pedantic but I just don't like wasting resources. Dns caching has been developed for a reason. If every OSI layer/ architectural component is just implemented less efficient it will sum up at some point and everybody will say: My Implementation could be more performant but its not that bad.

If you check the docs, it can really impact other services doing DNS lookups on the same device. So the answer is a resounding "maybe".

On my machine the default resolver (as used by dns.resolve) is initialized to the same as /etc/resolv.conf lists. I assume that in general the two methods will query the same sources unless explicitly otherwise configured.

That means that you need to focus on what docker does. Since the node-red container doesn't have any other services it comes down to docker configuration and then host configuration. (I'd be surprised if docker doesn't use the local host DNS config after handling the special inter-container cases). If you're on linux and want local dns resolution you can use systemd-resolved or dnsmasq.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.