Hi fellows,
I'm using Node-RED for some years now and I couldn't be happier with it.
But today I stumbled over the statistics of DNS requests from Node-RED in Pi-hole. There are tons of DNS queries. It seems like there is no DNS caching available at all and each and every http(s) request or influx query cause a DNS lookup.
For me missing DNS caching applies to all nodes. Especially http and influx is causing tons of requests. After finding this -> https://httptoolkit.com/blog/configuring-nodejs-dns/ it seems like an issue in NodeJS.
To me the solution doesn't look like it could easily be applied centrally in Node-RED.
I'd be happy to hear some suggestions how to solve that issue.
So it actually depends on how you code your lookups as to whether you get the benefit of the OS's handling (caching) but at the possible issue of excessive external thread usage, or avoid the external thread but have to deal with your own caching.
Are you using Docker? That seems to mess up DNS caching anyway from that 1st link.
It seems anyway that the resolution of these issues comes down to the individual libraries knowing that there is an issue and coding around it accordingly. For InfluxDB, you'd probably need to raise an issue against the underlying library.
In other cases in Node-RED, each node would need to be checked to see what libraries it is using and following up with each. Do you have a list of probably culprits?
So many problems
First issue: TTL for my internal domains was 0 by default in PiHole. After setting local-ttl in PiHole at least this is fixed.
Second issue: Yes I'm using docker. When pinging my internal domain three times from the container PiHole shows three requests. Guess that's what needs to be fixed next.
Third issue: Like you said. Depending on the way each node is doing the lookup changes could be required. Still have some hope that when the docker issue is fixed somehow the node package "dnscache-ng" could solve this. That would avoid updating many different nodes.
I can tell for sure that the following node's requests are currently not cached (but doesn't mean anything with the docker issue in mind):
The only solution I found is installing another docker container running a caching DNS and replace the DNS server setting in all my docker containers. For sure it makes sense that docker ist providing DNS service for the containers because otherwise services in a stack could not be resolved by their name. Strange though that it doesn't cache.
Learned a lot about docker and node yesterday
95% of my dns requests are coming from the influx node. For my small Home environment I decided to ditch the https influx connection (terminated on Caddy reverse proxy) via domain and simply use http with the IP of the influx container. Shouldn't be a security issue as both are running as services on the same stack (docker compose) and no one else has access to the system.
It was like 100k requests per day. I just don't like the idea of of having so many useless dns requests each day.
Beside that maybe others will be hit harder by this topic and have massive performance issues in bigger Installations.
100k/day is a bit over 1/second, which is going to have absolutely no measurable effect on your network. Mine is running at 30k/day and on a Pi4 it shows mostly 0.0% of 1 core CPU, and occasionally pops up to 0.3%, so again no measurable effect on anything.
Got your point. Call me pedantic but I just don't like wasting resources. Dns caching has been developed for a reason. If every OSI layer/ architectural component is just implemented less efficient it will sum up at some point and everybody will say: My Implementation could be more performant but its not that bad.
On my machine the default resolver (as used by dns.resolve) is initialized to the same as /etc/resolv.conf lists. I assume that in general the two methods will query the same sources unless explicitly otherwise configured.
That means that you need to focus on what docker does. Since the node-red container doesn't have any other services it comes down to docker configuration and then host configuration. (I'd be surprised if docker doesn't use the local host DNS config after handling the special inter-container cases). If you're on linux and want local dns resolution you can use systemd-resolved or dnsmasq.
