home@home:~$ dig mail.google.com
; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> mail.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47227
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;mail.google.com. IN A
;; ANSWER SECTION:
mail.google.com. 604698 IN CNAME googlemail.l.google.com.
googlemail.l.google.com. 198 IN A 126.96.36.199
;; Query time: 14 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sun Dec 20 12:12:57 GMT 2020
;; MSG SIZE rcvd: 87
home@home:~$ nslookup mail.google.com
mail.google.com canonical name = googlemail.l.google.com.
So that is the same I assume?
Assuming that you are getting back a definitive IP address that you can access, as far as I can see, that should work. The CNAME to IP address conversion should happen at the DNS level so I don't believe that Node-RED would have any issues.
Have you tried doing the nslookup from an exec node inside Node-RED? I think the problem is much more likely to be related to your environment.
Unfortunately, without an equivalent example, I can't reproduce your issue so not sure what I can do to help.