Node-RED does provide an API that lets to be installed without restarting the process; this is what the palette manager uses in the editor - https://nodered.org/docs/api/admin/methods/post/nodes/
Your node will need to be packaged properly as a module, with a package.json file etc.
It can only be used to install a module - it cannot be used to reload a module already loaded. If you want Node-RED to reload your node because you've changed something, then you must restart node-red. There is no avoiding that.
