Trying to understand your demands, this sounds like a typical use case for Configuration nodes.
In short, this is a special type of node used to store (config) data that is shared between a number of nodes. Thus it eliminates the need to access settings.js or store data (manually) in the file system.
Depends on your definition of "access". Check the chapter "Using a config node" in the linked documentation for how this looks like in the editor.
No. Config nodes don't appear in the palette. They are created on demand via the property editor of a node that uses this config data. Again: Check the linked documentation for how this looks like; picture borrowed from there:
Config nodes that carry data (e.g. have been created) show up at the right hand panel labeled "Configuration nodes".
Are you wanting to persist the file so the data survives between restarts of Node-RED or the server?
If not, all you need is a shared module accessible to all instances of your nodes.
To persist the data, you could use node-red's context store but that requires users of your node to set up a retained context store in settings.js. Or you could simply write to the filing system of the node-red server, though that will limit your node to only be used on servers with filing systems (FlowForge does not have one).
Otherwise, a config node will work. If you only ever want a single config node, you could automatically create it in code and wouldn't need to show it to users.
You would probably want the node context store. However, content stores are not persisted by default. You have to configure Node-RED with a persistent store, for example the file store or redis store. That is done in each Node-RED instance in settings.js. So it really depends on you having some control over that - which may be fine for you but if you are providing for multiple customers, it might be tricky.
No, you cannot do that directly. The standard method is for your custom node to provide an admin API to get access.
Again though, unless you want other nodes to have access to the data - which obviously can be dangerous - use the node context (confusingly just called "context") if you can because only instances of your custom node will have access and each node instance will have its own copy of the data.