Couple of points there. I believe it was always intended to eventually allow the base URL to change, it hasn't been implemented yet.
In regard to D2 "Pages" - don't forget that they are not actually "pages" in an HTTP sense, they are front-end routed virtual pages. Unless I've misunderstood, this is actually the same as D1 but better hidden I think?
Not that easy - see my next comment. However, yes, this is how UIBUILDER does it. In UIBUILDER's case, it adds a few Express Router's and attaches endpoints to them. As it does these things, it maintains a list of everything that has been added. These are shown on the "Full Details" link in any uibuilder node.
The harder part stems from the fact that Node-RED provides so much flexibility in terms of its configuration. This means that root URL's may change based on settings in settings.js. These settings have to be taken into account as well. Certainly possible but just a little more complex. This is why I always recommend and use relative URL's in UIBUILDER.
That is an interesting take on things. A standardised search API for finding endpoints. Not sure how that would work but it sounds feasible maybe. Rather more complex than getting nodes to register their endpoints though.
The other complexity either way would be the need to indicate whether an endpoint URL was a leaf or a branch. Branches would, of course, allow further sub-folders and pages, leaf's would not.
and type of either "leaf" or "branch". Possibly you might also want to indicate whether branches were server-side or client-side. For example, a uibuilder node's "URL" setting can actually be either a server-side branch (since it uses static serving) or a client-side branch (using the uib-router for an SPA) or even both. Either way, the node does not actually "know" all of the possible end-points. This gets even worse (or better depending on your viewpoint) for Markweb nodes because, while all of the pages are server-generated, the generation is fully dynamic and tracking all of the possible endpoints from a large content website would certainly be a large dataset to manage. It could amount to 10's of thousands of folders and pages.
None of this is insurmountable of course, I'm just highlighting some of the possible complexities that would need to be thought through.
We perhaps also need to think about the split between the admin and user web servers that Node-RED maintains and the fact that any node can create additional web servers - uibuilder has that as an option of course. Websocket endpoints also need to be accounted for. And http-in nodes allow wild-card server endpoint definitions as well.