# Getting to 3D world with node-red-contrib-web-babylonjs

**URL:** <https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844>\
**Category:** Share Your Nodes\
**Created:** [15 April 2020 04:55 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844 "2020-04-15T04:55:00Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![ahmadsayed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ahmadsayed/32/24312_2.png) [@ahmadsayed](https://discourse.nodered.org/u/ahmadsayed)\
**Post date:** [15 April 2020 04:55 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/1 "2020-04-15T04:55:00Z")

</div>

This is a very early usable draft of my idea, about 3D visualization , may be AR in later phases not now, using node-red based on webGL and babylonjs library

> **[node-red-contrib-web-babylonjs](https://www.npmjs.com/package/node-red-contrib-web-babylonjs)**
>
> Realize an IoT project with 3D objects, it is in very early stage, The aim of this project is build augemented reality nodes, for visualizing IoT real data in Realtim

  
 ![](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/b/a/baad0e802770c7c0ca5513cbd282e9ca9eee9066.gif)  
FEAUTRE:  
Draw basic mesh Sphere, box, plan , box  
Do spatial transformation using websocket

LIMITATION:  
Only work localhost, still the websockt hardcoded localhot

---

<div class="post-metadata">

**Author:** ![BartButenaers](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bartbutenaers/32/10476_2.png) [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)\
**Post date:** [15 April 2020 05:13 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/2 "2020-04-15T05:13:41Z")

</div>

Morning Ahmed,

Looks like a _ **VERY** _ useful contribution!

### Question 1

I had never heard about [babylon.js](https://www.babylonjs.com/) until now. Are there any special _ **IOT related use cases** _ you had already in mind for your node?

Our friend @dceejay had a proposal some time ago, based on this article:

> **[The Smart Home Gains An Extra Dimension](https://hackaday.com/2019/12/29/the-smart-home-gains-an-extra-dimension/)**
>
> With an ever-growing range of smart-home products available, all with their own hubs, protocols, and APIs, we see a lot of DIY projects (and commercial offerings too) which aim to provide a “…

That article uses [three.js](https://threejs.org/) and has a nice 3D use case to control lights in a house:

![](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/2/d23a324aa38e55bcdfc26cc5ffa28af7b88fdbca.gif)

I see that babylon.js also supports [lights](https://doc.babylonjs.com/babylon101/lights). So this kind of fancy stuff might be possible with your node?

### Question 2

Had no time yet to play with your node, so perhaps this is already possible? Is there any kind of babylon.js _ **web based drawing editor that could be embedded** _ with your node. So we can go from your node directly to an editor, like we integrated DrawSvg into our node-red-contrib-ui-svg node?

Thanks for your time and effort!!!  
Bart

---

<div class="post-metadata">

**Author:** ![ahmadsayed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ahmadsayed/32/24312_2.png) [@ahmadsayed](https://discourse.nodered.org/u/ahmadsayed)\
**Post date:** [15 April 2020 05:21 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/3 "2020-04-15T05:21:08Z")

</div>

Hi Bart,

to be honest, I was working on it and hesitated to share it till I have a valid use case , but I shared it to explore with community the use-cases, I was exploring AR as well

> **[The missing link between AR, IoT and the Industry 4.0](https://medium.com/@wikitude/the-missing-link-between-ar-iot-and-the-industry-4-0-8d03d916e78d)**
>
> The Augmented Reality (AR) space is evolving, both from the hardware standpoint and the way in which enterprises are using the technology…

but what you propose is really interesting usecase as well

for the editor, yes, there is editor, not sure yet about the capabilities still exploring it as well

---

<div class="post-metadata">

**Author:** ![ahmadsayed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ahmadsayed/32/24312_2.png) [@ahmadsayed](https://discourse.nodered.org/u/ahmadsayed)\
**Post date:** [15 April 2020 07:46 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/4 "2020-04-15T07:46:01Z")

</div>

@BartButenaers I am exploring DrawSvg is very impressive, specially when you click in the screen it will reflect on the node-red  
Full of useful idea. thank you very much for referencing it.  
May be I can think to add similar features.

---

<div class="post-metadata">

**Author:** ![BartButenaers](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bartbutenaers/32/10476_2.png) [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)\
**Post date:** [15 April 2020 07:54 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/5 "2020-04-15T07:54:17Z")

</div>

Damn, I need more time to explore all this nice stuff 🙄  
P.S. I have also a [node-red-contrib-drawsvg](https://github.com/bartbutenaers/node-red-contrib-drawsvg) node, that allows to host a DrawSvg service for offline installations. Perhaps (in a far future) we could implement a similar node to host an offline BabylonJs editor...

I did not yet have a chance to look at your code. Am I correct that you only use the BabylonJs _ **viewer** _ to render the result? I saw this morning that there is also a BabylonJs _ **editor** _ available. Don't see immediately how you could integrate this into your jQuery config screen. Although I assume it should be possible somehow, because (if I'm not mistaken) they show [here](https://doc.babylonjs.com/resources/babylonjs_and_reactjs) how to integrate it within a React web app?

---

<div class="post-metadata">

**Author:** ![BartButenaers](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bartbutenaers/32/10476_2.png) [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)\
**Post date:** [15 April 2020 17:46 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/6 "2020-04-15T17:46:22Z")

</div>

@ahmadsayed,  
Finally I had time to have a look at your repository. Now I see that it is a "...-web-..." node instead of a dashboard "...-ui-..." node, so you show the rendering in a separate window. And you have created a separate node for each shape (sphere, plane, ...):

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/c/0/c041e9650fd5cdeac49accdef7a712484d462b0c.png)

It is **VERY** user friendly this way, since you can just drop new shapes in the flow. But - based on the feedback and feature requests we got from the UI SVG node users - I 'think' this approach will have also some disadvantages:

- Not possible to create extra shapes dynamically.
- Due to the large number of available shapes, the palette sidebar on the left will become very large.
- Is it possible to create multiple (completely separate) scenes, or do all the shapes belong to a single scene?
- It might be a bit more difficult if you want to add user interaction with a scene. Suppose a user wants to make a sphere (e.g. a light bulb) clickable: then you need to add a wire the output of that 'sphere' node. Which will result in lots of wires.

But don't hesitate to correct me if I'm wrong!!!!!!

Thought this morning (by having a quick look at it) that it was a bit more like the UI SVG node. I mean that there was a single BabylonJs node, which:

- can dynamically be controlled via input messages:

- sends output messages when a clickable shapes would be clicked.

- contains a large edit box on the config screen, where you can manually enter a the JSON scene definition manually.

- (Optionally) had some way to open the BabylonJs online/offline scene editor via a button, to allow the JSON scene to be edited via that tool.

Keep up the good work! I'm sure your node has a lot of potential...  
Bart

---

<div class="post-metadata">

**Author:** ![ahmadsayed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ahmadsayed/32/24312_2.png) [@ahmadsayed](https://discourse.nodered.org/u/ahmadsayed)\
**Post date:** [15 April 2020 18:50 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/8 "2020-04-15T18:50:08Z")

</div>

Hi Brat,

> [@BartButenaers](#):
>
> - Not possible to create extra shapes dynamically.
> - Due to the large number of available shapes, the palette sidebar on the left will become very large.

You are absolutely right, having a node that build the scene dynamically, in declarative way is on my TODO list.  
Let me tell my thinking process, most of the 3D software give the user basic 3D mesh a.k.a primitive mesh, even some tools such as blender has some easter eggs like the monkey head as primitives, as basic directly drag and drop components, to give the user jump start or rapid protoyping still I believe node-red is the best may be the only tools that give this intuitive rapid protoyping, without almost reading any documentation.

ofcourse the 3D tools give some capabilities to morph those primitives later on, but my thinking is if I even add this morphing it will a procedural morphing by code, not artistic (still not in my TODO list as of now honestly)

by providing those basic mesh it will give the user an easy jump start , still a declarative way to provide advanced mesh, can be provided, but in that case will use a standard file format such as gITF, that can be created by professional software tools, and will rendered and controlled by node-red and this is in my TODO list, but this complex mesh will be single unit, with single input and output.

> [@BartButenaers](#):
>
> It might be a bit more difficult if you want to add user interaction with a scene. Suppose a user wants to make a sphere (e.g. a light bulb) clickable: then you need to add a wire the output of that 'sphere' node. Which will result in lots of wires.

to be honest the idea of making the 3d object clickable just came to me from your initial comment, and it become almost the first in my TODO list, i did not put much thinking on it yet, I am using my 8 years son to test it, whatever he find convenient I do it 🙂

> [@BartButenaers](#):
>
> Is it possible to create multiple (completely separate) scenes, or do all the shapes belong to a single scene?

as of now no, but why the scene node exists at all not just add it implicitly ,the scene itself, has some defaults presets for camera and right, again in my TODO list 😊 is to add some properties to control the global lighting as well camera behavior , not as advanced as professional 3D software it will be usecase opinionated presets, that is my thinking

> [@BartButenaers](#):
>
> Keep up the good work! I'm sure your node has a lot of potential...

Thank you very much for the support.

---

<div class="post-metadata">

**Author:** ![BartButenaers](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bartbutenaers/32/10476_2.png) [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)\
**Post date:** [15 April 2020 19:55 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/9 "2020-04-15T19:55:39Z")

</div>

> [@ahmadsayed](#):
>
> to give the user jump start or rapid protoyping

That is indeed true! If you would have the time to offer both solutions, then you could support all kind of users. Just be aware that lots of users don't like their palette being stuffed with tons of nodes, especially which they don't use. So perhaps you should (somehow?) try to split it in two modules: i.e. a separate node-red-xxx suite with all the shape nodes?

> [@ahmadsayed](#):
>
> making the 3d object clickable

In the SVG node we have a dual way communication with the shapes:

- By _input messages_ you can change shapes (e.g. change the color of a sphere, let a light source shine, ...).
- By _output messages_ you know which event has happened, so when a light source is clicked you could really activate that light bulb in your house.

BTW I [see](https://doc.babylonjs.com/how_to/interactions) that BabylonJs supports mouse/key/touch events are supported. We will soon (in the svg node version 2.0) release touch events, because lots of users have a Node-RED dashboard running on e.g. a wall-mounted touch screen...

But of course our SVG node is a dashboard node, which means the drawing is rendered in the dashboard. Yours is a normal node, so assume you have based it on the worldmap node from @dceejay? And that can also be displayed in the dashboard (I thought?)??

> [@ahmadsayed](#):
>
> I am using my 8 years son to test it, whatever he find convenient I do it

If you find a female test person, that would even be better to determine the [WAF-factor](https://en.wikipedia.org/wiki/Wife_acceptance_factor) 😂

> [@ahmadsayed](#):
>
> as of now no

When you would ever have that declarative node, you could support a scene per node perhaps?

> [@ahmadsayed](#):
>
> Thank you very much for the support.

You are welcome. Don't hesitate to ask for feedback!

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [15 April 2020 21:20 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/10 "2020-04-15T21:20:00Z")

</div>

Just to join in the fun I did get this far using ar.js

> **[dceejay/node-red-arjs](https://github.com/dceejay/node-red-arjs)**
>
> An ARjs node for plotting thing in the real world from Node-RED - dceejay/node-red-arjs

It should install manually - but I've not added it to the flows site (as it's a bit clunky still)  
Being me - it is indeed based on the same concepts as worldmap - idea being that some of the same messages would "just work"... (for simple blob icons at a location etc). But that you could specify various other primitives and colours etc...

Main problem is that AR.js support is not great on non-phone devices - and then only on some phones - and exposing Node-RED to internet so phones can render it is a pain (though ngrok helps for developing). Secondly due to scaling anything an interesting distance away tended to be very small (invisible)... But hey it works - sort of.

of course that was 3 months ago - it may be much better now 🙂

---

<div class="post-metadata">

**Author:** ![ahmadsayed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ahmadsayed/32/24312_2.png) [@ahmadsayed](https://discourse.nodered.org/u/ahmadsayed)\
**Post date:** [16 April 2020 02:12 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/11 "2020-04-16T02:12:32Z")

</div>

AR was my initial motivation to make those nodes, and I faced the same issues, with AR I do not know about arjs, if it implements AR natively, but in my case I tried to enable AR, The library I am using depends on webxr ([https://www.w3.org/TR/webxr/](https://www.w3.org/TR/webxr/)) so it is expecting ARCore to be installed on the phone, and for windows it expect AR headset to be connected in order to work via windows mixed reality, tried the emulator no luck.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [14 June 2020 04:55 UTC](https://discourse.nodered.org/t/getting-to-3d-world-with-node-red-contrib-web-babylonjs/24844/12 "2020-06-14T04:55:04Z")

</div>

This topic was automatically closed after 60 days. New replies are no longer allowed.