Need some expert guidance If I want to connect 10k or above device with node-red via MQTT what is the best approach. Also, I want to show statistics on the dashboard for each and every device. a device you can assume ESP8266 or RaspberryPi.

See Will there be any performance hit because of using many MQTT nodes I wouldn't have thought it was practicable to have one dashboard showing statistics for 10000 devices I'd thought that feeding 100 devices into one computer might be possible and then sending aggregate information from those 10…

Do you have 10k topics, one for each MQTT node? If so you need to configure 10k MQTT nodes in a flow which is not possible so you should use multiple flows. You will have very hard time to do the configuration of flows like drag and drop and give topics etc. You will face Will there be any …

I would never even consider trying to create flows of this magnitude by hand... do you at least have a spreadsheet or database of the 10k devices? If so, you could use them to feed a master flow which generates the worker flows and deploys them to an array of PIs. As for the dashboard, you will nee…

[image] nithinbandaru1: If so you need to configure 10k MQTT Hopefully the topic structure is designed a bit more sensibly then that and you can use wildcards in the topics to subscribe to multiple topics with a single node. For example, (given we have absolutely no context for the original …

Thanks for advice.. If I skip the dashboard with real-time data and store them in database first. still I need some expert knowledge to make scalable solution because data is huge and very frequent. please suggest best approach to push data in database via nodered.

@piyushbarua According to me kafka or cassandra could be good options for storing huge amount of data.

[image] piyushbarua: If I skip the dashboard with real-time data and store them in database first. still I need some expert knowledge to make scalable solution because data is huge and very frequent. please suggest best approach to push data in database via nodered. You will need to share a l…

We deployed mobile signs on different locations. these signs have different sets of sensors. getting data of position, location, sun-light, wind and radar. all data coming in json format. we have different topics for data because size of data is large we can't send in single shot. we created a servi…

[image] piyushbarua: Guys, need best approach as I said to handle this problem. We all like someone to solve our problems for us :slight_smile: But you are proposing something that is way beyond most (if not all) people on this forums, use of NodeRED What sort of project is this BTW - comme…

As Nick suggested above - one of the main keys is a well chosen topic tree to allow you to make best use of wildcard subscriptions, so that you can aggregate the data from one type of thing (whether that be by device or by reading type). Then feed as directly as possible to an appropriate database.

Node-RED

Node-RED Forum

Huge number of devices with nodered via mqtt

General

cymplecy 18 September 2018 07:42 10

We all like someone to solve our problems for us

But you are proposing something that is way beyond most (if not all) people on this forums, use of NodeRED

What sort of project is this BTW - commercial? research?

Simon

Topic		Replies	Views
MQTT/Node Red UI limitations General node-red-dashboard , mqtt	4	634	14 February 2022
How much is big is node-red General	29	941	19 December 2022
Some help and advice please! General	3	311	10 May 2021
Node-red limitations in Big industrial project Industrial	20	4075	5 April 2021
Scalability of Node RED General	3	2480	10 April 2021

Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Huge number of devices with nodered via mqtt

Related topics