Are multiple threads a viable way to profile NodeRED?

General
1

I made a previous thread on profiling and observability in NR (below), but ran into an issue with the single-threaded nature of Node.js. I'm looking for advice on how I might best overcome this in relation to the NodeRED environment.

The problem: I'm using the node-red-contrib-prometheus-exporter (node) - Node-RED node to scrape useful metrics about my Node.js instance and I have custom nodes that run CPU-intensive tasks which occupy the main thread (WebAssembly execution). This means that Prometheus cannot poll my server for metrics when my nodes are executing because the Node.js endpoints become unavailable. Foolish that I didn't see this coming, but there we go. Here are the metrics I'm trying to investigate at the moment:

["nodejs_external_memory_bytes", "nodejs_heap_size_total_bytes", "nodejs_heap_size_used_bytes",
 "process_cpu_seconds_total", "process_cpu_system_seconds_total", "process_cpu_user_seconds_total",
"process_resident_memory_bytes"]

The metrics can be polled again once the main thread is freed up. This is fine for metrics like cpu_seconds where I can take the end value from the start value to find how much was used during execution. Still, I'm keen to track memory usage during WebAssembly execution as it uses memory that is cleared once execution completes.

Proposed solutions:

  1. Make my WebAssembly execution take place on separate threads using Worker threads | Node.js v18.19.0 Documentation, but I'm unsure how this'd affect the order of execution of my nodes.
  2. Try to make the Prometheus metric polling happen on a worker thread, which I think will mean manually instrumenting the Node.js instance.
  3. Measure metrics only at the start and end of execution instead of continuously (current, accidental, implementation - to be avoided if possible)
  4. Move to a different measurement technique, possibly at OS level (Windows) or using the Chrome debugger method Bart explained in my previous thread.

Questions:

  1. Am I overlooking a Node.js metric for memory usage that essentially only counts up? So I can record how much memory is used between the started and end of execution, rather than at a specific instant in time?
  2. Which solution would you recommend and why? I'd like to use 2. manual instrumentation to avoid having to make my nodes multithreaded (so the workflow can remain general to all of NR), but this still wouldn't unblock the main thread so is 4. OS/Chrome level profiling what you'd recommend?

Previous thread:

2

TL;DR: Use the main thread as your monitor and your worker thread(s) for the CPU intensive applications.

I don't know of anything specific in Javascript that would pull this low of a level of system information. Because Javascript is scripted, it will generally have information that can be pulled from environmental variables or through some APIs, but low-level system information, while possible, can be patchy. Then again, I've been proven wrong before so I wouldn't be surprised if there is something you can just grab through a function.

This may be difficult to explain. I think you may be overthinking this, but not because the answer is simple. I think you're overthinking this because you're trying to think of the concurrent solution using sequential thought. Does that make sense? I hope so because it took me a while to word that... :slight_smile:

The worker thread concept was built specifically for situations just like yours. When you need those extra threads to perform certain tasks that need to run separately from the normal thread, you can spin them off and use the extra CPU space to make them work. But in your instance, I wonder if maybe you're using it backwards?

First off, I'm just guessing on your instance, because I don't have the flow to work with. But that's fine. From what you're describing, the reason I wonder if you're using it backwards is because I would think spinning off the CPU intensive tasks FROM the main thread would be more ideal rather than running the CPU intensive tasks IN the main thread. Since you're trying to run a concurrent setup (run something intensive and run something to monitor the intensive task), what you need is a control thread and a separate work thread. The supervisor/laborer relationship. In most supercomputer setups, the main thread is actually the control thread while the CPU intensive tasks that get broadcast to the super part of the computer are the sub-threads to the control thread. Each task or task sub-set is worked on separately by each core and then the results are sent back to the main thread for compilation/validation. What you're describing is doing the opposite. You're tasking your main thread to do the heavy work while the sub-thread is doing the monitoring. While this is easy to build, it's difficult to maintain because when the main thread needs help, it needs to stop its execution of some intensive task to spin off help that it then has to manage, which diverts more CPU time to the managing. If you flip-flop it and have a sub-task do the work, the task gets full CPU attention on the sub-thread while the main thread can use full CPU attention to monitor the sub-thread and spin off any other tasks necessary.

The other thing I think you're not utilizing is the asynchronous nature of Javascript. In order to do what you're describing, you somewhat need a synchronous setup.

  1. Start monitor
  2. Start intensive task
  3. Pause task
  4. Check monitor
  5. Restart task
  6. Pause task
  7. Check monitor
  8. Restart task
    ...
  9. (Thread finishes)
  10. Stop monitor
  11. Tally results

If you were to utilize both the subthread and the asynchronous nature of Javascript, it would look like this:

  1. Start monitor
  2. Spinoff intensive task
  3. Check task
  4. Check task
  5. Check task
    ...
  6. (Receive results finished message)
  7. Stop monitor
  8. Tally results

There's a big difference in how the execution is performed doing it this way. Any CPU intensive task you're doing in your main thread can be spunoff into a subthread and ran there just as easily.

To answer question 2, I would use the worker thread to do your intensive tasks if that's possible and use your main thread to do the monitoring. That should help both significantly.

3

Amazing, thank you for the detailed response. I’m going to look into it now!

4

The only internal memory metric for node.js comes from the V8 javascript engine. Here is something that I use at the beginning of my settings.js to show the starting memory use of node-red. I can then use a similar function elsewhere to show the current memory use at specific points.

/** Optionally display the Node.js/v8 engine heap size on startup */
const v8 = require('v8')
console.info(`V8 Total Heap Size: ${(v8.getHeapStatistics().total_available_size / 1024 / 1024).toFixed(2)} MB`)
let mem = process.memoryUsage()
const formatMem = (m) => ( m/1048576 ).toFixed(2)
console.info(`Initial Memory Use (MB): RSS=${formatMem(mem.rss)}. Heap: Used=${formatMem(mem.heapUsed)}, Tot=${formatMem(mem.heapTotal)}. Ext C++=${formatMem(mem.external)}`)

But note that JavaScript is a garbage-collect type engine which means that the figures can and will change periodically when GC kicks in and free's up memory.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.