I made a previous thread on profiling and observability in NR (below), but ran into an issue with the single-threaded nature of Node.js. I'm looking for advice on how I might best overcome this in relation to the NodeRED environment.
The problem: I'm using the node-red-contrib-prometheus-exporter (node) - Node-RED node to scrape useful metrics about my Node.js instance and I have custom nodes that run CPU-intensive tasks which occupy the main thread (WebAssembly execution). This means that Prometheus cannot poll my server for metrics when my nodes are executing because the Node.js endpoints become unavailable. Foolish that I didn't see this coming, but there we go. Here are the metrics I'm trying to investigate at the moment:
["nodejs_external_memory_bytes", "nodejs_heap_size_total_bytes", "nodejs_heap_size_used_bytes", "process_cpu_seconds_total", "process_cpu_system_seconds_total", "process_cpu_user_seconds_total", "process_resident_memory_bytes"]
The metrics can be polled again once the main thread is freed up. This is fine for metrics like
cpu_seconds where I can take the end value from the start value to find how much was used during execution. Still, I'm keen to track memory usage during WebAssembly execution as it uses memory that is cleared once execution completes.
- Make my WebAssembly execution take place on separate threads using Worker threads | Node.js v18.12.1 Documentation, but I'm unsure how this'd affect the order of execution of my nodes.
- Try to make the Prometheus metric polling happen on a worker thread, which I think will mean manually instrumenting the Node.js instance.
- Measure metrics only at the start and end of execution instead of continuously (current, accidental, implementation - to be avoided if possible)
- Move to a different measurement technique, possibly at OS level (Windows) or using the Chrome debugger method Bart explained in my previous thread.
- Am I overlooking a Node.js metric for memory usage that essentially only counts up? So I can record how much memory is used between the started and end of execution, rather than at a specific instant in time?
- Which solution would you recommend and why? I'd like to use 2. manual instrumentation to avoid having to make my nodes multithreaded (so the workflow can remain general to all of NR), but this still wouldn't unblock the main thread so is 4. OS/Chrome level profiling what you'd recommend?