How to execute a nodejs file inside function node

General
1

Hi!

Sorry if it's a stupid question, I am new to node-red.

Trying to execute this code inside a function node:

"use strict";

const { KafkaStreams } = require("./../index.js");
const { nativeConfig: config } = require("./../test/test-config.js");

const kafkaStreams = new KafkaStreams(config);
const stream = kafkaStreams.getKStream("test12345");

kafkaStreams.on("error", (error) => {
    console.log("Error occured:", error.message);
});

console.log("Started");

stream
    .mapJSONConvenience()
    .mapWrapKafkaValue()
    .filter((rec) => rec != null)
    .tap((msg) => console.log(msg.node.ipaddr))
    .wrapAsKafkaValue()
    .to("output-topic-12", 1, "buffer");

stream.start();

What do I need to change in order to be able to execute this code inside a function node?
I was able to do it via a exec node, but want to set certain parameters inside the code and eventually create a kafka streams node.

2

Before implementing it yourself in a function node as code, I would take a look in the Flow Library for existing nodes to your problem. That way it can be integrated much better into a flow-based environment like Node-RED. :grinning:

As it turns out, there are some Kafka nodes available: https://flows.nodered.org/search?term=Kafka

3

Aside from the good point that @kuema makes about checking for existing nodes, on the more specific point about the Function node, I recommend reading the docs https://nodered.org/docs/user-guide/writing-functions

In particular, the section on loading additional modules as you cannot use require in the Function node directly - https://nodered.org/docs/user-guide/writing-functions#loading-additional-modules

2 Likes
4

And, if you really wanted to implement it yourself inside a Function node, error handling becomes really important. For example, an unhandled error emitted by a stream (or any EventEmitter) will crash the whole NodeJS process! :boom:

5

Thanks! There is a lot of kafka nodes already, but unfortunetly no kafka streams node. I will keep in mind the error handling :slight_smile:

6

Well, you can always implement one that can. Or, even better, take one of the existing ones and extend it. It isn't that hard if you already know your way around NodeJS. :slightly_smiling_face:

After all, doing this kind of reusable "low-level" stuff in a custom node makes it much easier to use in Node-RED and improves the ecosystem around it as well.

2 Likes
7

This is on a slightly different topic, but I ran across this thread when looking for people who had tried what I was about to try: node streams in Node-RED. I didn't really find anything more on it, so I've done my own thing; I've just published my first nodes so I guess I'm now in very early beta, but I'd love some input!

The existing project is gulpetl, and we're adding Node-RED capabilities to a project already built around streams. It's for ETL but not Kafka, but we're doing some of the same type of things. Anyway, we're headed toward 'reusable "low-level" stuff in a custom node', so hopefully we'll be improving the ecosystem a little bit. :grinning: But given what @kuema said, I feel like I must have missed some nodes that are using streams. Are there really some more out there?

Anyway, here's the new Node-RED section of our docs, with Getting Started and what-not. I hope to get a brief tour finished up in the next couple of days.