MongoDB node improvements

Core Development Feature Requests
1

Hello everyone,

I've posted about an issue on the mongo node ( Add output on mongodb OUT node to provide status (success/fail) ) provided in the extra nodes.

Basically, the mongo-out doesn't have a way to retrieve when and if an operation is done. A workaround is to use the status emitted by the node, but this cannot be used in conjunction with the http in/out nodes for example (as the msg.req, msg.res... properties are lost).

I've forked the node to add another output, among another improvements. (https://flows.nodered.org/node/node-red-contrib-mongodb). knolleary suggested that I implement those changes in the "official" mongo node.

Here's what I would change :

  • Merge the two nodes (mongo-in and mongo-out) so that every action has an output emitted when the action is done, with the query result (node emits the mongo response object).

  • Emit a result when any operation is done (status only or result)

  • I've also removed the "store whole msg instead of payload only" option for the save/insert actions, I think it's bad practice, I might be wrong ?

  • Not implemented yed, but I would like to make the node try to convert the _id property to ObjectID, as is annoying to do in a function node beforehand every time.

It's a pretty big breaking change (me might not want to merge the two nodes so it stays retro-compatible), so feel free to leave any additions/suggestions/remarks/comments :slight_smile:

2

Overall sounds good. Though I've not used MongoDB for a while - not since I found that it couldn't cope with large datasets on the original Pi and would regularly corrupt the db long before it reached anywhere near the db size it should have been able to reach.

I suspect that is a bit too much of a breaking option and doesn't seem to have any real benefit. If anyone has ever used that feature, you are blocking them from updating without rework.

That would need to be optional I think.

3

Makes sense, you're right.

Also a good suggestion.

4

Is there any other suggestions/remarks ? Should I put up the PR ? :slight_smile:

5

Hi, let me know if I can collaborate on this. Not sure what's still missing. My use case:

  • Update a mongodb collection (stored points in a map) know when the operation ends to refresh the map.

Right now I'm adding a timer to trigger the reload a few seconds after that.

6

Hi albfan,

I've already made the change, the Pull Request is up here :

It's been a while that it's been waiting to be merged : i'm sure they have a lot to do :wink:
Maybe leave a reply on the pull request to bump it and let them know people need the feature

EDIT : Didn't see you already posted a reply on the PR :wink:

7

Sure, hehe.Will push then on PR, sadly the fork model do not allow to push commits on same PR, so if I fix anything just cherry-pick it. (maybe I will PR on your branch to make it easier)

8

Yes - many apologies all round - must admit we've been rather busy (and continue to be) getting 1.0 out the door. That coupled with the fact that we no longer use mongo for anything real means we haven't been paying much attention to it. One thing to note is that we do now have the done() syntax for a node to indicate when it has finished with a message - that can be caught by the complete node - see https://nodered.org/blog/2019/09/20/node-done for details. This is the preferred way for nodes that are logically at the end of a flow to indicate they have completed successfully.

As I recall - at the time the main issue was that the proposed changes completely broke the node - which was unacceptable... I guess they may now be ok - but obviously we need to double check to make sure.