Clarify - Time Series Visualization

Despite all the enthusiasm shown here my main worry about a cloud solution to show my personal data is that at some point it will all be behind a Paywall.

1 Like

I've just tried inviting an existing user to join my team and get the same error message.

Just been trying out some of the 'calculation' features in Clarify and just as an experiment thought I'd try calculating the cost of my instantaneous power consumption for electricity. OK - I know it's not accurate as I need to take power-factor correction in to account, but hey... this is an experiment.

Once I had done that I thought... it would be great if I could send a value for the cost of my kWh so the results plotted by Clarify reflect my current tariff. My current tariff is 20.49p going to to 28p in April !!!!!
I added this piece of code to the code I posted earlier in this thread.

msg.topic = 'kwh_price';
msg.signal = {name: 'kwh_price', type: 'numeric', description: 'Home energy readings'};
msg.payload = {times: [date_now], values: [20.49]};
node.send(msg);

Here are some screen-shots of how things look in Clarify.
Screen Shot 02-06-22 at 09.14 AM

Screen Shot 02-06-22 at 09.19 AM
In the formula 'a' is voltage, 'b' is kWh price and 'c' is current.


The graph shows, the cost of my power consumption at 09:04am (one single point in time) was 39 pence.

I believe my energy supplier has an API that provides all sorts of details. I suspect price of electricity is one of them, so it should be an easy task to obtain this value and make my code even more accurate.

1 Like

Not sure that the calculation is correct though, because the energy is measured in kWh, and you are using instantaneous readings, so there is no time element in the amount of power being consumed to calculate energy consumption.
I think that you would need to use node-RED to feed the power (voltage x current) through the watt2kwh node (shameless plug!), which then give you the energy consumed to use in the cost calculation.

OR if your voltage/current data arrives at regular intervals, say every 5 seconds? then you could build that time period into your energy calculation.

PS The resulting numbers will be extremely small - 0.00000102 Cost £ !

1 Like

If you get something like a PZEM 016 100A power monitor and a PZCT-02 current clamp you can very cheaply get surprisingly accurate instantaneous power readings.

This is the device I'm currently using... bought it about 3-years ago. It's in a nice plastic box so is fairly safe as the mains connection is protected.
It connects to a Wemos D1 Mini flashed with Tasmota. MQTT to Node-RED.

The calculations are useful for converting degC to degF (to keep my OH happy!)

formula

As it's weather data (that does not change that quickly) I'm also using node-red-contrib-aggregator to create a 10 minute average to update the charts, so I'm not needlessly hammering the api, instead of posting the sensor data every 10 seconds.

chart4

I'll pass those comments on to the product team. I know we have a lot planned on the visualization part going forward, so this is great input!
I'm not reading anything as negative, it's feedback and comparison, that's how we evolve.

I find it super exciting to see how you have been able to get up and running on Clarify in such a short time. There are a lot of features that might be hard to spot right away, but it seems you've found many of them. We are soon releasing our new developer/documentation pages where one will find updated guides and tutorials for the product. They will replace https://docs.clarify.io , but on the same address!

2 Likes

That's a totally valid point @ghayne and I feel the same about a lot of my own data. Data ownership and the freedom to choose the best solution have been a guiding star for us when developing Clarify. That's why we provide APIs where you can get all your data out in the full resolution. We have made a promise to keep the free plan free for all that have signed up for it, but if and when one chooses to leave our product there will never be any data lockup.

Our reaction will always be:
We will be sorry to see you leave, but will gladly let you export all that belongs to you for easy transfer to the time-series database of your liking.

Best regards
Bernt-Johan

3 Likes

That is a very reassuring policy! Thank you for explaining.

Just found out that there is a Clarify app in the Google play store (don't know about the apple store?).

It makes the charts easy to access, zoom & pinch etc. on a mobile device.

1 Like

Great that you found the Android App. We do have a native app for iOS as well.
The mobile app makes it really easy to get access to the data on the fly, as well as commenting, adding pictures etc. An image is priceless when looking back trying to remember why something happened..

When adding multiple users to your organization it's easy to tag them with @username anywhere in the timeline.

Another piece of the jigsaw falls into place. I've written some code to obtain my electricity tariff (kWh) on a daily basis, round it up to 2 decimal places and use it in the formatted readings I send to Clarify.

2 Likes

Finally got around to adding my power/diverter data to Clarify!

5 Likes

Hello @bbergshaven,

That is very kind of your company to support our community in this way!!

Normally I prefer to have all my data local, but in this case I have made an exception. I don't have enough free time currently to setup (and maintain) my own local timeseries database, and a visualisation tool like e.g. Grafana. So in this case a tool like Clarify is a real blessing.

With good assistance from @Paul-Reed and @dynamicdave I managed to have Clarify up and running for a few sensors in very short amount of time. I would never have been able to accomplish the same result in another way...

Below you can find a list of things that might be improved. Please consider it as constructive feedback, because the tool is already very nice. But with some adjustments, I might have been able to install it even more quickly, without the help from Paul and David ...

  1. I could not get my own flow working, because I got the message

    image

    The problem was that my topic contained "SENSOR_KITCHEN", but it seems that Clarify doesn't like uppercase characters. When I changed it to "sensor_kitchen" everything was solved. It would be nice if you could make it somehow a bit more clear why it fails (in the node status, in the info panel, ...). Because it causes a lot of headache if you don't know what is going on.

  2. It was not very clear to me whether I had to create manually (in the Clarify web interface) items and signals, or that those were created automatically when injecting a new topic. Perhaps you can add a some extra Inject nodes in your example flow, to demonstrate the different types of input messages (one with msg.signal, one without, and so on...). And add some comment nodes in your example flow, where you give some basis explanation (and what you need to do to setup everything). That way you immediately get an idea of all the available message possibilies, without having to read through the documentation.

  3. Although the initial setup of the json token is rather simple, it was not clear to me in the beginning where I had to get the token. I was wondering if you could e.g. add a button to the config node, to automate this a bit more. I mean to get a token somehow after entering credentials. Not sure if that is doable....

  4. My first sensors that I have published in Clarify are reed relays. So they have binary values, which indicate the status of some doors (0=open and 1=closed). Currently the lines between the points in the charts are interpolated, which is not correct in my state. Indeed I would like to see only "steps". Would be nice if this could it would be customizable in the future, how the lines between the points are being drawn:

    image

  5. In the above discussion you say that it is possible to zoom in on a chart via touchpad gestures. Indeed in the Android app this works fine. But I can't get it working via the touchpad on my Windows 10 portable. So perhaps you can add some other ways to achieve this, for folks that have already passed their midlife crisis :wink:

I am very pleased that - thanks to Clarify - I now finally have historical data available for my sensors.

Kudos :pray:
Bart

1 Like

Thank you @BartButenaers for the kind words. I've been a part of the Node-Red community for quite some time and I really appreciate being able to give something back.

It's great that you have been able to get started with Clarify in such a short time, and that there seems to be forming a small Clarify community within Node-Red. Community support is always welcomed :slight_smile:

Our CTO and I would love to do an AMA (ask me anything), webinar, or similar at some time where we could gather interested people from the community and go deeper into questions about Clarify.

I love constructive feedback and will give a short comment on your points below.

  1. I'll make sure that we improve the error messages displayed. We're also discussing to be more liberal on what we accept as inputID.

  2. We're releasing a new documentation portal this week that will include a bit more material to explain the differences between signals and items etc. Getting a grip on our nomenclature is maybe the hardest thing starting with Clarify, so we need to Clarify this a bit :wink:

  3. We can add a bit more documentation. It should be as easy as just adding the credentials.json file via the upload button and deploying the flow. Token fetching is automatic, but the flow needs to be deployed before the fetch can happen. Credentials are unique to each integration, so adding a button to automatically get the credentials file would be hard.

  4. I've forwarded the request for customizable drawing of graphs to our product team. For state values (enums) as you have, I can recommend trying changing the Item type to enum and adding the necessary states. (see attached image). It will then draw as a status bar instead of a graph.

  5. I'll pass this on to team-product as well. I saw one other user mentioning the same issue with a windows touchpad and he resolved it by updating the drivers. I have ordered a new windows laptop for our frontend team to test if they are able to reproduce the issues. If you could DM me some more info about your laptop type etc. that would be helpful :slight_smile:

Thank you so much for taking the time to write such thorough feedback :raised_hands:

Best regards
Bernt-Johan

1 Like

@bbergshaven,
Very, very nice. That screenshot of yours looks really good for my use case! I have updated my types, and my timeline look now indeed better.

Little remark about this. It was not quite clear to me what I had to change to accomplish this:

  1. I have updated the type to 'enumeration' on my binary signals (and specified all the keys and labels).
  2. I have updated the type to 'enumeration' on my binary items (and specified all the keys and labels).
  3. I have removed my timelines and created new ones.

Is that the normal way of working, that you need to specify the enum values both on items and signals? Perhaps that is somewhere explained in the docs, but it might be useful to show a popup - e.g. when you change the signal type - of other things you need to do to.

Well you are the expert, so no arguments against that. A list of steps in the info panel of the config node could be a great help too:

It is a normal Acer Windows 10 portable. However @Paul-Reed has explained me that I can zoom in/out in the charts, by swiping two fingers up or down on the trackpad. But every time I forget which gesture is for zooming-in and zooming-out, so every time I do it wrong. Looks my ancient brain is not able to coordinat gestures :wink: I am from the generation that tries the + and - buttons for zooming...

I have another question. I wanted tonight to store the values of a new sensor in Clarify. I was in a hurry (like always) so didn't had a look a my flow from last weekend, and I simply injected my messages (with temperature value in the payload and a lowercase topic). But the node status showed that the value was incorrect. After about 2 minutes I realized that I had to convert my messages to a format that the Clarify node accepts. So I was wondering if it would be possible to do this conversion somehow automatically within the Clarify node. For example when the payload is a number, that the clarify node automaticaly adds a timestamp. That way I can keep my flows simple.

BTW for other people reading this: once you have completed the initial setup, you can very easily (in a few minutes) adjust your flow to store data from an extra sensor into Clarify. I am very pleased with this way of working, because it saves me tons of time.

BTW For those who are wondering: I am not the main shareholder of Clarify :wink:

Thanks again!!

I just found this thread after trying out Clarify with Node-RED as a datasource.

First of all, I am amazed about Clarify, especially how easy it is to set it up.

However, as a longtime Grafana user, I quickly identified a crucial feature that I am missing:

As @BartButenaers already mentioned, only linear interpolation is available for the plots inside the graphs. That might be fine for data with a fixed sampling rate. But in a report by exception (rbe) scenario, where the data is only transmitted if the value has changed, linear interpolation is not suitable. It could be that there is no value change for a certain signal over a long period of time, therefore there are no datapoints available for that certain signal over that time period. If then the value changes after it has not changed for a long period of time, then simply drawing a line between those two datapoints does not reflect the true waveform of that signal. This can create a lot of confusion for the user.
That is why I only use "step after" as a line interpolation mode for rbe data within Grafana.

In other words: You guys from Clarify have my vote for the feature request "step after interpolation mode" :wink:

1 Like