# Scraping Voltaware meter data?

**URL:** <https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144>\
**Category:** General\
**Created:** [30 August 2022 18:43 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144 "2022-08-30T18:43:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mart](https://avatars.discourse-cdn.com/v4/letter/m/e47774/32.png) [@Mart](https://discourse.nodered.org/u/Mart)\
**Post date:** [30 August 2022 18:43 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144/1 "2022-08-30T18:43:35Z")

</div>

I want to poll a Voltaware monitor that’s measuring power consumption in my house. I can use a browser to view the data direct from the monitor (image below). I thought I could “just” use an “http request” node in Node-RED and parse the html UTF-8 response. However, the payload is gobbledegook - example below? What am I doing wrong?

 ![measurement](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/4/44c19d58a22c3394d89f8d2ce08960ea72ef902b.jpeg)

��&�b�meterdata.html�VMs�6��W �E� �t�C’9�q�ine;�LO�%�����\_�)%�d�b���[`�vW�W������ZTdt~6���L�oB�4柝&X�Gqo���AB?w�� �����% ��ߒ���Y�T����DHg -CW��*+q�$���PV�� �_���4f���a�P�S� �� ��R 5{��1�O3Ԅ�ϟ��U�0ie�G�%-0T�|���\<K�B�2���_��P�&��eI��G6����a�UM"xYe�닯!�'�n�8��K` �q�%���_�`x6w�\<��\g��2KjX�,~�η�+�R(�_@\>,�kl9S&�\*�K���!e�4���܉)�D��h�AR�Xv���� |֮t��q�[��ʀ����.�ȝ͒�zIJ��2U��6˔]�?ٕ7��w{Hؼ�� qO����f�j�3R��ԫE� p�J+���I~�M�pć�%\>^�:��r�=i���V{��������a��Sܤ�~b%Zy�t�/�}/� h�} ���V^�̃�/y�8Q��m�����B�Dž�w�k��-Z�2��%\�m�A\<��Y+^y` �K�[����\_�;�e�eҧ�k��W��u���k�e�o3�L���ߒ6��;K��:���B�[u��fT�#���.u�al��i9���4x��N�\>��nt�x%7�`4�/�|�˻!���7Kqm�/�b4=g���0�2����we#�a��!�KIQ�[�/C��v�F"  
Q�Wj�B)q����=r;�����;F\<�!����P,�a�f,’��.�k޽Ͻ�1w��5�C^���݀�e�s��ņ�3

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [30 August 2022 19:09 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144/2 "2022-08-30T19:09:48Z")

</div>

What headers does it send, it is probably compressed.  
this post may help

> [@Http request node - gzip encrypted response](https://discourse.nodered.org/t/http-request-node-gzip-encrypted-response/30300/15):
>
> Adding a node-red-contrib-gzip node to the flow, plus a json node, and the output is correctly formed json!! Thanks @BartButenaers [output]

---

<div class="post-metadata">

**Author:** ![Mart](https://avatars.discourse-cdn.com/v4/letter/m/e47774/32.png) [@Mart](https://discourse.nodered.org/u/Mart)\
**Post date:** [31 August 2022 08:50 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144/3 "2022-08-31T08:50:46Z")

</div>

Brilliant - found the gzip node, added it, changed the http request node to return a binary buffer ... and the gzip node produced recognizable html.

I have a debug node connected - but it only shows the first 28 lines of the web page...

I've added a write file node and it outputs the full html for the page OK.

The html creates a table to show data but there's no data for me to extract in the html itself - looks like the page uses some javascript routines to populate the data e.g.

```
<script src="jquery.js"></script>
<script src="jquery.mobile.js"></script>
<script src="common.js"></script>
<script src="meterdata.js"></script>

```

Not sure where to go next - do I need to run the javascript somehow to read the data?

---

<div class="post-metadata">

**Author:** ![Mart](https://avatars.discourse-cdn.com/v4/letter/m/e47774/32.png) [@Mart](https://discourse.nodered.org/u/Mart)\
**Post date:** [31 August 2022 11:36 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144/4 "2022-08-31T11:36:12Z")

</div>

Had a rummage around in the javascript and there was another URL that returns the data in json format.

I can now poll the Voltaware monitor and get “live” power consumption data - just what I wanted!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [30 October 2022 11:36 UTC](https://discourse.nodered.org/t/scraping-voltaware-meter-data/67144/5 "2022-10-30T11:36:42Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.