MCU plugin tooling works from a shell, but not from the plugin sidebar

General
,
1

I have been following along with great interest as @phoddie and @ralphwetzel develop bridges between Node-RED flows and MCU programming -- one of the most exciting new areas for me to extend my smart-home skills. As someone who has never worked this close to printed circuits and sensors, I am so grateful for the commitment you have to this community... and to @BartButenaers for his detailed tutorials, especially this one, which has put me very close to having my own ESP32 development environment. :partying_face:

Similar to Bart, I chose to install node-red-mcu-plugin on my RPi 4 (although mine is NOT headless). The MCU I'm using is the Espressif-WRover-E, and my O/S is Ubuntu Jammy Jellyfish:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

After many attempts to get ESP-IDF installed and running, I am finally able to initiate a Build from the MCU Plugin sidebar --

image
image823Ă—833 53.5 KB

which cannot complete due to these errors:

Starting build process...
Host system check: #41-Ubuntu SMP PREEMPT Fri Sep 8 12:38:39 UTC 2023
MCU Build system check: p1.4.1-beta.1 + #548dc37 @ m4.2.1-11-gcdcab49
HOME directory check: /home/srickus
Creating build environment for platform esp32.
Working directory: /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
$IDF_PATH is defined: /home/srickus/develop/esp32/esp-idf
$IDF_PYTHON_ENV_PATH is defined: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
> cd /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
UPLOAD_PORT = /dev/ttyUSB1
Creating build script file...
> /bin/bash ./build.sh
>> source /home/srickus/develop/esp32/esp-idf/export.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
WARNING: No IDF variables to remove from environment found. Deactivation of previous esp-idf version was not successful.
Adding ESP-IDF tools to PATH...
ERROR: tool xtensa-esp32-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s2-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s3-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool openocd-esp32 has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp-rom-elfs has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
>> IDF_PYTHON_ENV_PATH: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
>> mcconfig -d -x localhost:5004 -l -m -p esp32
# warning: -l deprecated. use -dl instead.!
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
WARNING: No IDF variables to remove from environment found. Deactivation of previous esp-idf version was not successful.
ERROR: tool xtensa-esp32-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s2-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s3-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool openocd-esp32 has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp-rom-elfs has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.

And yes, I have run that install.sh script many times... The strange thing is that this build.sh script does work when I invoke it from a shell:

srickus@ shrpi4: ~/.node-red/mcu-plugin-cache/sqd9j8rnmna
$ /bin/bash ./build.sh
>> source /home/srickus/develop/esp32/esp-idf/export.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Constraint file: /home/srickus/develop/esp32/esp-idf/espidf.constraints.v5.1.txt
Requirement files:
 - /home/srickus/develop/esp32/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python
Python requirements are satisfied.
Added the following directories to PATH:
  /home/srickus/develop/esp32/esp-idf/components/espcoredump
  /home/srickus/develop/esp32/esp-idf/components/partition_table
  /home/srickus/develop/esp32/esp-idf/components/app_update

Detected installed tools that are not currently used by active ESP-IDF version.
For removing old versions of test_mkdfu, catch, docker, ci, test_idf_py, test_apps, gen_soc_caps_kconfig, unit-test-app, esp_prov, templates, test_sbom, __pycache__, mocks, esp_app_trace, ldgen, test_build_system, test_idf_tools, idf_py_actions, mass_mfg, cmake, kconfig_new, ble, requirements, test_mkuf2 use command 'python /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall'
To free up even more space, remove installation packages of those tools. Use option 'python3 /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall --remove-archives'.

Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

>> IDF_PYTHON_ENV_PATH: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
>> mcconfig -d -x localhost:5004 -l -m -p esp32
# warning: -l deprecated. use -dl instead.!
/home/srickus/develop/esp32/esp-idf/tools/idf.py
Using recommended ESP-IDF v5.1.1
# nodered2mcu flows
# xsc Resource.xsb
# xsc time.xsb
# xsc timer.xsb
# xsc deepEqual.xsb
...
# xsc flows.xsb
# Reconfiguring ESP-IDF...
Executing action: reconfigure
Running cmake in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python -DESP_PLATFORM=1 -DSDKCONFIG_DEFAULTS=/home/srickus/develop/moddable/build/tmp/esp32//debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc -DSDKCONFIG_HEADER=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/config/sdkconfig.h -DCMAKE_MESSAGE_LOG_LEVEL=ERROR -DDEBUGGER_SPEED=460800 -DIDF_TARGET=esp32 -DESP32_SUBCLASS=esp32 -DCCACHE_ENABLE=0 /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32"...
Loading defaults file /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
# cc xsAll.c (strings in flash)
# cc xsAPI.c (strings in flash)
...
# cc xsDebug.c (strings in flash)
# cc xsDate.c (strings in flash)
# cc xsDataView.c (strings in flash)
# ld xs_esp32.bin
Executing action: all (aliases: build)
Running cmake in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python -DESP_PLATFORM=1 -DmxDebug=1 -DINSTRUMENT=1 -DTMP_DIR=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna -DCMAKE_MESSAGE_LOG_LEVEL=ERROR -DDEBUGGER_SPEED=460800 -DESP32_SUBCLASS=esp32 -DSDKCONFIG_DEFAULTS=/home/srickus/develop/moddable/build/tmp/esp32//debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc -DSDKCONFIG_HEADER=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/config/sdkconfig.h -DCCACHE_ENABLE=0 /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32"...
Loading defaults file /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Running ninja in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "ninja all"...
[5/887] Generating ../../partition_table/partition-table.binPartition table binary generated. Contents:
*******************************************************************************
# ESP-IDF Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,24K,
phy_init,data,phy,0xf000,4K,
factory,app,factory,0x10000,4032K,
*******************************************************************************
[250/887] Performing configure step for 'bootloader'-- Found Git: /usr/bin/git (found version "2.34.1") 
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
-- Project sdkconfig file /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/sdkconfig
Compiler supported targets: xtensa-esp32-elf

-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of time_t
-- Check size of time_t - done
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/soc/esp32/ld/esp32.peripherals.ld
-- App "bootloader" version: v5.1.1
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.api.ld
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/bootloader/subproject/main/ld/esp32/bootloader.ld
-- Adding linker script /home/srickus/develop/esp32/esp-idf/components/bootloader/subproject/main/ld/esp32/bootloader.rom.ld
-- Components: bootloader bootloader_support efuse esp_app_format esp_common esp_hw_support esp_rom esp_system esptool_py freertos hal log main micro-ecc newlib partition_table soc spi_flash xtensa
-- Component paths: /home/srickus/develop/esp32/esp-idf/components/bootloader /home/srickus/develop/esp32/esp-idf/components/bootloader_support /home/srickus/develop/esp32/esp-idf/components/efuse /home/srickus/develop/esp32/esp-idf/components/esp_app_format /home/srickus/develop/esp32/esp-idf/components/esp_common /home/srickus/develop/esp32/esp-idf/components/esp_hw_support /home/srickus/develop/esp32/esp-idf/components/esp_rom /home/srickus/develop/esp32/esp-idf/components/esp_system /home/srickus/develop/esp32/esp-idf/components/esptool_py /home/srickus/develop/esp32/esp-idf/components/freertos /home/srickus/develop/esp32/esp-idf/components/hal /home/srickus/develop/esp32/esp-idf/components/log /home/srickus/develop/esp32/esp-idf/components/bootloader/subproject/main /home/srickus/develop/esp32/esp-idf/components/bootloader/subproject/components/micro-ecc /home/srickus/develop/esp32/esp-idf/components/newlib /home/srickus/develop/esp32/esp-idf/components/partition_table /home/srickus/develop/esp32/esp-idf/components/soc /home/srickus/develop/esp32/esp-idf/components/spi_flash /home/srickus/develop/esp32/esp-idf/components/xtensa
-- Configuring done
-- Generating done
-- Build files have been written to: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/bootloader
[103/104] Generating binary image from built executableesptool.py v4.7.dev2
Creating esp32 image...
Merged 1 ELF section
Successfully created esp32 image.
Generated /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/bootloader/bootloader.bin
[104/104] cd /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/x.../build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/bootloader/bootloader.binBootloader binary size 0x46f0 bytes. 0x2910 bytes (37%) free.
[763/887] Building C object esp-idf/console/CMakeFiles/__idf_console.dir/esp_console_repl.c.obj/home/srickus/develop/esp32/esp-idf/components/console/esp_console_repl.c:479:13: warning: 'esp_console_repl_task' defined but not used [-Wunused-function]
  479 | static void esp_console_repl_task(void *args)
      |             ^~~~~~~~~~~~~~~~~~~~~
/home/srickus/develop/esp32/esp-idf/components/console/esp_console_repl.c:375:18: warning: 'esp_console_common_init' defined but not used [-Wunused-function]
  375 | static esp_err_t esp_console_common_init(size_t max_cmdline_length, esp_console_repl_com_t *repl_com)
      |                  ^~~~~~~~~~~~~~~~~~~~~~~
/home/srickus/develop/esp32/esp-idf/components/console/esp_console_repl.c:354:18: warning: 'esp_console_setup_history' defined but not used [-Wunused-function]
  354 | static esp_err_t esp_console_setup_history(const char *history_path, uint32_t max_history_len, esp_console_repl_com_t *repl_com)
      |                  ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/srickus/develop/esp32/esp-idf/components/console/esp_console_repl.c:329:18: warning: 'esp_console_setup_prompt' defined but not used [-Wunused-function]
  329 | static esp_err_t esp_console_setup_prompt(const char *prompt, esp_console_repl_com_t *repl_com)
      |                  ^~~~~~~~~~~~~~~~~~~~~~~~
[886/887] Generating binary image from built executableesptool.py v4.7.dev2
Creating esp32 image...
Merged 25 ELF sections
Successfully created esp32 image.
Generated /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/xs_esp32.bin
[887/887] cd /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/x...elop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/xs_esp32.binxs_esp32.bin binary size 0x11e730 bytes. Smallest app partition is 0x3f0000 bytes. 0x2d18d0 bytes (72%) free.

Project build complete. To flash, run this command:
/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python ../../../../../../../esp32/esp-idf/components/esptool_py/esptool/esptool.py -p (PORT) -b 460800 --before default_reset --after hard_reset --chip esp32  write_flash --flash_mode dio --flash_size 4MB --flash_freq 80m 0x1000 build/bootloader/bootloader.bin 0x8000 build/partition_table/partition-table.bin 0x10000 build/xs_esp32.bin
or run 'idf.py -p (PORT) flash'
Executing action: flash
Serial port /dev/ttyUSB1
Connecting.....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Running ninja in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "ninja flash"...
[1/5] cd /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/esp-idf/esptool_py && /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python /home/srickus/develop/esp32/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/partition_table/partition-table.bin /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/xs_esp32.bin
xs_esp32.bin binary size 0x11e730 bytes. Smallest app partition is 0x3f0000 bytes. 0x2d18d0 bytes (72%) free.
[2/5] Performing build step for 'bootloader'
[1/1] cd /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/bootloader/esp-idf/esptool_py && /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python /home/srickus/develop/esp32/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x1000 /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/bootloader/bootloader.bin
Bootloader binary size 0x46f0 bytes. 0x2910 bytes (37%) free.
[2/3] cd /home/srickus/develop/esp32/esp-idf/components/esptool_py && /usr/bin/cmake -D IDF_PATH=/home/srickus/develop/esp32/esp-idf -D "SERIAL_TOOL=/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python;;/home/srickus/develop/esp32/esp-idf/components/esptool_py/esptool/esptool.py;--chip;esp32" -D "SERIAL_TOOL_ARGS=--before=default_reset;--after=hard_reset;write_flash;@flash_args" -D WORKING_DIRECTORY=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build -P /home/srickus/develop/esp32/esp-idf/components/esptool_py/run_serial_tool.cmake
esptool.py --chip esp32 -p /dev/ttyUSB1 -b 115200 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 4MB 0x1000 bootloader/bootloader.bin 0x10000 xs_esp32.bin 0x8000 partition_table/partition-table.bin
esptool.py v4.7.dev2
Serial port /dev/ttyUSB1
Connecting....
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 54:43:b2:dc:8d:1c
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash will be erased from 0x00001000 to 0x00005fff...
Flash will be erased from 0x00010000 to 0x0012efff...
Flash will be erased from 0x00008000 to 0x00008fff...
Compressed 18160 bytes to 12653...
Writing at 0x00001000... (100 %)
Wrote 18160 bytes (12653 compressed) at 0x00001000 in 1.4 seconds (effective 106.4 kbit/s)...
Hash of data verified.
Compressed 1173296 bytes to 753124...
Writing at 0x00010000... (2 %)
Writing at 0x0001a24a... (4 %)
Writing at 0x000223b7... (6 %)
Writing at 0x0002bac6... (8 %)
Writing at 0x000355b9... (10 %)
...
Writing at 0x0011e32d... (95 %)
Writing at 0x00123a46... (97 %)
Writing at 0x00128c12... (100 %)
Wrote 1173296 bytes (753124 compressed) at 0x00010000 in 66.6 seconds (effective 141.0 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 83...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (83 compressed) at 0x00008000 in 0.1 seconds (effective 400.9 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Done

Sorry for the lengthy post, but I'm hoping someone has seen this error, maybe?
__
Steve

2

Hi @shrickus
I've no access to my development system (for this week) - thus guessing now: This looks like install.sh defines some environment variables that are not available in the build environment that the plugin creates. Each of those needed to be forwarded individually. It might be that this is an effect induced by the Moddable SDK changing to IDF5 recently. I'll keep you posted in a few days - after further analysis.

3

Thanks Ralph... no real hurry here, since I'm able to get around it by using the shell.

I did find someone else having the same problem -- here is a link to the github issue he created.

It seems to be related to differences in the IDF_PATH env variable used to clone the esp-idf repo vs. that used to run the export.sh script. AFAIK, these should be the same for my setup. So I'm not sure how that applies to my case, unless the plugin sidebar is using a different path than where I cloned/installed the esp-idf code?

In case it matters, here is the environment file I have defined in my ~/.node-red directory:

srickus@ shrpi4: ~/.node-red
$ cat environment 
MODDABLE=/home/srickus/develop/moddable
ESP_IDF_VERSION=5.1
IDF_PATH=/home/srickus/develop/esp32/esp-idf
IDF_TOOLS_PATH=/home/srickus/develop/esp32/esp-idf/tools

My understanding is that this will be passed to the runtime env for the node-red service, right?

4

Well I've verified that this is not just an esp-idf version 5 error -- after cloning v4.4 and rebuilding everything I'm still not able to run the Build tools inside the node-red-mcu-plugin sidebar.

Looks like my pool controller project is dead in the water - hahaha

But seriously, I'm not able to understand how to get a usable node-red flow deployed to my ESP-32 mcu -- mostly because I don't understand how to use GPIO, I guess. So here are some MCU things that are not working for me (and I'm not always sure why):

  • The clock node throws an exception that some real-time clock module is not available (so I guess my ESP-32 does not have an RTC?)
  • the mqtt-out node was included in the mcu flow code, but the global mqtt-config node that goes with it did NOT get built into the mcu code
  • so I created a separate mqtt-config node, assigned it to the same MCU flow (tab), and now it throws a different exception, because the generated code cannot find its config node (returns undefined?)
  • according to the docs, "If the "rpi-gpio in" node from node-red-node-pi-gpio is used in flows, it is translated to a Digital In node." -- however this did not work for me, so I had to switch to the MCU digital in node
  • I've used a couple nodes to try and connect the ESP-32 through an RS-422/485 module to the remote control port on my pool controller -- honestly, I just have no idea where to start with the wiring or comm protocols necessary (even after reading through these tutorials and watching many videos)

Sorry for all the griping -- at some point it will probably make sense to me. All I can offer right now is a dysfunctional system for debugging purposes, if that helps anybody?

5

I could try to gide you through this jungle! :wink:

One first comment regarding your last post:

For node-red-mcu, I never needed to clone anything from IDF (after I installed it once & as long as it stays in the family, e.g. ESP32) or take care of the type of MCU I'm currently targeting. The required includes are managed (to 100%) by the Moddable SDK; you only need to nominate the target platform ... and the rest is done for you.

Even if you're targeting different families (e.g. Pico & ESP) in parallel, the plugin together w/ the SDK does all the heavy stuff for you...

6

Hi Steve!

I've pushed a small change to the repo that should - once run - show us the difference in the env variable setting - either from within the plugin or when you run the script from a terminal.

Please scan the beginning of the log for the following sequence & post your debug status here - for both cases!

UPLOAD_PORT = /dev/cu.usbserial-210
IDF_TOOLS_EXPORT_CMD = undefined
IDF_PYTHON_ENV_PATH = undefined
OPENOCD_SCRIPTS = undefined
ESP_IDF_VERSION = undefined

Thank you in advance!

7

Do you need a RTC? The current time could be updated via ntp as well - assuming your MCU has wireless access.

I checked this, yet cannot confirm.
There's no mqtt-config thing on my systems. mqtt-out uses a config node called mqtt-broker, and this one is included in the exported flow. Do you use another mqtt - node?

EDIT:

I just checked this; cannot confirm as well.
It's nodered2mcu.js, kind of a preprocessor in the Moddable SDK, that does the flip when launching the build process. Given your working directory is

Working directory: /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna

and

MODDABLE=/home/srickus/develop/moddable

goto

/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/modules/flows.js

for verification. There you'll find the (extremely optimized) source code that's finally build for the MCU.

8

Thanks Ralph,

I have installed the latest, and am still having the same issue -- but the new logs show which of the env vars are undefined:

Starting build process...
Host system check: #45-Ubuntu SMP PREEMPT Thu Oct 12 09:20:16 UTC 2023
MCU Build system check: p1.4.1-beta.1 + #548dc37 @ m4.3.0-26-g27d2260
HOME directory check: /home/srickus
Creating build environment for platform esp32.
Working directory: /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
$IDF_PATH is defined: /home/srickus/develop/esp32/esp-idf
$IDF_PYTHON_ENV_PATH is defined: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
> cd /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
UPLOAD_PORT = /dev/ttyUSB1
IDF_TOOLS_EXPORT_CMD = undefined
IDF_PYTHON_ENV_PATH = /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
OPENOCD_SCRIPTS = undefined
ESP_IDF_VERSION = undefined
Creating build script file...
> /bin/bash ./build.sh
>> source /home/srickus/develop/esp32/esp-idf/export.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
WARNING: No IDF variables to remove from environment found. Deactivation of previous esp-idf version was not successful.
Adding ESP-IDF tools to PATH...
ERROR: tool xtensa-esp-elf-gdb has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf-gdb has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s2-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s3-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp32ulp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool openocd-esp32 has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp-rom-elfs has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
>> IDF_PYTHON_ENV_PATH: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
>> mcconfig -d -x localhost:5004 -l -m -p esp32
# warning: -l deprecated. use -dl instead.!
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
WARNING: No IDF variables to remove from environment found. Deactivation of previous esp-idf version was not successful.
ERROR: tool xtensa-esp-elf-gdb has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf-gdb has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s2-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool xtensa-esp32s3-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool riscv32-esp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp32ulp-elf has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool openocd-esp32 has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.
ERROR: tool esp-rom-elfs has no installed versions. Please run '/home/srickus/develop/esp32/esp-idf/install.sh' to install it.

Those same vars from my build env are defined as follows:

$ env | egrep '(IDF|OCD)'
IDF_PATH=/home/srickus/develop/esp32/esp-idf
ESP_IDF_VERSION=5.1
IDF_TOOLS_PATH=/home/srickus/develop/esp32/esp-idf

So is this just a matter of adding the missing vars?

9

It's not that simple - unfortunately.
ESP-IDF - per default - just needs IDF_PATH and does the rest on its own. The more of these env vars you define (additionally) the higher the risc for a mis-configuration thus a bail out.
In this case the issue is that the plugin transfers IDF_PYTHON_ENV_PATH into it's build environment, but not ESP_IDF_VERSION. The solution is rather "not to transfer IDF_PYTHON_ENV_PATH" than to add more env vars.

I'll push this to the repo in a minute. Feedback welcome.

10

I am able to re-create the plugin errors in my build environment by NOT setting the env var IDF_TOOLS_PATH -- so apparently the export.sh script cannot figure that out?

Once I run this in my shell, the build.sh script starts working...

export IDF_TOOLS_PATH=$IDF_PATH
11

Per default, it's IDF_TOOLS_PATH_DEFAULT = os.path.join('~', '.espressif').

Yours once it is

and in another post

Any reason you're playing around with this setting? Should only be necessary if you instructed install.sh to use a different-than-default-path when installing the tools... :thinking:

Anyway: It's a valid option to define a different IDF_TOOLS_PATH. I made a small modification that should pull this into the build env. Is it getting better now... ?

12

Yep, it's now invoking the build.sh script from the plugin -- thanks! Must be because I'm using a non-standard location for the tools? Anyway, you are correct in that the env var points to the 'tools' directory:

So now I can work on getting past the next set of errors:

Starting build process...
Host system check: #45-Ubuntu SMP PREEMPT Thu Oct 12 09:20:16 UTC 2023
MCU Build system check: p1.4.1-beta.1 + #548dc37 @ m4.3.0-26-g27d2260
HOME directory check: /home/srickus
Creating build environment for platform esp32.
Working directory: /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
$IDF_PATH is defined: /home/srickus/develop/esp32/esp-idf
$IDF_TOOLS_PATH is defined: /home/srickus/develop/esp32/esp-idf
> cd /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
UPLOAD_PORT = /dev/ttyUSB1
IDF_TOOLS_EXPORT_CMD = undefined
IDF_PYTHON_ENV_PATH = undefined
OPENOCD_SCRIPTS = undefined
ESP_IDF_VERSION = undefined
Creating build script file...
> /bin/bash ./build.sh
>> source /home/srickus/develop/esp32/esp-idf/export.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Python requirements are satisfied.
Constraint file: /home/srickus/develop/esp32/esp-idf/espidf.constraints.v5.1.txt
Requirement files:
 - /home/srickus/develop/esp32/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python
Added the following directories to PATH:
  /home/srickus/develop/esp32/esp-idf/components/espcoredump
  /home/srickus/develop/esp32/esp-idf/components/partition_table
  /home/srickus/develop/esp32/esp-idf/components/app_update
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp-elf-gdb/12.1_20221002/xtensa-esp-elf-gdb/bin
  /home/srickus/develop/esp32/esp-idf/tools/riscv32-esp-elf-gdb/12.1_20221002/riscv32-esp-elf-gdb/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32s2-elf/esp-12.2.0_20230208/xtensa-esp32s2-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32s3-elf/esp-12.2.0_20230208/xtensa-esp32s3-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/riscv32-esp-elf/esp-12.2.0_20230208/riscv32-esp-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/openocd-esp32/v0.12.0-esp32-20230419/openocd-esp32/bin
  /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin
  /home/srickus/develop/esp32/esp-idf/tools

Detected installed tools that are not currently used by active ESP-IDF version.
For removing old versions of test_mkdfu, catch, tools, docker, ci, test_idf_py, test_apps, gen_soc_caps_kconfig, unit-test-app, esp_prov, templates, test_sbom, __pycache__, mocks, esp_app_trace, ldgen, test_build_system, test_idf_tools, idf_py_actions, mass_mfg, dist, cmake, kconfig_new, ble, requirements, test_mkuf2 use command 'python /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall'
To free up even more space, remove installation packages of those tools. Use option 'python3 /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall --remove-archives'.

Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

>> IDF_PYTHON_ENV_PATH: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
>> mcconfig -d -x localhost:5004 -l -m -p esp32
# warning: -l deprecated. use -dl instead.!
/home/srickus/develop/esp32/esp-idf/tools/idf.py
# Reconfiguring ESP-IDF...
# nodered2mcu flows
Using recommended ESP-IDF v5.1.1
# xsc main.xsb
### Error: mqtt broker id "3a17ec55.a70c14" not found
Error translating node ID "b783d4199a70d2b9, type "mqtt out", name "RPi-4 MQTT": Error: mqtt broker id "3a17ec55.a70c14" not found
make: *** [/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/makefile:970: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/modules/flows.js] Error 1
make: *** Waiting for unfinished jobs....
Executing action: reconfigure
Running cmake in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python -DESP_PLATFORM=1 -DSDKCONFIG_DEFAULTS=/home/srickus/develop/moddable/build/tmp/esp32//debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc -DSDKCONFIG_HEADER=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/config/sdkconfig.h -DCMAKE_MESSAGE_LOG_LEVEL=ERROR -DDEBUGGER_SPEED=460800 -DIDF_TARGET=esp32 -DESP32_SUBCLASS=esp32 -DCCACHE_ENABLE=0 /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32"...
Loading defaults file /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build

Right again -- the config node is the core mqtt-broker, defined outside of the flows. From the error listed above, it seems the mqtt out node's reference to the broker id cannot be found?

This is the optimized flows.js code -- notice that the broker id 3a17ec55.a70c14 is not included in the code:

$ cat /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/modules/flows.js
// auto-generated by nodered2mcu on Fri Nov 17 2023 13:04:45 GMT-0500

import "nodered"

function build(flows, createFlow, createNode) {
        let flow, node, nodes;
        flow = createFlow("__config", "");
        flow = createFlow("dd38a67fb0a6b1d0", "ESP32 WROVER");
        createNode("inject", "100daa3a86dad1cb", "", flow);
        createNode("debug", "4272788766d7aa52", "Analog 4", flow);
        createNode("debug", "9d920756d3ee6ea1", "Timestamp", flow);
        createNode("mqtt out", "b783d4199a70d2b9", "RPi-4 MQTT", flow);
        createNode("mcu_analog", "ded448ddbf38cf87", "Analog - Pin 4", flow);
        createNode("mcu_digital_in", "e6eedce4b081e2f6", "GPIO: Pin 12", flow);
        createNode("mqtt-broker", "2fdc761efdac55e7", "mosquito@192.168.1.4", flow);

        flows = flows.values();
        flow = flows.next().value;      // __config

        nodes = flow.nodes();
        flow = flows.next().value;      // dd38a67fb0a6b1d0

        nodes = flow.nodes();
        node = nodes.next().value;      // inject - 100daa3a86dad1cb
        node.onStart({
                wires: [["9d920756d3ee6ea1","ded448ddbf38cf87"]],
                trigger: function () {
                        const msg = {};
                        msg.payload = Date.now();
                        msg.topic = "WROVER";
                        this.send(msg);
                },
        });

        node = nodes.next().value;      // debug - 4272788766d7aa52
        node.onStart({
                active: true,
                tosidebar: false,
                console: false,
                tostatus: true,
                complete: "payload",
                targetType: "msg",
                statusVal: function (msg) {
                        return msg.payload;
                },
                statusType: "auto",
                getter: function (msg) {
                        return msg.payload;
                },
        });

        node = nodes.next().value;      // debug - 9d920756d3ee6ea1
        node.onStart({
                active: true,
                tosidebar: false,
                console: false,
                tostatus: true,
                complete: "payload",
                targetType: "msg",
                statusVal: function (msg) {
                        return msg.payload;
                },
                statusType: "auto",
                getter: function (msg) {
                        return msg.payload;
                },
        });

        node = nodes.next().value;      // mqtt out - b783d4199a70d2b9
        node.onStart({
                topic: "WROVER/GPIO/18",
                qos: 0,
                respTopic: "",
                contentType: "",
                userProps: "",
                correl: "",
                expiry: "",
                broker: "2fdc761efdac55e7",
        });

        node = nodes.next().value;      // mcu_analog - ded448ddbf38cf87
        node.onStart({
                pin: 4,
                wires: [["4272788766d7aa52"]],
        });

        node = nodes.next().value;      // mcu_digital_in - e6eedce4b081e2f6
        node.onStart({
                pin: 12,
                mode: "Input",
                edge: 3,
                debounce: 25,
                wires: [["b783d4199a70d2b9"]],
        });

        node = nodes.next().value;      // mqtt-broker - 2fdc761efdac55e7
        node.onStart({
                broker: "192.168.1.4",
                port: "1883",
                clientid: "",
                autoConnect: true,
                protocolVersion: "4",
                keepalive: 60000,
                cleansession: true,
                autoUnsubscribe: true,
                birthTopic: "",
                birthQos: "0",
                birthRetain: "false",
                birthPayload: "",
                birthMsg: {},
                closeTopic: "",
                closeQos: "0",
                closeRetain: "false",
                closePayload: "",
                closeMsg: {},
                willTopic: "",
                willQos: "0",
                willRetain: "false",
                willPayload: "",
                willMsg: {},
                userProps: "",
                sessionExpiry: "",
                credentials: {},
        });

}

export default Object.freeze({
        build,
}, true);
13

Yes, it is not.
And - according to the data - it isn't necessary either ... as mqtt out b783d4199a70d2b9 references mqtt-broker 2fdc761efdac55e7 ... which is present.

Could you please post

  • an export of your flow (JSON) as Node-RED creates it.
  • the flows.json that is generated in /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna.

There's definitely something ... interesting ... happening!

14

Ok, I've narrowed the flows down to the bare minimum, and I'm still seeing the same error -- mqtt broker id "3a17ec55.a70c14" not found

Starting build process...
Host system check: #45-Ubuntu SMP PREEMPT Thu Oct 12 09:20:16 UTC 2023
MCU Build system check: p1.4.1-beta.1 + #548dc37 @ m4.3.0-26-g27d2260
HOME directory check: /home/srickus
Creating build environment for platform esp32.
Working directory: /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
$IDF_PATH is defined: /home/srickus/develop/esp32/esp-idf
$IDF_TOOLS_PATH is defined: /home/srickus/develop/esp32/esp-idf
> cd /home/srickus/.node-red/mcu-plugin-cache/sqd9j8rnmna
UPLOAD_PORT = /dev/ttyUSB1
IDF_TOOLS_EXPORT_CMD = undefined
IDF_PYTHON_ENV_PATH = undefined
OPENOCD_SCRIPTS = undefined
ESP_IDF_VERSION = undefined
Creating build script file...
> /bin/bash ./build.sh
>> source /home/srickus/develop/esp32/esp-idf/export.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Python requirements are satisfied.
Constraint file: /home/srickus/develop/esp32/esp-idf/espidf.constraints.v5.1.txt
Requirement files:
 - /home/srickus/develop/esp32/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python
Added the following directories to PATH:
  /home/srickus/develop/esp32/esp-idf/components/espcoredump
  /home/srickus/develop/esp32/esp-idf/components/partition_table
  /home/srickus/develop/esp32/esp-idf/components/app_update
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp-elf-gdb/12.1_20221002/xtensa-esp-elf-gdb/bin
  /home/srickus/develop/esp32/esp-idf/tools/riscv32-esp-elf-gdb/12.1_20221002/riscv32-esp-elf-gdb/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32s2-elf/esp-12.2.0_20230208/xtensa-esp32s2-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/xtensa-esp32s3-elf/esp-12.2.0_20230208/xtensa-esp32s3-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/riscv32-esp-elf/esp-12.2.0_20230208/riscv32-esp-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /home/srickus/develop/esp32/esp-idf/tools/openocd-esp32/v0.12.0-esp32-20230419/openocd-esp32/bin
  /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin
  /home/srickus/develop/esp32/esp-idf/tools

Detected installed tools that are not currently used by active ESP-IDF version.
For removing old versions of test_mkdfu, catch, tools, docker, ci, test_idf_py, test_apps, gen_soc_caps_kconfig, unit-test-app, esp_prov, templates, test_sbom, __pycache__, mocks, esp_app_trace, ldgen, test_build_system, test_idf_tools, idf_py_actions, mass_mfg, dist, cmake, kconfig_new, ble, requirements, test_mkuf2 use command 'python /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall'
To free up even more space, remove installation packages of those tools. Use option 'python3 /home/srickus/develop/esp32/esp-idf/tools/idf_tools.py uninstall --remove-archives'.

Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

>> IDF_PYTHON_ENV_PATH: /home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env
>> mcconfig -d -x localhost:5004 -l -m -p esp32
# warning: -l deprecated. use -dl instead.!
/home/srickus/develop/esp32/esp-idf/tools/idf.py
Using recommended ESP-IDF v5.1.1
# nodered2mcu flows
# xsc Resource.xsb
# xsc time.xsb
# xsc timer.xsb
# xsc deepEqual.xsb
# xsc modules.xsb
# xsc structuredClone.xsb
# xsc hex.xsb
# xsc socket.xsb
# xsc net.xsb
# xsc sntp.xsb
# xsc wifi.xsb
# xsc preference.xsb
# xsc nodered.xsb
# xsc main.xsb
# xsc base64.xsb
# xsc mqttnodes.xsb
# xsc analog.xsb
# xsc digital.xsb
# xsc setup/network.xsb
# xsc commodetto/Bitmap.xsb
# xsc ~.embedded/io/analog.xsb
# xsc ~.embedded/io/digital.xsb
# xsc ~.embedded/io/digitalbank.xsb
# xsc ~.embedded/io/i2c.xsb
# xsc ~.embedded/implementation/i2csync.xsb
# xsc ~.embedded/io/smbus.xsb
# xsc ~.embedded/implementation/smbussync.xsb
# xsc ~.embedded/io/pwm.xsb
# xsc ~.embedded/io/serial.xsb
# xsc ~.embedded/io/spi.xsb
# xsc ~.embedded/io/socket/tcp.xsb
# xsc ~.embedded/io/socket/udp.xsb
# xsc ~.embedded/io/socket/listener.xsb
# xsc system.xsb
# xsc ~.embedded/provider/builtin.xsb
# xsc ~.embedded/io/pulsecount.xsb
# xsc ~.embedded/io/pulsewidth.xsb
# xsc setup/timezone.xsb
# xsc wifi/connection.xsb
# xsc ~.embedded/network/dns/resolver/udp.xsb
# xsc dns/parser.xsb
# xsc dns/serializer.xsb
# xsc dns.xsb
# xsc mqttclient/config.xsb
# xsc ~.embedded/network/mqtt/client.xsb
# xsc mc/config.xsb
# xsid Resource.c.xsi
# xsid modTime.c.xsi
# xsid modTimer.c.xsi
# xsid timer.c.xsi
# xsid modPreference.c.xsi
# xsid modInstrumentation.c.xsi
# xsid modLwipSafe.c.xsi
# xsid structuredClone.c.xsi
# xsid deepEqual.c.xsi
# xsid modules.c.xsi
# xsid modHex.c.xsi
# xsid modNet.c.xsi
# xsid modResolve.c.xsi
# xsid modSocket.c.xsi
# xsid modsntp.c.xsi
# xsid xs6wifi.c.xsi
# xsid modSPI.c.xsi
# xsid builtinCommon.c.xsi
# xsid digitalbank.c.xsi
# xsid _i2c.c.xsi
# xsid spi.c.xsi
# xsid modGPIO.c.xsi
# xsid nodered.c.xsi
# xsid modBase64.c.xsi
# xsid moddnsparser.c.xsi
# xsid commodettoBitmap.c.xsi
# xsid _analog.c.xsi
# xsid _pwm.c.xsi
# xsid serial.c.xsi
# xsid tcp.c.xsi
# xsid udp.c.xsi
# xsid system.c.xsi
# xsid _pulsecount.c.xsi
# xsid _pulsewidth.c.xsi
# xsid modTimer.h.xsi
# xsid modPreference.h.xsi
# xsid modInstrumentation.h.xsi
# xsid modLwipSafe.h.xsi
# xsid modSocket.h.xsi
# xsid modSPI.h.xsi
### Error: mqtt broker id "3a17ec55.a70c14" not found
# xsid builtinCommon.h.xsi
# xsid modGPIO.h.xsi
# xsid commodettoBitmap.h.xsi
# Reconfiguring ESP-IDF...
Error translating node ID "b783d4199a70d2b9, type "mqtt out", name "RPi-4 MQTT": Error: mqtt broker id "3a17ec55.a70c14" not found
make: *** [/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/makefile:970: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/modules/flows.js] Error 1
make: *** Waiting for unfinished jobs....
Executing action: reconfigure
Running cmake in directory /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/home/srickus/develop/esp32/esp-idf/python_env/idf5.1_py3.10_env/bin/python -DESP_PLATFORM=1 -DSDKCONFIG_DEFAULTS=/home/srickus/develop/moddable/build/tmp/esp32//debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc -DSDKCONFIG_HEADER=/home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build/config/sdkconfig.h -DCMAKE_MESSAGE_LOG_LEVEL=ERROR -DDEBUGGER_SPEED=460800 -DIDF_TARGET=esp32 -DESP32_SUBCLASS=esp32 -DCCACHE_ENABLE=0 /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32"...
Loading defaults file /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/sdkconfig.mc...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/srickus/develop/moddable/build/tmp/esp32/debug/sqd9j8rnmna/xsProj-esp32/build

This is the flows.json file as created by node-red:

[
    {
        "id": "dd38a67fb0a6b1d0",
        "type": "tab",
        "label": "ESP32 WROVER",
        "disabled": false,
        "info": "",
        "env": [],
        "_mcu": {
            "mcu": true
        }
    },
    {
        "id": "0d66edcee19b33b9",
        "type": "tab",
        "label": "Pool Sensors",
        "disabled": false,
        "info": "",
        "env": [],
        "_mcu": {
            "mcu": false
        }
    },
    {
        "id": "3a17ec55.a70c14",
        "type": "mqtt-broker",
        "name": "mosquito@shrpi4",
        "broker": "192.168.1.4",
        "port": "1883",
        "clientid": "",
        "autoConnect": true,
        "usetls": false,
        "compatmode": false,
        "protocolVersion": "4",
        "keepalive": "60",
        "cleansession": true,
        "autoUnsubscribe": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "birthMsg": {},
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "closeMsg": {},
        "willTopic": "",
        "willQos": "0",
        "willPayload": "",
        "willMsg": {},
        "userProps": "",
        "sessionExpiry": ""
    },
    {
        "id": "100daa3a86dad1cb",
        "type": "inject",
        "z": "dd38a67fb0a6b1d0",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "*/1 0-23 * * *",
        "once": false,
        "onceDelay": 0.1,
        "topic": "WROVER",
        "payload": "",
        "payloadType": "date",
        "_mcu": {
            "mcu": true
        },
        "x": 160,
        "y": 80,
        "wires": [
            [
                "9d920756d3ee6ea1",
                "ded448ddbf38cf87"
            ]
        ]
    },
    {
        "id": "4272788766d7aa52",
        "type": "debug",
        "z": "dd38a67fb0a6b1d0",
        "name": "Analog 4",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "_mcu": {
            "mcu": true
        },
        "x": 620,
        "y": 140,
        "wires": []
    },
    {
        "id": "9d920756d3ee6ea1",
        "type": "debug",
        "z": "dd38a67fb0a6b1d0",
        "name": "Timestamp",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "_mcu": {
            "mcu": true
        },
        "x": 630,
        "y": 80,
        "wires": []
    },
    {
        "id": "b783d4199a70d2b9",
        "type": "mqtt out",
        "z": "dd38a67fb0a6b1d0",
        "name": "RPi-4 MQTT",
        "topic": "WROVER/GPIO/18",
        "qos": "0",
        "retain": "",
        "respTopic": "",
        "contentType": "",
        "userProps": "",
        "correl": "",
        "expiry": "",
        "broker": "3a17ec55.a70c14",
        "_mcu": {
            "mcu": true
        },
        "x": 630,
        "y": 220,
        "wires": []
    },
    {
        "id": "ded448ddbf38cf87",
        "type": "mcu_analog",
        "z": "dd38a67fb0a6b1d0",
        "name": "Analog - Pin 4",
        "pin": "4",
        "resolution": "",
        "moddable_manifest": {
            "include": "$(NODEREDMCU)/nodes/mcu/analog/manifest.json"
        },
        "_mcu": {
            "mcu": true
        },
        "x": 420,
        "y": 140,
        "wires": [
            [
                "4272788766d7aa52"
            ]
        ]
    },
    {
        "id": "e6eedce4b081e2f6",
        "type": "mcu_digital_in",
        "z": "dd38a67fb0a6b1d0",
        "name": "GPIO: Pin 12",
        "pin": "12",
        "mode": "Input",
        "edge": "3",
        "debounce": "25",
        "initial": false,
        "invert": false,
        "moddable_manifest": {
            "include": "$(NODEREDMCU)/nodes/mcu/digital/manifest.json"
        },
        "_mcu": {
            "mcu": true
        },
        "x": 410,
        "y": 220,
        "wires": [
            [
                "b783d4199a70d2b9"
            ]
        ]
    },
    {
        "id": "8f344ada70985e50",
        "type": "mqtt in",
        "z": "0d66edcee19b33b9",
        "name": "WROVER/#",
        "topic": "WROVER/#",
        "qos": "0",
        "datatype": "auto-detect",
        "broker": "3a17ec55.a70c14",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "_mcu": {
            "mcu": false
        },
        "x": 150,
        "y": 120,
        "wires": [
            [
                "5847c0bd9e9021c4"
            ]
        ]
    },
    {
        "id": "5847c0bd9e9021c4",
        "type": "debug",
        "z": "0d66edcee19b33b9",
        "name": "Pool readings",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "_mcu": {
            "mcu": false
        },
        "x": 560,
        "y": 120,
        "wires": []
    }
]

The flows.js file is NOT generated successfully, due to the missing broker it seems?!?

15

On a whim, I stopped node-red, edited the flows.json to remove the . from the broker id, and restarted node-red... AND NOW THE MCU BUILD IS WORKING! :partying_face:

Older versions of node-red generated node ids matching XXXXXXXX.XXXXXX but it seems that changed since I started building my flows last year... now the new node ids are always a 16-digit hex string.

So whatever code it is that tries to look up nodes by id, apparently it does not support the old 8X.6X format. Any idea why the id string format would matter during the node lookup?

16

Great catch. Indeed the plugin (!) didn't export a config node with an id in 8.6 notation. I'm pretty sure, we had a similar issue about a year ago ... but apparently not considered to check for that case.

A fix is already in the repo.

17

Thanks again, Ralph...

FYI, I did a quick search of my existing flows, and found some node ids are as short as 12 characters! I've added a comment to the latest commit, suggesting a regex for validating the id strings. I also verified that it matches all the old and new style ids in my flows.

1 Like
18

Already implemented!
Thank you once again for your proposal. :+1:

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.