HID Joystick / Gamepad inputs

Hi everyone

I've spent a few days looking around for a HID / Joystick / Gamepad input solution for a Linux Debian based system.

There is a kind of joystick emulator for the dashboard, but that doesn't work well on a mobile device and it's just not what I was looking for.

What I wanted was the option to use a physical joystick / gamepad.

For example

  • Main Axis
  • Throttle (resolution of ~64000)
  • Thumb joystick
  • 4 buttons

After many hours of searching, I stumbled across some amazing C code by Jason White

With a little bit of work and some playing around, I now have a FULL PTZ control solution using Visca over IP or RS232 / RS485 / RS422 (any serial output)

The Dashboard UI is nothing to write home about, but really with a Joystick, all that a touch screen needs to provide is a way to select the camera and call presets.
(The bigger plan is to have the camera preselected by whichever one is "in preview" on a vision desk, maybe have an OverRide to control the Live camera)

The following is the Joystick tab, not the PTZ code, if anyone wants the Visca flow, please make a comment here or PM me.

[{"id":"d500de06a0c0486b","type":"exec","z":"e66541cd9d074485","command":"./opt/joystick/joystick /dev/input/js0","addpay":"","append":"","useSpawn":"true","timer":"","winHide":false,"oldrc":false,"name":"","x":660,"y":400,"wires":[["cb77dfef0d6416f5","80278a50dea8e506"],[],["48f0a027df21a672","6b96ba9bc46c219c"]]},{"id":"d87d614be93c7a79","type":"inject","z":"e66541cd9d074485","name":"Start Joystick app","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":270,"y":380,"wires":[["d500de06a0c0486b","f1e0644ab051bc4b"]]},{"id":"5644b13cd10ddaee","type":"inject","z":"e66541cd9d074485","name":"Stop Joystick app","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Now","payloadType":"str","x":300,"y":640,"wires":[["977ad04d20a3cd2f"]]},{"id":"977ad04d20a3cd2f","type":"exec","z":"e66541cd9d074485","command":"sudo pidof joystick","addpay":false,"append":"","useSpawn":"true","timer":"","winHide":false,"oldrc":false,"name":"","x":630,"y":640,"wires":[["5f350b2f32c8643d"],[],[]]},{"id":"5f350b2f32c8643d","type":"exec","z":"e66541cd9d074485","command":"sudo kill","addpay":true,"append":"","useSpawn":"true","timer":"","oldrc":false,"name":"","x":940,"y":620,"wires":[[],[],[]]},{"id":"80278a50dea8e506","type":"function","z":"e66541cd9d074485","name":"Derive Angles etc","func":"// set variables\nvar throttle = 0;\nvar button =[];\n\nvar thumb = [];\nvar pan=0;\nvar tilt =0;\nvar axis0Thresshold = 10;\nvar NS = \"\";\nvar EW = \"\";\nvar y = 0;\nvar x = 0;\n\n\n\n// Not sure if the next bit helps or not\n//  msg.button = \"\";\n//  msg.throttle = \"\";\n//  msg.axis = \"\";\n\n\nvar data = msg.payload.split(\":\");\nmsg.payload = data;\n\nif (msg.payload[0] == \"axis\"){\n    // do stuff with Axis\n\n// Axis 0 - Main movement\n\nif (msg.payload[1]==\"0\"){\n    msg.topic=\"main\";\n    \n    // test to change to numbers\n    msg.payload[2] = Number(msg.payload[2]);\n    msg.payload[3] = Number(msg.payload[3]);\n    \n    \n   // if (Number(msg.payload[2])==0 && Number(msg.payload[3]) ==0){\n    if (msg.payload[2]==0 && msg.payload[3] ==0){\n        \n        msg.topic = \"rested\";\n    }\n    else {\n        msg.topic =\"moving\";\n    }\n    \n// Convert Axis[0] to Compass points or direction for example \"downleft\"\n\n        // North\nif (msg.payload[3]  <= -axis0Thresshold ){\n    NS = \"up\"\n}\n\n        // SOUTH\nif (msg.payload[3]  >= axis0Thresshold ){\n    NS = \"down\"\n}\n\n\n        // EAST\nif (msg.payload[2]  >= axis0Thresshold ){\n    EW = \"right\"\n}\n\n        // WEST\nif (msg.payload[2]  <= -axis0Thresshold ){\n    EW = \"left\"\n}\n\nif (msg.payload[2] == 0 && msg.payload[3] ==0){\n    \n    \n    NS = \"center\";\n    EW = \"\";\n}\n\n\n    \n msg.compass = NS+EW;\n \n // Pan Speed\n pan = msg.payload[2];\n\n \n    if (pan <=0.000001){\n    pan = - pan ;\n \n    }\n    \n         pan = (pan / 32767)*100;\n         \n       msg.pan =  pan.toFixed(0);\n\n    \n \n    \n// Tilt Speed \ntilt = msg.payload[3];\n\n    if (tilt <= 0.000001){\n        \n        tilt = - tilt;\n    }\n\n    tilt = (tilt / 32767) * 100;\n\n     msg.tilt=tilt.toFixed(0);\n \n  //set Degree of main axis\n\n\nif (msg.payload[0] == \"axis\"){\n    \n    msg.payload[2] = Number(msg.payload[2]);\n    msg.payload[3] = Number(msg.payload[3]);\n    \n    x = msg.payload[2];\n    y = msg.payload[3];\n\n\n\n        // Checking if the joystick is at center (0,0) and returns a 'stand still'\n        // command by setting the return to 99999\n        if (x == 0 && y == 0) {\n\n            msg.degree = \"center\";\n\n            // Returns a value based on the quadrant and does some math to deliver the angle\n            // of the coordinates\n        }\n        \n        else if (x >= 0 && y > 0) {\n            // south correct\n            msg.degree =  (90 - (Math.atan(y / x) * 180 / Math.PI));\n\n        } \n        \n        else if (x > 0 && y <= 0) {\n            // correct - East\n            msg.degree =  (90 - (Math.atan(y / x) * 180 / Math.PI));\n\n        }\n        \n        else if (x <= 0 && y < 0) {\n        // north?\n            msg.degree =  (270 - (Math.atan(y / x) * 180 / Math.PI));\n\n        }\n        \n        else if (x < 0 && y >= 0) {\n            // correct West\n            msg.degree =  (270 - (Math.atan(y / x) * 180 / Math.PI));\n\n        } \n        \n        else {\n            msg.degree =  \"ERROR\";\n\n        }\n\n}\n\n\n        // Can't work out why the North position reports 360, when the rest is perfect\nif (msg.degree == 360){\n    msg.degree = 180;\n                    }\n  \n    // end of Axis 0 (Main stick)  \n    \n}\n\n\n // Axis 1 = Throttle & E/W Thumb\n    if (msg.payload[1] == \"1\"){\n        // X value of Axis 1 = Throttle, Convert to Percentage\n        throttle = msg.payload[2];\n      throttle = throttle   - 32767;\n      throttle = -throttle\n      msg.throttlevalue= throttle;\n      throttle = (throttle / 65534)*100;\n        msg.payload[2] = throttle.toFixed(0); // Change this to set decimal point\n        msg.throttle = msg.payload[2];\n       \n       // Y value of Axis 1 = E & W of Thumb \n        thumb[0] = Number(msg.payload[3]);\n          \n                flow.set(\"JSThumbEW\",thumb[0]);\n            thumb[1] = flow.get(\"JSThumbNS\")||0;\n    }\n    \n  // Axis 2 = Thumb N/S  \n  if (msg.payload[1] == \"2\"){\n      \n      // X value is Thumb N/S\n      thumb[0] = flow.get(\"JSThumbEW\")||0;\n      thumb[1] = Number(msg.payload[2])\n      \n            flow.set(\"JSThumbNS\",thumb[1]);\n       \n      \n  }\n    \n// Thumb stop\nif (thumb[0] ==0 && thumb[1]==0){\n    thumb = \"STOP\"; // can be a String or a Boolean\n}\n    \n    \n// End of Axis \"IF\"    \n}\n\n\n\n\n\n//  Button events\n\n\nif (msg.payload[0] == \"button\"){\n    // do stuff with Buttons\n    msg.topic = \"button\";\n    button[0] = Number(msg.payload[1]);\n    button[1] = msg.payload[2];\n    \n    if (msg.payload[2] == \"pressed\"){\n    button[2] = true;}\n    else {\n        button[2] = false;\n    }\n    \n\n\n        flow.set(\"JSbutton\"+msg.payload[1],button[2]); // button[1] for String, button[2] for Boolean\n}\n\nmsg.button = button;\n\nmsg.thumb = thumb;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1110,"y":460,"wires":[["b5349a609d15da2e","713a8098152ca0d6","c137459b95ea9faf","7bd4c5ffb77330a6","3094cdb5f71e9c85","62f7eefc098f7bf0","fe76e69c61cf33a4","bbbd4129bcbd57ef","a3ba16b4fa441e6d"]]},{"id":"b5349a609d15da2e","type":"debug","z":"e66541cd9d074485","name":"Everything","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload","statusType":"msg","x":1470,"y":300,"wires":[]},{"id":"713a8098152ca0d6","type":"debug","z":"e66541cd9d074485","name":"Topic","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"topic","statusType":"msg","x":1450,"y":220,"wires":[]},{"id":"38c6a484edda4978","type":"debug","z":"e66541cd9d074485","name":"Throttle %","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"throttle","statusType":"msg","x":990,"y":80,"wires":[]},{"id":"73153d1ef7353f44","type":"debug","z":"e66541cd9d074485","name":"Button","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"button","statusType":"msg","x":2130,"y":280,"wires":[]},{"id":"c137459b95ea9faf","type":"switch","z":"e66541cd9d074485","name":"Throttle","property":"throttle","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":800,"y":240,"wires":[["934f234194014ac6","38c6a484edda4978","67d7542fba3da305"]]},{"id":"d67f2c7c785313bc","type":"debug","z":"e66541cd9d074485","name":"tilt speed ","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"tilt","statusType":"msg","x":1200,"y":880,"wires":[]},{"id":"ee7e31c3317302ef","type":"debug","z":"e66541cd9d074485","name":"pan speed","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"pan","statusType":"msg","x":1010,"y":1020,"wires":[]},{"id":"7bd4c5ffb77330a6","type":"switch","z":"e66541cd9d074485","name":"Tilt Speed","property":"tilt","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":1050,"y":840,"wires":[["5dcc2cb7b89e1e2f","d67f2c7c785313bc"]]},{"id":"3094cdb5f71e9c85","type":"switch","z":"e66541cd9d074485","name":"Pan Speed","property":"pan","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":870,"y":980,"wires":[["612cda0e7637fd73","ee7e31c3317302ef"]]},{"id":"62f7eefc098f7bf0","type":"switch","z":"e66541cd9d074485","name":"Button?","property":"payload[0]","propertyType":"msg","rules":[{"t":"eq","v":"button","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":1900,"y":420,"wires":[["130ffcb2fc6f166b","73153d1ef7353f44"]]},{"id":"f1e0644ab051bc4b","type":"ui_dropdown","z":"e66541cd9d074485","name":"Joystick App control","label":"Joystick App","tooltip":"","place":"Select option","group":"73c5347dac61dd36","order":26,"width":0,"height":0,"passthru":false,"multiple":false,"options":[{"label":"Start","value":true,"type":"bool"},{"label":"Stop","value":"SIGTERM","type":"str"}],"payload":"","topic":"topic","topicType":"msg","className":"","x":160,"y":540,"wires":[["b616511e0b21e1b9"]]},{"id":"6b96ba9bc46c219c","type":"change","z":"e66541cd9d074485","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.signal","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":140,"y":480,"wires":[["f1e0644ab051bc4b"]]},{"id":"b616511e0b21e1b9","type":"switch","z":"e66541cd9d074485","name":"","property":"payload","propertyType":"msg","rules":[{"t":"true"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":370,"y":540,"wires":[["d500de06a0c0486b"],["977ad04d20a3cd2f"]]},{"id":"fe76e69c61cf33a4","type":"switch","z":"e66541cd9d074485","name":"Thumb?","property":"thumb","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":1340,"y":680,"wires":[["022ef97b770f2dfb","1cca1619e34a3dca","639ec059c9f433b4","36c0b6cae5206384"]]},{"id":"022ef97b770f2dfb","type":"debug","z":"e66541cd9d074485","name":"Thumb","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"thumb","statusType":"msg","x":1780,"y":620,"wires":[]},{"id":"94596b6eb041c562","type":"debug","z":"e66541cd9d074485","name":"Compass points","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"compass","statusType":"msg","x":1640,"y":480,"wires":[]},{"id":"bbbd4129bcbd57ef","type":"switch","z":"e66541cd9d074485","name":"Main Axis Compass","property":"compass","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":1390,"y":480,"wires":[["94596b6eb041c562","d19f7d4c3b395b6c"]]},{"id":"8958bce7aaa238df","type":"comment","z":"e66541cd9d074485","name":"joystick.c source code","info":"The applicatrion that is doing the magic is based on Jason White's code\n\nhttps://gist.github.com/jasonwhite/c5b2048c15993d285130\n\nChanged parts are the final section that formats the output, replacing spaces with \":\" for delimiting\n\n\nVersion used here is\n\n\n/**\n * Author: Jason White\n *\n * Description:\n * Reads joystick/gamepad events and displays them.\n *\n * Compile:\n * gcc joystick.c -o joystick\n *\n * Run:\n * ./joystick [/dev/input/jsX]\n * or to use the default ./joystick\n * \n *\n * See also:\n * https://www.kernel.org/doc/Documentation/input/joystick-api.txt\n */\n#include <fcntl.h>\n#include <stdio.h>\n#include <unistd.h>\n#include <linux/joystick.h>\n\n\n/**\n * Reads a joystick event from the joystick device.\n *\n * Returns 0 on success. Otherwise -1 is returned.\n */\nint read_event(int fd, struct js_event *event)\n{\n    ssize_t bytes;\n\n    bytes = read(fd, event, sizeof(*event));\n\n    if (bytes == sizeof(*event))\n        return 0;\n\n    /* Error, could not read full event. */\n    return -1;\n}\n\n/**\n * Returns the number of axes on the controller or 0 if an error occurs.\n */\nsize_t get_axis_count(int fd)\n{\n    __u8 axes;\n\n    if (ioctl(fd, JSIOCGAXES, &axes) == -1)\n        return 0;\n\n    return axes;\n}\n\n/**\n * Returns the number of buttons on the controller or 0 if an error occurs.\n */\nsize_t get_button_count(int fd)\n{\n    __u8 buttons;\n    if (ioctl(fd, JSIOCGBUTTONS, &buttons) == -1)\n        return 0;\n\n    return buttons;\n}\n\n/**\n * Current state of an axis.\n */\nstruct axis_state {\n    short x, y;\n};\n\n/**\n * Keeps track of the current axis state.\n *\n * NOTE: This function assumes that axes are numbered starting from 0, and that\n * the X axis is an even number, and the Y axis is an odd number. However, this\n * is usually a safe assumption.\n *\n * Returns the axis that the event indicated.\n */\nsize_t get_axis_state(struct js_event *event, struct axis_state axes[3])\n{\n    size_t axis = event->number / 2;\n\n    if (axis < 3)\n    {\n        if (event->number % 2 == 0)\n            axes[axis].x = event->value;\n        else\n            axes[axis].y = event->value;\n    }\n\n    return axis;\n}\n\nint main(int argc, char *argv[])\n{\n    const char *device;\n    int js;\n    struct js_event event;\n    struct axis_state axes[3] = {0};\n    size_t axis;\n\n    if (argc > 1)\n        device = argv[1];\n    else\n        device = \"/dev/input/js0\";\n\n    js = open(device, O_RDONLY);\n\n    if (js == -1)\n        perror(\"Could not open joystick\");\n\n    /* This loop will exit if the controller is unplugged. */\n    while (read_event(js, &event) == 0)\n    {\n        switch (event.type)\n        {\n            case JS_EVENT_BUTTON:\n                printf(\"button:%u:%s:\\n\", event.number, event.value ? \"pressed\" : \"released\");\n                break;\n            case JS_EVENT_AXIS:\n                axis = get_axis_state(&event, axes);\n                if (axis < 3)\n                    printf(\"axis:%zu:%6d:%6d:\\n\", axis, axes[axis].x, axes[axis].y);\n                break;\n            default:\n                /* Ignore init events. */\n                break;\n        }\n        \n        fflush(stdout);\n    }\n\n    close(js);\n    return 0;\n}\n","x":520,"y":300,"wires":[]},{"id":"a26c2d2a68e557be","type":"debug","z":"e66541cd9d074485","name":"Degree","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"degree","statusType":"msg","x":1620,"y":540,"wires":[]},{"id":"a3ba16b4fa441e6d","type":"switch","z":"e66541cd9d074485","name":"Main Axis Degree","property":"degree","propertyType":"msg","rules":[{"t":"nnull"}],"checkall":"true","repair":false,"outputs":1,"x":1390,"y":540,"wires":[["a26c2d2a68e557be"]]},{"id":"67d7542fba3da305","type":"debug","z":"e66541cd9d074485","name":"Throttle full value","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"throttlevalue","statusType":"msg","x":1010,"y":160,"wires":[]},{"id":"73c5347dac61dd36","type":"ui_group","name":"Movement","tab":"c34e359e.fa03f8","order":2,"disp":true,"width":"7","collapse":true},{"id":"c34e359e.fa03f8","type":"ui_tab","name":"PTZ","icon":"dashboard","order":3,"disabled":false,"hidden":false}]

I would have loved to use this joystick, but maybe that is a bit TOO MUCH


Library links
Joystick
https://flows.nodered.org/flow/7d88b8c4a5789bb452365efc606d27a8

VISCA PTZ
https://flows.nodered.org/flow/03fb3e10dded33d788f0990f4915f2c1

Ooh! I like that!! I can think of all sorts of uses for that thing. What it needs though is for someone to hack an ESP32 onto it can have all the controls output to MQTT! :smile_cat:

That's beyond my skills, but it would be very easy to add an SBC with a USB port as a MQTT client.

Jason's C code app makes it incredibly easy.

1 Like

Well, I say it's beyond my skills...

But after looking a little deeper, it seems there is a USB host adapter for an M5Stack, so I guess it's not impossible.

It even suggests it's suitable for a "Flight Stick"

M5Stack USB module

Ah! I have an M5Stack Basic and an Atom. Nice if a little expensive. Certainly well made though. I've picked my M5Stack kit up in various of their sales. I run ESPHome on them. The Basic is a display on my desk with an external light sensor and an internal temperature/humidity sensor (in the stand).

I also have an old Microsoft Flight Sim Joystick. Another project to add to the post-retirement list - 7 years away! :slight_smile:

The USB library should work on other ESP's as well though so the repo should be adaptable.

1 Like

Hmm, I wonder if this may be an option for a blind person to control a dashboard

Are you thinking of a haptic feedback thing?

(Jason's app is Input only)

Interesting idea. Don't know that a standard analogue joystick would be too helpful to someone who is blind but a digital one might be. A gamepad attached to an ESP32 maybe with a loudspeaker for audio. Linked via MQTT.

Now that would be a nice project for someone. :slight_smile:

1 Like

Yes , those are in the lines for what I'm thinking.
I have this thread bookmarked so when I have everything functioning, I can then work on the full accessibility settings.
Also I want to do something really cool. I plan to have a brewer in other countries, to join up with me with my blind friend to brew a beer while everyone is in there area all remotely. I honestly don't think that its ever been done!

2 Likes

Hi @9toejack

I'm not sure if this helps, but Jason has just posted a link to how to send rumble commands back to gamepad.....

In case you are wondering how to send rumble effects to the gamepad, it's done with 5. Force feedback for Linux — The Linux Kernel documentation

via a comment on his Gist

1 Like

It turns out that I've accidently created a single hand solution for selecting, adjusting and cross-fading between multiple PTZ cameras.

(I've only got 1 in stock right now, when I get another couple to hand I'll create a better example)

1 Like