HTTP POST content too large?

General
1

I am trying out surrealDB (this is a fantastic NewSQL type of database project which supports live data streaming to websockets when tables change, among many other unique features like build-in rest/rpc/graphql, schemaless/schemaful while having similar SQL language, joins are no longer needed, really awesome crazy stuff, watch this short intro).

I think this DB will become very popular in the near future and would a perfect companion for node-red.

Anyway, when I do a POST via curl:

root@oracle:~# DATA="INFO FOR DB;"
curl -vv --request POST \
        --header "Content-Type: application/json" \
        --user "root:root" \
        --data "${DATA}" \
        http://10.0.0.224:8000/sql

...
* Server auth using Basic with user 'root'
> POST /sql HTTP/1.1
> Host: 10.0.0.224:8000
> Authorization: Basic cm9vdDpyb290
> User-Agent: curl/7.74.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 12
>
* upload completely sent off: 12 out of 12 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< version: surreal-1.0.0-beta.7
< server: SurrealDB
< content-length: 75    
< date: Sun, 11 Sep 2022 06:23:01 GMT
<
* Connection #0 to host 10.0.0.224 left intact
[{"time":"15.842µs","status":"ERR","detail":"Specify a namespace to use"}]

This is a correct response.

Via node-red I receive an http 405:

injecting msg.payload:INFO FOR DB; into a http request node configured as POST with basic auth:

{
  "_msgid": "f91be86db477e2cb",
  "payload": {
    "code": 405,
    "details": "Request content length too large",
    "description": "The requested http method is not allowed for this resource. Refer to the documentation for allowed methods."
  },
  "topic": "",
  "statusCode": 405,
  "headers": {
    "content-type": "application/json",
    "version": "surreal-1.0.0-beta.7",
    "server": "SurrealDB",
    "content-length": "181",   
    "date": "Sun, 11 Sep 2022 05:55:45 GMT",
    "x-node-red-request-node": "73741076"
  },
  "responseUrl": "http://10.0.0.224:8000/sql",
  "redirectList": [],
  "retry": 0
}

Does node-red underwater send more than i see ?

1 Like
2

Resolved: had to set an explicit Content-Type:application/json header.

3

It certainly does look interesting.

If you (or anyone is interested) there is also an NodeJS NPM module : Node.js | SurrealDB Docs

Here is a demo...

chrome_XhsrVMNNsR

[{"id":"4b4cf3ac570e1cfa","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"connect","x":2320,"y":100,"wires":[["f5680ee059365d25"]]},{"id":"f5680ee059365d25","type":"function","z":"10ece3ab98e93683","name":"surrealDB","func":"let db = context.get(\"db\")\n\nif (msg.topic === \"connect\") {\n    db = await init()\n    context.set(\"db\", db)\n    return\n}\n\nif (!db) {\n    node.warn(\"DB not connected\");\n    return null\n}\n\nif (msg.topic === \"close\") {\n    await db.close()\n    db = null\n    context.set(\"db\", null)\n    return\n}\n\nif(msg.topic) {\n    msg.payload = await query(msg.topic, msg.payload)\n    node.send(msg)\n} else {\n    msg.payload = await testquery1()\n    node.send(msg) \n}\n\n\nasync function init() {\n    const db = new Surreal('http://127.0.0.1:8000/rpc');\n    try {\n        \n        // Signin as a namespace, database, or root user\n        await db.signin({\n            user: 'root',\n            pass: 'root',\n        });\n\n        // Select a specific namespace / database\n        await db.use('test', 'test');\n        return db\n\n    } catch(err) {\n        node.error(err, msg);\n    }\n}\n\nasync function testquery1() {\n    try {\n        // Perform a custom advanced query\n        return await db.select(\"person\")\n    } catch (e) {\n        node.error(e, msg);\n    }\n} \nasync function testquery2() {\n    try {\n        // Perform a custom advanced query\n        return await db.query('SELECT marketing, count() FROM type::table(tb) GROUP BY marketing', {\n            tb: 'person',\n        });\n    } catch (e) {\n        node.error(e, msg);\n    }\n} \nasync function query(sql, payload) {\n    const operation = ['select', 'create', 'update', 'change', 'modify', 'delete'].indexOf(sql) > -1 ? sql : 'sql'\n\n\n    // db.query(sql, vars)\tRuns a set of SurrealQL statements against the database\n    // db.select(thing)\tSelects all records in a table, or a specific record\n    // db.create(thing, data)\tCreates a record in the database\n    // db.update(thing, data)\tUpdates all records in a table, or a specific record\n    // db.change(thing, data)\tModifies all records in a table, or a specific record\n    // db.modify(thing, data)\tApplies JSON Patch changes to all records in a table, or a specific record\n    // db.delete(thing)\tDeletes all records, or a specific record\n\n    try {\n        switch (operation) {\n            case \"sql\":\n                if (payload) {\n                    return await db.query(sql, payload);\n                }\n                return await db.query(sql);\n            case \"create\":\n            case \"update\":\n            case \"change\":\n            case \"modify\":\n                if (typeof payload !== \"object\" || !payload.thing || !payload.data) {\n                    node.error(\"exepected a payload object with properties '.thing' aand '.data'\", msg);\n                }\n                return await db[operation](payload.thing, payload.data)\n            case \"select\":\n            case \"delete\":\n                return await db[operation](typeof payload === \"object\" ? payload.thing : payload)\n        }\n    } catch (e) {\n        node.error(e, msg);\n    }\n} \n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"Surreal","module":"surrealdb.js"}],"x":2600,"y":260,"wires":[["d66f06ab9ed1191e"]]},{"id":"eb06b5242cd934cb","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"close","x":2310,"y":140,"wires":[["f5680ee059365d25"]]},{"id":"8ee7c3e1c8b86774","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"select * from person","x":2350,"y":180,"wires":[["f5680ee059365d25"]]},{"id":"3f8236b3afb200ce","type":"inject","z":"10ece3ab98e93683","name":"create person:katie","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"create","payload":"{\"thing\":\"person:katie\",\"data\":{\"title\":\"promotions manager\",\"name\":{\"first\":\"katie\",\"last\":\"mac\"},\"marketing\":true,\"engineering\":false}}","payloadType":"json","x":2350,"y":280,"wires":[["f5680ee059365d25"]]},{"id":"4e0ba175eabfbed4","type":"inject","z":"10ece3ab98e93683","name":"change \"person:steve\"","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"change","payload":"{\"thing\":\"person:steve\",\"data\":{\"marketing\":true,\"engineering\":false,\"title\":\"promotions director\"}}","payloadType":"json","x":2360,"y":340,"wires":[["f5680ee059365d25"]]},{"id":"54728f84fde9a7dc","type":"inject","z":"10ece3ab98e93683","name":"delete \"person:steve\"","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"delete","payload":"person:steve","payloadType":"str","x":2360,"y":500,"wires":[["f5680ee059365d25"]]},{"id":"961af2939949a6d4","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"select","payload":"person","payloadType":"str","x":2330,"y":400,"wires":[["f5680ee059365d25"]]},{"id":"706d2622f18eb978","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"select","payload":"person:steve","payloadType":"str","x":2350,"y":440,"wires":[["f5680ee059365d25"]]},{"id":"4fc5157ca504fdc9","type":"inject","z":"10ece3ab98e93683","name":"create person:steve","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"create","payload":"{\"thing\":\"person:steve\",\"data\":{\"title\":\"developer\",\"name\":{\"first\":\"steve\",\"last\":\"mac\"},\"marketing\":false,\"engineering\":true}}","payloadType":"json","x":2350,"y":240,"wires":[["f5680ee059365d25"]]},{"id":"6d9a07bba1c68a8f","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"delete","payload":"person","payloadType":"str","x":2330,"y":540,"wires":[["f5680ee059365d25"]]},{"id":"d66f06ab9ed1191e","type":"debug","z":"10ece3ab98e93683","name":"debug 78","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":2620,"y":340,"wires":[]}]
4

Thanks, yes it is very cool, especially when you look at the "join" concept with relations.

eg

SELECT * from persons fetch jobdescriptions;

which is like a join, but a lot simpler.
or when you work with intermediary tables;

SELECT <-purchased<-person->purchased->product FROM product:laptop;

is like a 2-way binding.

I was thinking like (very simplistic...) - push everything (?) into a db and you have can live streaming data without the need for any other sources, realtime pushed into node-red and anywhere else. Having rest/rpc/graphql endpoints available directly can make moving data across very simple. (ie. feed it into grafana for example).

This is also awesome:

SELECT * FROM http::get('https://surrealdb.com');

you can actually do SQL on remote json objects :')

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.