Elasticsearch Alternative - Zinc search

For the uninitiated, Elasticsearch is a full text search/analytics engine for bigdata lakes.

It can ingest loads of data, somewhat similar to Influxdb, but where influxdb is geared towards timeseries based data, elastic is geared towards "documents", which can be anything, including time.

Having used Elasticsearch on and off, I find it hard to deal within a personal environment (ingesting data/setting up indexes not straight forward).

I came across Zinc search (still in alpha stage), which is an alternative and supports the elasticsearch DSL query language. Ingestion is also compatible with elastic, with the ndjson format.

It has a rudimentary kibana-like search interface and has an API for ingesting/searching. It is schemaless and can create indexes on the fly.

This is pretty cool stuff to store/search IoT data as you can post your payloads directly to an index of your liking.

Example setup with docker-compose

    container_name: zinc
    image: public.ecr.aws/h9e2j3o7/zinc:latest
      - "4080:4080"
      TZ: 'Europe/Amsterdam'
      ZINC_TELEMETRY: 'disabled'
      ZINC_FIRST_ADMIN_USER: 'admin'
      ZINC_FIRST_ADMIN_PASSWORD: 'yoursecretpassword'
      - '/<path>/data:/data'
    restart: unless-stopped

This exposes port 4080 with a webinterface for searching.
It uses basic auth for its API.

To post data, setup a http request node with basic authentication, set the method to PUT and the url to:

http://<ip>:4080/api/<index name>/document

The index name can be anything you want, eg: weather
it will create the index on the fly if it does not exist.

Inject an msg.payload with an object and see your data magically appear into the index (may need to refresh the page).

To search query, add a new http request node, set the method to POST, using basic auth:

const index = "<the index you used>"
msg.url = `http://<ip>:4080/api/${index}/_search`
msg.payload = {
return msg;

The output is not (fully?) compatible with grafana, but there are requests to support it.

Personally I find this easier to work with than Influx, as influx needs a specific input format with tags/metrics, whereas with zinc you can simply inject your complete data object, which becomes searchable and aggregatable (like influx).

I don't know how it will perform with huge data loads, but for personal IoT I doubt it becomes a problem.



Thanks for sharing that, looks interesting and ElasticSearch is a monster!

Another thing to add to the ever-growing list of things to look at :sigh: :slight_smile:

1 Like

I would love to see someone integrate the systemd journal logs as per this ElasticSearch example. That might be REALLY useful.

Another potentially really useful integration would be with Telegraf since that tool (from the same people as InfluxDB) already makes system monitoring really easy. telegraf/plugins/outputs/elasticsearch at master · influxdata/telegraf (github.com)

Indeed, monitoring comes in different flavors, elastic likes logs, influx likes metrics, especially with prometheus. I see that zinc is compatible with filebeat (which is a shipper used for elastic).

I will try to set it up on a linux container.

1 Like

I cannot get Filebeat to work (i see there is an issue reported).

syslog-ng does work

Ok got filebeat working now too. Had to switch to a more recent image.

Nice. I can certainly see some uses for it.