influx2cortex is a proxy that accepts Influx Line protocol and writes it to Cortex. While today it only accepts writes, I have plans to add Flux read support too!
The Influx write proxy accepts the ingest requests and then translates the incoming Influx metrics into Prometheus metrics. The name mapping scheme for the looks as follows:
Influx metric: cpu_load_short,host=server01,region=us-west value=0.64 1658139550000000000
Prometheus metric: cpu_load_short{__proxy_source__="influx",host="server01",region="us-west"}
To build the proxy:
($ indicates the command line prompt)
$ go mod tidy
$ make build
$ make test
This should place a build of influx2cortex in the dist subdirectory.
Here we show how to configure and run the Influx write proxy to talk to an existing Mimir installation running on port 9090 on localhost. If no existing Mimir installation is available, or you would like to quickly install a test installation then follow the getting-started instructions.
In order to configure a write proxy we need to know the following pieces of information at a minimum:
- The TCP port that the write proxy should listen on
- The endpoint for remote writes within Mimir
The default TCP port for the write proxy is 8000 however it is best to choose a unique non-default port, especially if you are going to be running multiple write proxies (Graphite, Datadog, Influx, etc) on the same host.
If Mimir is configured to listen on port 9009 on localhost then the remote write endpoint will be http://localhost:9009/api/v1/push
(Pre-built binaries/docker images are on our list of things to do.)
To run the proxy:
$ dist/influx2cortex \
-auth.enable=false \
-server.http-listen-address 127.0.0.1 \
-server.http-listen-port 8007 \
-write-endpoint http://localhost:9009/api/v1/push
Details of configurable options are available in the -help output.
$ NOW=`date +%s000000000` ; curl -H "Content-Type: application/json" "http://localhost:8007/api/v1/push/influx/write" -d 'cpu_load_short,host=server01,region=us-west value=0.64 $NOW'
The data can now be queried from Mimir via the HTTP API or via Grafana. To find the above series via the HTTP API we can issue:
$ curl -G http://localhost:9009/prometheus/api/v1/series -d 'match[]=cpu_load_short'
{"status":"success","data":[{"__name__":"cpu_load_short","__proxy_source__":"influx","host":"server01","region":"us-west"}]}
If the destination Mimir installation is part of a Grafana cloud instance the -write-endpoint argument should be of the form:
-write-endpoint https://_username_:_password_@_grafana_net_instance_/api/v1/push
where the exact server details can be found on Prometheus instance details page for the stack on grafana.com
The _username_ is the numeric Username / Instance ID
The _password_ is a Grafana Cloud API Key with the MetricsPublisher role.
The _grafana_net_instance_ is server part of the URL to push Prometheus metrics.
Telegraf can be configured in two ways to send data to the Influx proxy.
- Using the
httpplugin - Using the
influxdbplugin
If connecting to a local influx proxy running on localhost:8000 the two configs would look something like this:
Note: The url has a path of /api/v1/push/influx. The trailing /write is not required as this is appended by the telegraf agent when using the outputs.influxdb output plugin.
[[outputs.influxdb]]
urls = ["https://localhost:8000/api/v1/push/influx"]
data_format = "influx"
skip_database_creation = true
The skip_database_creation = true option is to prevent errors such as:
022-09-27T16:20:20Z W! [outputs.influxdb] When writing to [https://localhost:8000/api/v1/push/influx]: database "telegraf" creation failed: 500 Internal Server Error
Note: The url has a path of /api/v1/push/influx/write. The trailing /write is required as the outputs.http output plugin uses the URL without modification (unlike the outputs.influxdb output plugin above).
[[outputs.http]]
url = "http://localhost:8000/api/v1/push/influx/write"
data_format = "influx"
timeout = "10s"
method = "POST"
interval = "300s"
flush_interval = "150s"
If you are a Grafana Cloud customer and wish to use telegraf to write to an Influx Proxy running inside Grafana Cloud you can use a config similar to this:
[[outputs.influxdb]]
urls = ["https://_grafana_net_instance_/api/v1/push/influx/write"]
username = "_username_"
password = "_password_"
data_format = "influx"
skip_database_creation = true
As above, the _username_, _password_ and _grafana_net_instance_ are adapted from the Prometheus instance details for the stack information page on grafana.com.
For example, if your username was 123456789 and the Prometheus write endpoint was listed as https://prometheus-prod-26-prod-ap-south-0.grafana.net/api/prom/push then the corresponding config to send to Grafana Cloud would look something like:
[[outputs.influxdb]]
urls = ["https://influx-prod-26-prod-ap-south-0.grafana.net/api/v1/push/influx/write"]
username = "123456789"
password = "_ELIDED_"
data_format = "influx"
skip_database_creation = true
Note: The hostname in the URL has influx instead of prometheus.
More information, including example python, ruby and Node.js snippets to push to the influx2cortex proxy can be found in the Push metrics from Influx Telegraf to Prometheus blog post.
The influx2cortex binary exposes internal metrics on a /metrics endpoint on a separate port which can be scraped by a local prometheus installation. This is configurable with the internalserver command line options.
- Consolidate `pkg/internalserver' into mimir-proxies