Skip to content

A simple Prometheus (aggregated) push gateway allowing stateless/serverless workloads, ephemeral and batch jobs to easily expose their metrics.

License

Notifications You must be signed in to change notification settings

eidam/cf-workers-prometheus-push-gateway

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Serverless Prometheus (aggregated) Push Gateway

A simple Prometheus (aggregated) push gateway allowing stateless/serverless workloads, ephemeral and batch jobs to easily expose their metrics. Aggregated metrics are exposed via /metrics endpoint and optionally pushed to a remote Prometheus instance.

Powered by Cloudflare Workers, Durable Objects, and CRON Triggers.

Prometheus metrics

Features

  • Simple PATCH counter - no need for a client libraries for a counter metric, just send PATCH /metrics/:metricName?label1=value1&label2=value2 to increment its value.
  • Aggregated metrics - received values are added up to the current (label matching) metrics. All metrics exposed on GET /metrics.
  • Automatic push to a remote prometheus server - metrics are (optionally) pushed to a remote_write prometheus endpoint on each CRON Trigger execution. See Grafana Cloud (generous free tier) for a managed Prometheus with Grafana.

Deployment

Cloudflare Account with Paid Workers is needed.

  1. yarn
  2. yarn deploy
  3. (optional) add following secrets (either via Cloudflare Dashboard or with npx wrangler@beta secret put)
    • SECRET_PROM_ENDPOINT (e.g. https://prometheus-us-central1.grafana.net/api/prom/push)
    • SECRET_PROM_USER (e.g. 23333)
    • SECRET_PROM_TOKEN (e.g. xxyy)

Send metrics

There are a couple of easy ways to push your metrics, the POST /metrics endpoint is compatible with any prometheus client library configured to use it as a push gateway.

In case Cloudflare Workers are the only clients, its recommended to deploy this Workers with no routes attached to it and use Service bindings.

  1. POST /metrics - accepts prometheus text-based format.
  2. PATCH /metrics/:metricName?label1=value1&label2=value2 - simplified counter increments that works well with no libraries needed.

Examples

curl

  1. Send curl
    curl -x PATCH https://worker.example.com/metrics/metric_name?foo=bar
    
  2. Send POST request with body in prometheus text-based format
    cat <<EOF | curl --data-binary @- https://worker.example.com/metrics
    # TYPE some_metric counter
    some_metric{label="val1"} 42
    # TYPE another_metric gauge
    # HELP another_metric Just an example.
    another_metric 2398.283
    EOF

Workers

  1. A simple worker_request counter that will keep track of request countries.
    export default {
        async fetch(request, env, ctx) {
            ctx.waitUntil(
                fetch(`https://worker.example.com/metrics/worker_request?country=${request.cf.country}&anotherLabel=value`, {
                    method: "PATCH"
                })
            )
            return new Response("Hello World!")
        }
    }
  2. promjs (or any other js library)
     import prom from 'promjs';
     export default {
         async fetch(request, env, ctx) {
             const registry = prom()
             const counter = registry.create('counter', 'my_counter', 'A counter for things')
             
             // process tasks, e.g. orders
             counter.inc({user: "user1", plan: "pro"})
             counter.add(3, {user: "user2", plan: "free"})
             counter.inc({user: "user3", plan: "pro"})
    
             ctx.waitUntil(
                 fetch(`https://worker.example.com/metrics`, {
                     method: "POST",
                     body: JSON.stringify(registry.metrics())
                 })
             )
    
         }
     }

Delete metrics

  • DELETE /metrics/__all to delete all metrics
  • DELETE /metrics/:metricName to delete a specific metric,
  • DELETE /metrics/:metricName?foo=bar to delete a metric matching specific labels only (all labels need to match)

Limitations

  • Max ~100rps, as no Durable Objects sharding is in place. (TODO)
  • counter and gauge metric types supported only. (histogram WIP)
  • No auth. If a public route needed, Cloudflare Access is highly recommended.

Thank you

Inspired by Prometheus Push Gateway and Prometheus Aggregation Gateway

About

A simple Prometheus (aggregated) push gateway allowing stateless/serverless workloads, ephemeral and batch jobs to easily expose their metrics.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published