--include= filter does not work

[laurentiuspurba]

I am having issue using include= filter.

Using the following filter, no data was sent to Stackdriver

- args:
        - --stackdriver.project-id=MY-PROJECT
        - --prometheus.wal-directory=/prometheus/wal
        - --stackdriver.kubernetes.location=us-east1
        - --stackdriver.kubernetes.cluster-name=MY-CLUSTER
        - --include={job=~"kubernetes-cadvisor"}
        - --include={job=~"kubernetes-pods|kubernetes-service-endpoints"}
        - --log.level=debug

The log that I see:

level=debug ts=2020-06-02T21:26:09.784Z caller=series_cache.go:354 component="Prometheus reader" msg="target not found" labels="{__name__=\"container_memory_failures_total\",endpoint=\"http-metrics\",failure_type=\"pgfault\",id=\"/system.slice/system-systemd\\\\x2dfsck.slice\",instance=\"10.154.0.7:10255\",job=\"kubernetes-cadvisor\",node=\"MY-GKE-NODE\",scope=\"hierarchy\",service=\"prometheus-prometheus-oper-kubelet\"}"

I was able to get data sent to Stackdriver if I use only the following:

• --include={__name__=~\".+\"}
• --include={job__=~\".+\"}
• --include={__name__=~\"container_network_transmit_packets_total\"}
• --include={__name__=~\"node_exporter_build_info\"}

Any help is appreciated.

Thank you,

[immber]

The documentation on the ReadMe for filters is incorrect and results in parsing errors. To reference a metric by name you must use this format, and do not include the extra quotes shown on the ReadMe example:

For example:
--include='{__name__="http_requests_total"}'
Should be:
--include={__name__="http_requests_total"}

Or you will get err="cannot parse --include flag