How to use proxyConnectHeader in ScrapeConfig

[Duvieusam]

What happened?

Description

Hi,
I am trying to use the proxyConnectHeader field in ScrapeConfig object but prometheus doesn't like the generated output configuration.

I am using a secret named squid-user wich contains this data:

b64encoded: [Basic xxxxxxxx]

This secret is generated via External Secret.

My ScrapeConfig object look like this:

apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: exporter
  namespace: my-namespace
  labels:
    release: kube-prometheus-stack
spec:
  metricsPath: /metrics
  proxyUrl: http://myproxy.com:3128
  proxyConnectHeader:
    Proxy-Authorization:
      name: squid-user
      key: b64encoded
  staticConfigs:
    - targets:
      - my-target.com

This output this config in Prometheus Operator:

- job_name: scrapeConfig/my-namespace/exporter
  metrics_path: /metrics
  proxy_url: http://myproxy.com:3128
  proxy_connect_header:
    Proxy-Authorization: '[Basic xxxxxxxx]'
  static_configs:
  - targets:
    - my-target.com
    labels: {}
  relabel_configs:
  - source_labels:
    - job
    target_label: __tmp_prometheus_job_name

As you can see there is single quotes around the Proxy-Authorization value. From what I can see in the Prometheus documentation they should not be there.

Also prometheus is telling me that he can't load the configuration because he cannot unmarshal a "!!str" into []config.Secret which seems logical here.

Can you help me here ? Is this a bug or am I misusing something ?

Steps to Reproduce

Use a secret with this value:

b64encoded: [Basic xxxxxxxx]

Use this Scrape Config:

apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: exporter
  namespace: my-namespace
  labels:
    release: kube-prometheus-stack
spec:
  metricsPath: /metrics
  proxyUrl: http://myproxy.com:3128
  proxyConnectHeader:
    Proxy-Authorization:
      name: squid-user
      key: b64encoded
  staticConfigs:
    - targets:
      - my-target.com

Expected Result

Prometheus configuration should look like this:

- job_name: scrapeConfig/my-namespace/exporter
  metrics_path: /metrics
  proxy_url: http://myproxy.com:3128
  proxy_connect_header:
    Proxy-Authorization: [Basic xxxxxxxx]
  static_configs:
  - targets:
    - my-target.com
    labels: {}
  relabel_configs:
  - source_labels:
    - job
    target_label: __tmp_prometheus_job_name

Actual Result

Actual result of prom config is this:

- job_name: scrapeConfig/my-namespace/exporter
  metrics_path: /metrics
  proxy_url: http://myproxy.com:3128
  proxy_connect_header:
    Proxy-Authorization: '[Basic xxxxxxxx]'
  static_configs:
  - targets:
    - my-target.com
    labels: {}
  relabel_configs:
  - source_labels:
    - job
    target_label: __tmp_prometheus_job_name

Prometheus Operator Version

v2.50.1

Kubernetes Version

v1.28.6-eks-508b6b3

Kubernetes Cluster Type

EKS

How did you deploy Prometheus-Operator?

helm chart:prometheus-community/kube-prometheus-stack

Manifests

No response

prometheus-operator log output

ts=2024-04-17T15:00:06.789Z caller=main.go:1324 level=info msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml
ts=2024-04-17T15:00:06.811Z caller=main.go:1066 level=error msg="Error reloading config" err="couldn't load configuration (--config.file=\"/etc/prometheus/config_out/prometheus.env.yaml\"): parsing YAML file /etc/prometheus/config_out/prometheus.env.yaml: yaml: unmarshal errors:\n  line 3572: cannot unmarshal !!str `[Basic ...` into []config.Secret

Anything else?

No response

[simonpasquier]

It looks a legit bug. The golden test files seem to be invalid:

prometheus-operator/pkg/prometheus/testdata/ScrapeConfigSpecConfig_ProxySettings.golden

Lines 10 to 17 in 3e7eb79

	no_proxy: 0.0.0.0
	proxy_from_environment: false
	proxy_connect_header:
	header: value
	relabel_configs:
	- source_labels:
	- job
	target_label: __tmp_prometheus_job_name

It should rather be

 proxy_connect_header: 
   header: [value]

[simonpasquier]

Fixed by #6541