unstable value for metric duration in template

[sabreensalama]

Hi , I have an issue that I defined a metrics for an argo-workflow template and it started to be scraped but suddenly I found the argo-controller give me increased value for the duration, with each refresh to the endpoint metrics of the controller I gott a new valule for duration which not represent the real value , here is a sample of the workflow and I used helm to install controller

  - name: cassandra-data-loadv2-e2e
    metrics:
        prometheus:
        - name: exec_duration_gauge
          labels:
              - key: name
                value: cassandra_data_loadv2
          help: "Duration gauge by name"
          gauge:
                # realtime: true
                value: "{{`{{`}}duration{{`}}`}}"

here is an example of the value too , argo_workflows_exec_duration_gauge{name="cassandra_data_loadv2"} 7583.035233551
if I refreshed it will be 7584 and I don't know why

[agilgur5]

This may be because duration is an estimate?