Not able to see metrics exported by mlpipeline-metrics.json

[lucasg1]

Hi everyone!

I have a notebook that is creating the mlpipeline-metrics.json file as follows (just like the documentation):

metrics = {
    'metrics': [
        {
            'name': 'accuracy-score',
            'numberValue':  val_acc,
            'format': 'PERCENTAGE'
        },
        {
            'name': 'loss-score',
            'numberValue':  val_loss,
            'format': 'RAW'
        }
    ]
}

with open('mlpipeline-metrics.json', 'w') as f:
    json.dump(metrics, f)

But the thing is that I can't see the metrics at the Runs tab, I can only see that there is a Output Artifact called mlpipeline-metrics.json but I can see them anywhere:

Run output tab with nothing in it:

In this same pipeline I'm creating a confusion matrix which I can see only in Visualizations tab clicking on the specific pipeline step, but not in the Run Output (should it be shown at the Run Output tab?):

• How to make metrics show at Run Output (or anywhere)?
• How to make the confusion matrix shows at Run Output and not only in Visualizations tab?

I'm pasting here some screenshots so it's easier to understand and also the log file from the last pipeline step, appreciate any help.

Obs: the variables val_acc and val_loss are defined:

Log file: log.txt

The mlpipeline-metrics.json final file just in case:
mlpipeline-metrics.json.txt

[ptitzler]

Hi @lucasg1 !

As far as I know the only place where you can see these metrics is in the Experiments/runs view:

I had to look it up because it's been a while: https://www.kubeflow.org/docs/components/pipelines/v1/sdk/output-viewer/#introduction
If I recall the underlying idea is that in this view you can see multiple runs at once and it's easier to see wether there's any improvement to the metrics.

[lucasg1]

I believe it's 2.0.0-alpha.3, though I'm not sure...

I git cloned the https://github.com/kubeflow/manifests repository (in October 3) and installed everything with the command:

while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done

The only way I found to find out the version was to print the manifests/apps/pipeline/upstream/base/pipeline/kustomization.yaml file. But that still doesn't say what is the current running version. Is there a way to see the running Kubeflow Pipelines version?

kustomization.yaml file:

~/manifests/apps/pipeline/upstream$ cat base/pipeline/kustomization.yaml 
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
  - metadata-writer
resources:
  - ml-pipeline-apiserver-deployment.yaml
  - ml-pipeline-apiserver-role.yaml
  - ml-pipeline-apiserver-rolebinding.yaml
  - ml-pipeline-apiserver-sa.yaml
  - ml-pipeline-apiserver-service.yaml
  - ml-pipeline-persistenceagent-deployment.yaml
  - ml-pipeline-persistenceagent-role.yaml
  - ml-pipeline-persistenceagent-rolebinding.yaml
  - ml-pipeline-persistenceagent-sa.yaml
  - ml-pipeline-scheduledworkflow-deployment.yaml
  - ml-pipeline-scheduledworkflow-role.yaml
  - ml-pipeline-scheduledworkflow-rolebinding.yaml
  - ml-pipeline-scheduledworkflow-sa.yaml
  - ml-pipeline-ui-deployment.yaml
  - ml-pipeline-ui-configmap.yaml
  - ml-pipeline-ui-role.yaml
  - ml-pipeline-ui-rolebinding.yaml
  - ml-pipeline-ui-sa.yaml
  - ml-pipeline-ui-service.yaml
  - ml-pipeline-viewer-crd-role.yaml
  - ml-pipeline-viewer-crd-rolebinding.yaml
  - ml-pipeline-viewer-crd-deployment.yaml
  - ml-pipeline-viewer-crd-sa.yaml
  - ml-pipeline-visualization-deployment.yaml
  - ml-pipeline-visualization-sa.yaml
  - ml-pipeline-visualization-service.yaml
  - pipeline-runner-role.yaml
  - pipeline-runner-rolebinding.yaml
  - pipeline-runner-sa.yaml
  - container-builder-sa.yaml
  - viewer-sa.yaml
  - kfp-launcher-configmap.yaml
images:
  - name: gcr.io/ml-pipeline/api-server
    newTag: 2.0.0-alpha.3
  - name: gcr.io/ml-pipeline/persistenceagent
    newTag: 2.0.0-alpha.3
  - name: gcr.io/ml-pipeline/scheduledworkflow
    newTag: 2.0.0-alpha.3
  - name: gcr.io/ml-pipeline/frontend
    newTag: 2.0.0-alpha.3
  - name: gcr.io/ml-pipeline/viewer-crd-controller
    newTag: 2.0.0-alpha.3
  - name: gcr.io/ml-pipeline/visualization-server
    newTag: 2.0.0-alpha.3

[ptitzler]

Ah yes, there's the reason why the metrics are not surfaced. Things have changed in v2: https://www.kubeflow.org/docs/components/pipelines/v1/sdk/output-viewer/#v2-sdk-use-sdk-visualization-apis

What we have documented works unfortunately only for v1.

[lucasg1]

Oh okay! But is there a way to use that metrics.log_metric() function (and similars) with Elyra? Or I will have to downgrade to v1 to get the metrics in the output with Elyra?

[ptitzler]

I don't see how because Elyra only supports the v1 SDK. There's a v2 compatibility mode, but we don't support it and have no plans to. End of support for v2 compatibility was already announced last year (kfp == 1.9).

[lucasg1]

Thanks a lot for all the answears!