When running the examples, how to define the whalesay-template template ?

[EricBoix]

With Hera version 5.1.3, and when trying to run the upstream/workflow_template__steps.py I get the following exception

Exception: Server returned status code 500 with error: {'code': 2, 'message': 'templates.hello-hello-hello.steps[0].hello1 template reference workflow-template-whalesay-template.whalesay-template not found'}

I was unable to find (among the sources) the definition of the (ubiquitous in the examples)whalesay-template template.

Could it be that I missed the instructions allowing me to define (at the Hera level) the whalesay-template template ?

Notes

• Among the examples, there seems to be few examples (I could only find dag_inline_workflowtemplate.py and workflow_event_binding__event_consumer_workflowtemplate.py) that define a WorkflowTemplate.
  And none of these definitions seemed to be used (referred to) with a m.TemplateRef().
• Is this discussion somehow related to discussion Hera "community maintained templates" #677 ?

[flaviuvadan]

Thanks for the question @EricBoix! In that example whalesay-template is a template reference. That means there must be, in the cluster, a workflow template called workflow-template-whalesay-template, which contains a template called whalesay-template. So, if you first create a workflow template via Hera's WorkflowTemplate, create that via .create(...), and then use the above examples, it should all work fine!

[EricBoix]

Thanks @flaviuvadan . I had some notion that workflow templates had to be populated firrst (because I had some previous experience at the ArgoWorkflows level as opposed to the Hera level).

My concern was more with having at hand some concrete Hera code examples (for newbies like me) to define a workflow template.
I thus assembled one and submit it below in case it might be of some help for others. Notice the comments starting with the "CHANGED" keyword and pointing to the (sometimes painful) adaptations I had to make in order to get things working.

### Cluster specific
[ .... ]
# Mainly define GlobalConfig.host, GlobalConfig.token or GlobalConfig.namespace ...
cluster = define_my_cluster()    

### Some Hera level helper
from hera.workflows import WorkflowsService

def hera_clear_workflow_template(cluster, workflow_template_name):
    # Cluster must be properly defined for WorkflowsService to be properly
    # created (under the hood WorkflowsService uses Hera's GlobalConfig global
    # variable (e.g. GlobalConfig.host, GlobalConfig.token or
    # GlobalConfig.namespace ...). We thus pass the cluster variable as an
    # explicit reminder of this implicit dependency.
    service = WorkflowsService()
    workflow_templates = service.list_workflow_templates().items
    for workflow_template in workflow_templates:
      if workflow_template.metadata.name == workflow_template_name:
        # A workflow_template (with the same name) is already registered and
        # it must thus be flushed
        print("Deleting Workflow Template ", workflow_template_name)
        service.delete_workflow_template(workflow_template_name)

### The following is cluster independent (well almost)
from hera.workflows import (
    Container,
    models as m,
    Parameter,
    Step,
    Steps,
    Workflow,
    WorkflowTemplate,
)

### The following WorkflowTemplate is an adapted copy of
### https://github.com/argoproj-labs/hera/blob/5.1.3/examples/workflows/upstream/workflow_event_binding__event_consumer_workflowtemplate.py
with WorkflowTemplate(
    name="workflow-template-whalesay-template",
    entrypoint="whalesay-template",
    # CHANGED: the following line didn't seem to be used (didn't appear in the
    # generated yaml).
    #     arguments=Parameter(name="message", value="hello"),
    # BTW check that AW's workflow template does indeed accept such an
    # "arguments" entry (and what the associated semantics could/would be)
) as w:
    say = Container(
        name="argosay",
        # CHANGED: because the cluster in behind a seclusive/autistic firewall
        # there is no access to Docker Hub (hub.docker.com)
        #    image="argoproj/argosay:v2",
        image=cluster.docker_registry + "argoproj/argosay:v2",
        image_pull_policy=m.ImagePullPolicy.always,
        inputs=[
            Parameter(name="message"),
        ],
        args=["echo", "{{inputs.parameters.message}}"],
    )
    with Steps(
        name="whalesay-template",
        # CHANGED: had to add an inputs entry for whalesay-template to accept
        # arguments
        inputs=Parameter(name="message"),
    ):
        say(
            name="a",
            arguments=[
                Parameter(
                    name="message", value="{{inputs.parameters.message}}"
                )
            ],
        )
# CHANGED: if we want the workflow script (that is this file) to be run once
# re-edited then we need to remove a previously submitted version of this workflow
# template (or AW server will reject this new version with a name conflict):
hera_clear_workflow_template(cluster, "workflow-template-whalesay-template")

# CHANGED: following line added for the workflow template to be registered
w.create()

### The following Workflow is a copy of
### https://github.com/argoproj-labs/hera/blob/5.1.3/examples/workflows/upstream/workflow_template__steps.py
with Workflow(
    generate_name="workflow-template-steps-",
    entrypoint="hello-hello-hello",
) as w:
    with Steps(name="hello-hello-hello") as s:
        Step(
            name="hello1",
            template_ref=m.TemplateRef(
                name="workflow-template-whalesay-template",
                template="whalesay-template",
            ),
            arguments=Parameter(name="message", value="hello1"),
        )
        with s.parallel():
            Step(
                name="hello2a",
                template_ref=m.TemplateRef(
                    # CHANGED: the original script made reference to
                    #     name="workflow-template-inner-steps",
                    #     template="inner-steps"
                    # that just didn't seem to be resolved. It would trigger an
                    # error message of the form:
                    #     'templates.hello-hello-hello.steps[1].hello2a template
                    #     reference workflow-template-inner-steps.inner-steps
                    #     not found'
                    name="workflow-template-whalesay-template",
                    template="whalesay-template",
                ),
                arguments=Parameter(name="message", value="hello2a"),
            )
            Step(
                name="hello2b",
                template_ref=m.TemplateRef(
                    name="workflow-template-whalesay-template",
                    template="whalesay-template",
                ),
                arguments=Parameter(name="message", value="hello2b"),
            )
# CHANGED: following line added for the workflow to run
w.create()

IMVHO, maybe Hera should provide such "integrated" examples in addition to the gist/snippet of the documentation (e.g. the upstream "partial" examples) ? I believe such ready to run "integrated" examples would lower the barrier on entry for Hera.
Just my 2cts.

But thanks again @flaviuvadan ! And keep on going with the good work.

[flaviuvadan]

IMVHO, maybe Hera should provide such "integrated" examples in addition to the gist/snippet of the documentation (e.g. the upstream "partial" examples) ? I believe such ready to run "integrated" examples would lower the barrier on entry for Hera.

This is a great thought @EricBoix! I absolutely agree. We plan to focus more on this once we have Argo running in CD, so we can execute such examples and verify in CI - make a template, use it in an example to make sure it works in an E2E manner.

Thanks for the discussion!