How to transmit a simple string from a task to another ? #561
-
How to reproduce the difficultyConsider the following workflow where I wish to transmit a simple string from a task to another (in AW I use # With Hera version 4.4.1
from hera import Parameter, Task, ValueFrom, Workflow
def dummy():
...
def consume(msg: str):
print(f"Message was: {msg}")
with Workflow("io") as w:
t1 = Task(
"d", dummy, outputs=[Parameter("msg", value="some_valuable_message")]
)
t2 = Task("c", consume, inputs=[t1.get_parameter("msg")])
t1 >> t2
w.create() This workflow fails with an error message of the form: I tried some variations around the outputs argument of the t1 task e.g. outputs=[Parameter("msg", value=Value("some_valuable_message"))] # Hey, Value does exist in AW ...
outputs=[Parameter(name="msg", value="some_valuable_message")]
outputs=[Parameter("msg", value={"dummy": "some_valuable_message"})
outputs=[Parameter(name="msg", "some_valuable_message")]
outputs=[Parameter("some_valuable_message")] # Lost at sea
outputs=[Parameter(name="msg", default="some_valuable_message")] # Despaired
# Eventually, I gave up on Parameter() and tried another long shot
outputs=[ {"msg": "some_valuable_message"}]
outputs=[ "some_valuable_message" ]
outputs=[ json.dumps({"msg": "ball"}) ] # Many variations around that one
# I also tried the artifact way of things but this quickly got out of hand
# because you apparently need to configure some artifact storage (which
# doesn't seem easy to a newbie):
outputs=[RawArtifact("msg", path="/tmp/dummy", data="some_valuable_message")] QuestionMy question thus boils down to: how to transmit a simple string arising from the python/Hera context from a task to another task ? PS: thanks for the good work on Hera and for keeping alive the promise of writing workflows with less suffering than with |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments
-
I think the problem is with |
Beta Was this translation helpful? Give feedback.
-
Hi @elliotgunton and thanx for your answer. Following your proposition I thus tried # Hera version 4.4.2
from hera import Parameter, Task, Workflow
def dummy():
...
def consume(msg: str):
print(f"Message was: {msg}")
### The following workflow tries to transmit a simple string from
# task t1 (as output) to task t2 (as input):
with Workflow("io-", generate_name=True) as w:
t1 = Task(
"d", dummy, outputs=[Parameter("msg", value="some_valuable_message")]
)
t2 = Task("c", consume, args={"msg": t1.get_parameter("msg")})
t1 >> t2
w.create() but with no luck, the error message being
that somehow seems to indicate ( Lost at sea... |
Beta Was this translation helpful? Give feedback.
-
Updating code to Hera version 5.6.0. This goes from hera.workflows import DAG, Parameter, script, Task, Workflow
@script()
def dummy():
...
@script()
def consume(msg: str):
print(f"Message was: {msg}")
with Workflow(generate_name="io-") as w:
with DAG(name="main"):
t1: Task = dummy(
name="t1",
# Ditch attempt to piggyback some message in the outputs, when
# the original template didn't have any output (don't ask about
# the use case since I have to admit it's quite circumvolved).
outputs=[Parameter(name="msg", value="some_valuable_message")],
)
t2: Task = consume(arguments={"msg": t1.get_parameter("msg")})
t1 >> t2
w.create() Anyhow, closing this discussion because, with a bit more experience, I now understand they are more Hera-nic ways of getting this type of things done. |
Beta Was this translation helpful? Give feedback.
I think the problem is with
t2 = Task("c", consume, inputs=[t1.get_parameter("msg")])
- you want to pass anargument
to themsg
input parameter. i.e. I thinkt2 = Task("c", consume, arguments={"msg": t1.get_parameter("msg")}
will work. See the dag_with_param_passing and dag_with_script_param_passing for examples on how to do this