Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Supplied valueFrom ignores defaults after duration elapsed #12230

Open
3 tasks done
frauniki opened this issue Nov 20, 2023 · 2 comments · May be fixed by #12960
Open
3 tasks done

Supplied valueFrom ignores defaults after duration elapsed #12230

frauniki opened this issue Nov 20, 2023 · 2 comments · May be fixed by #12960
Labels
area/templating Templating with `{{...}}` P2 Important. All bugs with >=3 thumbs up that aren’t P0 or P1, plus: Any other bugs deemed important type/bug

Comments

@frauniki
Copy link

frauniki commented Nov 20, 2023 •
edited

Pre-requisites

  • I have double-checked my configuration
  • I can confirm the issues exists when I tested with :latest
  • I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

If the following WF is executed, the default value should be output to outputs.message after the duration of the approval step, but it is not.
The actual output is a template message "{{steps.approve.outputs.parameters.message}}".

This is the same issue as this discussion.
#7905

Expected output

 _________________ 
< default message >
 ----------------- 
    \
     \
      \     
                    ##        .            
              ## ## ##       ==            
           ## ## ## ##      ===            
       /""""""""""""""""___/ ===        
  ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~   
       \______ o          __/            
        \    \        __/             
          \____\______/

Actual output

 _________________________________________ 
/ {{steps.approve.outputs.parameters.mess \
\ age}}                                   /
 ----------------------------------------- 
    \
     \
      \     
                    ##        .            
              ## ## ##       ==            
           ## ## ## ##      ===            
       /""""""""""""""""___/ ===        
  ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~   
       \______ o          __/            
        \    \        __/             
          \____\______/

Version

v3.5.0

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: suspend-outputs
spec:
  entrypoint: suspend
  templates:
  - name: suspend
    steps:
    - - name: approve
        template: approve
    - - name: release
        template: whalesay
        arguments:
          parameters:
            - name: message
              value: "{{steps.approve.outputs.parameters.message}}"

  - name: approve
    suspend:
      duration: 20s
    inputs:
      parameters:
        - name: message
          default: default message
    outputs:
      parameters:
        - name: message
          valueFrom:
            #default: default message
            supplied: {}

  - name: whalesay
    inputs:
      parameters:
        - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

Logs from the workflow controller

time="2023-11-20T05:13:04.666Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.722Z" level=info msg="Updated phase  -> Running" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.722Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.722Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="Steps node suspend-outputs-7hhgl initialized Running" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="StepGroup node suspend-outputs-7hhgl-849034079 initialized Running" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="Suspend node suspend-outputs-7hhgl-3077708136 initialized Pending" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="node suspend-outputs-7hhgl[0].approve suspended" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="node suspend-outputs-7hhgl-3077708136 phase Pending -> Running" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="Workflow step group node suspend-outputs-7hhgl-849034079 not yet completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.723Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:04.735Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=70269789 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.724Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.725Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="node suspend-outputs-7hhgl[0].approve suspended" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="auto resuming node suspend-outputs-7hhgl[0].approve" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="node suspend-outputs-7hhgl-3077708136 phase Running -> Succeeded" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="node suspend-outputs-7hhgl-3077708136 finished: 2023-11-20 05:13:14.726177679 +0000 UTC" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="Step group node suspend-outputs-7hhgl-849034079 successful" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="node suspend-outputs-7hhgl-849034079 phase Running -> Succeeded" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="node suspend-outputs-7hhgl-849034079 finished: 2023-11-20 05:13:14.726321191 +0000 UTC" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="StepGroup node suspend-outputs-7hhgl-1990059266 initialized Running" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="SG Outbound nodes of suspend-outputs-7hhgl-3077708136 are [suspend-outputs-7hhgl-3077708136]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.726Z" level=info msg="Pod node suspend-outputs-7hhgl-2887505401 initialized Pending" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.763Z" level=info msg="Created pod: suspend-outputs-7hhgl[1].release (suspend-outputs-7hhgl-whalesay-2887505401)" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.763Z" level=info msg="Workflow step group node suspend-outputs-7hhgl-1990059266 not yet completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.771Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.771Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:14.786Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=70269882 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.770Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.772Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.772Z" level=info msg="node changed" namespace=argo-workflows new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=suspend-outputs-7hhgl-2887505401 old.message= old.phase=Pending old.progress=0/1 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.772Z" level=info msg="SG Outbound nodes of suspend-outputs-7hhgl-3077708136 are [suspend-outputs-7hhgl-3077708136]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.773Z" level=info msg="Workflow step group node suspend-outputs-7hhgl-1990059266 not yet completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.773Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.773Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:24.784Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=70269953 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.789Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.814Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.814Z" level=info msg="node unchanged" namespace=argo-workflows nodeID=suspend-outputs-7hhgl-2887505401 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.815Z" level=info msg="SG Outbound nodes of suspend-outputs-7hhgl-3077708136 are [suspend-outputs-7hhgl-3077708136]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.825Z" level=info msg="Workflow step group node suspend-outputs-7hhgl-1990059266 not yet completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.825Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:34.826Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.312Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.313Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.313Z" level=info msg="node unchanged" namespace=argo-workflows nodeID=suspend-outputs-7hhgl-2887505401 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.313Z" level=info msg="SG Outbound nodes of suspend-outputs-7hhgl-3077708136 are [suspend-outputs-7hhgl-3077708136]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.314Z" level=info msg="Workflow step group node suspend-outputs-7hhgl-1990059266 not yet completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.314Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:13:50.314Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.514Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=1 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="task-result changed" namespace=argo-workflows nodeID=suspend-outputs-7hhgl-2887505401 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="node changed" namespace=argo-workflows new.message= new.phase=Succeeded new.progress=0/1 nodeID=suspend-outputs-7hhgl-2887505401 old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="SG Outbound nodes of suspend-outputs-7hhgl-3077708136 are [suspend-outputs-7hhgl-3077708136]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="Step group node suspend-outputs-7hhgl-1990059266 successful" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="node suspend-outputs-7hhgl-1990059266 phase Running -> Succeeded" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="node suspend-outputs-7hhgl-1990059266 finished: 2023-11-20 05:14:17.516880854 +0000 UTC" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="Outbound nodes of suspend-outputs-7hhgl-2887505401 is [suspend-outputs-7hhgl-2887505401]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="Outbound nodes of suspend-outputs-7hhgl is [suspend-outputs-7hhgl-2887505401]" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="node suspend-outputs-7hhgl phase Running -> Succeeded" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="node suspend-outputs-7hhgl finished: 2023-11-20 05:14:17.516961617 +0000 UTC" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.516Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.517Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.517Z" level=info msg="Updated phase Running -> Succeeded" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.517Z" level=info msg="Marking workflow completed" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.517Z" level=info msg="Marking workflow as pending archiving" namespace=argo-workflows workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.523Z" level=info msg="cleaning up pod" action=deletePod key=argo-workflows/suspend-outputs-7hhgl-1340600742-agent/deletePod
time="2023-11-20T05:14:17.529Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Succeeded resourceVersion=70270646 workflow=suspend-outputs-7hhgl
time="2023-11-20T05:14:17.559Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-workflows/suspend-outputs-7hhgl-whalesay-2887505401/labelPodCompleted
time="2023-11-20T05:14:17.561Z" level=info msg="archiving workflow" namespace=argo-workflows uid=f19b2638-fefd-42d5-bd45-33d952801b5e workflow=suspend-outputs-7hhgl

Logs from in your workflow's wait container

time="2023-11-20T05:14:08.640Z" level=info msg="No output artifacts"
time="2023-11-20T05:14:08.642Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: (secret)/suspend-outputs-7hhgl/suspend-outputs-7hhgl-whalesay-2887505401/main.log"
time="2023-11-20T05:14:08.666Z" level=info msg="Creating minio client using AWS SDK credentials"
time="2023-11-20T05:14:09.019Z" level=info msg="Saving file to s3" bucket=(secret) endpoint=s3.amazonaws.com key=(secret)/suspend-outputs-7hhgl/suspend-outputs-7hhgl-whalesay-2887505401/main.log path=/tmp/argo/outputs/logs/main.log
time="2023-11-20T05:14:09.384Z" level=info msg="Save artifact" artifactName=main-logs duration=742.728571ms error="<nil>" key=(secret)/suspend-outputs-7hhgl/suspend-outputs-7hhgl-whalesay-2887505401/main.log
time="2023-11-20T05:14:09.385Z" level=info msg="not deleting local artifact" localArtPath=/tmp/argo/outputs/logs/main.log
time="2023-11-20T05:14:09.385Z" level=info msg="Successfully saved file: /tmp/argo/outputs/logs/main.log"
time="2023-11-20T05:14:09.620Z" level=info msg="Alloc=10149 TotalAlloc=16985 Sys=23397 NumGC=4 Goroutines=12"
time="2023-11-20T05:14:09.620Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-11-20T05:14:09.621Z" level=info msg="Deadline monitor stopped"
@agilgur5 agilgur5 added area/templating Templating with `{{...}}` P2 Important. All bugs with >=3 thumbs up that aren’t P0 or P1, plus: Any other bugs deemed important labels Nov 20, 2023
@jswxstw
Copy link
Contributor

jswxstw commented Apr 12, 2024

argo-workflows/workflow/controller/operator.go

Lines 3355 to 3372 in ec5b5d5

if tmpl.Suspend.Duration != "" {
node, err := woc.wf.GetNodeByName(nodeName)
if err != nil {
return nil, err
}
suspendDuration, err := parseStringToDuration(tmpl.Suspend.Duration)
if err != nil {
return node, err
}
suspendDeadline := node.StartedAt.Add(suspendDuration)
requeueTime = &suspendDeadline
if time.Now().UTC().After(suspendDeadline) {
// Suspension is expired, node can be resumed
woc.log.Infof("auto resuming node %s", nodeName)
_ = woc.markNodePhase(nodeName, wfv1.NodeSucceeded)
return node, nil
}
}

Node is marked to Succeeded when suspension is expired, but the value of node outputs parameters was not filled up with ValueFrom.Default if ValueFrom.Supplied != nil.

@jswxstw
Copy link
Contributor

jswxstw commented Apr 12, 2024

"{{steps.approve.outputs.parameters.message}}" can not be resolved when ValueFrom.Default in the outputs is not set.
The behavior now is that the raw text will be passed to the template which I think it is not very reasonable.

jswxstw pushed a commit to jswxstw/argo-workflows that referenced this issue Apr 22, 2024
fix: Set default value to output parameters if suspend node timeout. F…
772c5a2 
…ixes argoproj#12230

Signed-off-by: oninowang <oninowang@tencent.com>
jswxstw pushed a commit to jswxstw/argo-workflows that referenced this issue Apr 22, 2024
fix: Set default value to output parameters if suspend node timeout. F…
dc4c315 
…ixes argoproj#12230

Signed-off-by: oninowang <oninowang@tencent.com>
@jswxstw jswxstw linked a pull request Apr 22, 2024 that will close this issue
Open
jswxstw pushed a commit to jswxstw/argo-workflows that referenced this issue Apr 22, 2024
fix: Set default value to output parameters if suspend node timeout. F…
e048a76 
…ixes argoproj#12230

Signed-off-by: oninowang <oninowang@tencent.com>
jswxstw pushed a commit to jswxstw/argo-workflows that referenced this issue Apr 23, 2024
fix(test): workflow use name instead. Fixes argoproj#12230
2317852 
Signed-off-by: oninowang <oninowang@tencent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/templating Templating with `{{...}}` P2 Important. All bugs with >=3 thumbs up that aren’t P0 or P1, plus: Any other bugs deemed important type/bug
Projects
None yet
Milestone
No milestone
3 participants
@agilgur5 @jswxstw @frauniki