Enhanced depends logic not working when task is omitted

[luke-hemisphere]

Checklist

• Double-checked my configuration.
• Tested using the latest version.
• Used the Emissary executor.

Summary

What happened?
When a task is Skipped/Omitted because a dependent task is skipped you can't use enhanced depends on that task.

  O
 /  \
A    B <- B is skipped
|    |
C    D <- D gets omitted
|   /
E <- if C.Succeeded && (D.Succeeded || D.Skipped)*

*doesn't work

What you expected to happen?
If a task is omitted/skipped due a dependent predecessor task being skipped, you should be able to use depends.Skipped on that task in another task.

What version are you running? 3.3.1

Diagnostics

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: broken-depends
  namespace: argo
spec:
  serviceAccountName: argo-workflow-sa
  entrypoint: diamond
  arguments:
    parameters:
      - name: input_var
        default: "true"
  templates:
    - name: diamond
      inputs:
        parameters:
          - name: input_var
      dag:
        tasks:
          - name: A
            template: pass
          - name: B
            template: pass
            when: "{{ inputs.parameters.input_var }} == false"
          - name: C
            template: pass
            depends: B.Succeeded
          - name: D
            template: pass
            depends: A.Succeeded
          - name: E
            template: pass
            depends: D.Succeeded && C
    - name: pass
      container:
        image: alpine:3.7
        command:
          - sh
          - -c
          - exit 0
    - name: fail
      container:
        image: alpine:3.7
        command:
          - sh
          - -c
          - exit 1

# Logs from the workflow controller:
time="2022-05-06T05:20:29.162Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.176Z" level=info msg="Updated phase  -> Running" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.198Z" level=info msg="DAG node broken-depends-pn58z initialized Running" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="All of node broken-depends-pn58z.B dependencies [] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="Skipped node broken-depends-pn58z-2614862280 initialized Skipped (message: when 'true == false' evaluated false)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="Skipped node broken-depends-pn58z-2631639899 initialized Omitted (message: omitted: depends condition not met)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.200Z" level=info msg="All of node broken-depends-pn58z.A dependencies [] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.200Z" level=info msg="Pod node broken-depends-pn58z-2665195137 initialized Pending" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.228Z" level=info msg="Created pod: broken-depends-pn58z.A (broken-depends-pn58z-2665195137)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.229Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.229Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.267Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=359615338 workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.234Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.235Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=broken-depends-pn58z-2665195137 old.message= old.phase=Pending old.progress=0/1
time="2022-05-06T05:20:39.236Z" level=info msg="All of node broken-depends-pn58z.D dependencies [A] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.241Z" level=info msg="Pod node broken-depends-pn58z-2715527994 initialized Pending" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.262Z" level=info msg="Created pod: broken-depends-pn58z.D (broken-depends-pn58z-2715527994)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.263Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.263Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.278Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=359615414 workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.283Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/broken-depends-pn58z-2665195137/labelPodCompleted
time="2022-05-06T05:20:49.267Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.269Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=broken-depends-pn58z-2715527994 old.message= old.phase=Pending old.progress=0/1
time="2022-05-06T05:20:49.270Z" level=info msg="Outbound nodes of broken-depends-pn58z set to [broken-depends-pn58z-2631639899 broken-depends-pn58z-2715527994]" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="node broken-depends-pn58z phase Running -> Succeeded" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="node broken-depends-pn58z finished: 2022-05-06 05:20:49.270813223 +0000 UTC" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Checking daemoned children of broken-depends-pn58z" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Updated phase Running -> Succeeded" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Marking workflow completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Doesn't match with archive label selector. Skipping Archive" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.272Z" level=info msg="Checking daemoned children of " namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.277Z" level=info msg="cleaning up pod" action=deletePod key=argo/broken-depends-pn58z-1340600742-agent/deletePod
time="2022-05-06T05:20:49.288Z" level=info msg="Workflow update successful" namespace=argo phase=Succeeded resourceVersion=359615487 workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.300Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/broken-depends-pn58z-2715527994/labelPodCompleted

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[terrytangyuan]

This is fixed in #8672. Thanks for reporting!

[yavulan]

@terrytangyuan I'm using Omitted with dag, and sensor bumps into this error https://github.com/argoproj/argo-workflows/blob/master/workflow/common/ancestry.go#L102

[terrytangyuan]

@terrytangyuan I'm using Omitted with dag, and sensor bumps into this error https://github.com/argoproj/argo-workflows/blob/master/workflow/common/ancestry.go#L102

Thank you! I am fixing this in #8776.

[mhehle]

@terrytangyuan @luke-hemisphere

in version 3.3.8 this issue still exists.
Its impossible to launch/create a workflow with depends on .Omitted step.
You get an error: Internal Server Error: templates.main.tasks.E task result 'Omitted' for task 'D' is invalid

Example Workflow:

apiVersion: argoproj.io/v1alpha1
metadata:
  name: broken-depends
  generateName: broken-depends-
  namespace: argo
spec:
  serviceAccountName: workflow
  entrypoint: main
  templates:
    - name: main
      inputs:
        parameters:
          - name: input_var
            default: 'true'
      dag:
        tasks:
          - name: A
            template: pass
          - name: B
            template: pass
            when: '{{ inputs.parameters.input_var }} == false'
          - name: C
            template: pass
            depends: A.Succeeded
          - name: D
            template: pass
            depends: B.Succeeded
          - name: E
            template: pass
            depends: (D.Succeeded || D.Omitted) && C
    - name: pass
      container:
        image: 'alpine:3.7'
        command:
          - sh
          - '-c'
          - exit 0
    - name: fail
      container:
        image: 'alpine:3.7'
        command:
          - sh
          - '-c'
          - exit 1

[SiddharthMalhotra]

Facing the same issue as @mhehle

[mhehle]

the commit 961f731 with the fix was only picked for 3.3.9 release. So this issue should be resolved since version 3.3.9.

CherryPick list for 3.3.9
#9262