Skip to content

Commit

Permalink
feat: Artifact GC (#9255)
Browse files Browse the repository at this point in the history
* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact gc

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: print statements to be deleted

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: try out Artifact GC with one GC pod doing all deletions for a given strategy

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: didn't realize argoexec change for artifact deletion was already added

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: revert previous

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: modifying example for artifact GC to specify ArchiveLocation on the Template level

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: temporary debugging statements

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* chore: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: accidentally removed a line earlier

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - changes for referenced templates

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: remove accidental commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: remove accidental commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: remove unused code

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: remove commented out code

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: comment

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: remove accidental commits

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: unit test - doesn't seem like label 'completed' should be set??

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: result of 'make codegen -B'

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - e2e

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC e2e

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - passing secrets as volumes

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - just look for Pod types

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - unit test

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - distinguish between Undefined and Never GC strategies

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: temporarily commiting this file so we can run CI on this PR

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: unit test and lint

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: CI

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: lint errors

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: lint error

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: lint

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - don't re-check for Artifact GC if we already did

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC e2e

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: comment

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: move creation of new minio buckets to the e2e manifests so they don't mess up the standard deployment manifests

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC - remove ArtifactGCStrategy.AbbreviatedName() and relocate logic to code which was using it

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: revert unnecessary change

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: empty commit

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: reverting delete.go so I can use latest and greatest from master

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: resolving some merge conflicts

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* chore: comments

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* fix: codegen

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>

* feat: artifact GC

Signed-off-by: Julie Vogelman <julie_vogelman@intuit.com>
  • Loading branch information
juliev0 authored Aug 8, 2022
1 parent e246abe commit b356cb5
Show file tree
Hide file tree
Showing 71 changed files with 8,237 additions and 1,091 deletions.
58 changes: 47 additions & 11 deletions api/jsonschema/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -3324,6 +3324,30 @@
},
"type": "object"
},
"io.argoproj.workflow.v1alpha1.ArtGCStatus": {
"description": "ArtGCStatus maintains state related to ArtifactGC",
"properties": {
"notSpecified": {
"description": "if this is true, we already checked to see if we need to do it and we don't",
"type": "boolean"
},
"podsRecouped": {
"additionalProperties": {
"type": "boolean"
},
"description": "have completed Pods been processed? (mapped by Pod name) used to prevent re-processing the Status of a Pod more than once",
"type": "object"
},
"strategiesProcessed": {
"additionalProperties": {
"type": "boolean"
},
"description": "have Pods been started to perform this strategy? (enables us not to re-process what we've already done)",
"type": "object"
}
},
"type": "object"
},
"io.argoproj.workflow.v1alpha1.Artifact": {
"description": "Artifact indicates an artifact to place at a specified path",
"properties": {
Expand Down Expand Up @@ -3424,34 +3448,42 @@
"io.argoproj.workflow.v1alpha1.ArtifactGC": {
"description": "ArtifactGC describes how to delete artifacts from completed Workflows",
"properties": {
"podMetadata": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.Metadata",
"description": "PodMetadata is an optional field for specifying the Labels and Annotations that should be assigned to the Pod doing the deletion"
},
"serviceAccountName": {
"description": "ServiceAccountName is an optional field for specifying the Service Account that should be assigned to the Pod doing the deletion",
"type": "string"
},
"strategy": {
"description": "Strategy is the strategy to use. One of \"OnWorkflowCompletion\", \"OnWorkflowDeletion\"",
"description": "Strategy is the strategy to use.",
"type": "string"
}
},
"type": "object"
},
"io.argoproj.workflow.v1alpha1.ArtifactGCSStatus": {
"description": "ArtifactGCSStatus describes the result of the deletion",
"io.argoproj.workflow.v1alpha1.ArtifactGCSpec": {
"description": "ArtifactGCSpec specifies the Artifacts that need to be deleted",
"properties": {
"artifactResultsByNode": {
"artifactsByNode": {
"additionalProperties": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactResultNodeStatus"
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactNodeSpec"
},
"description": "ArtifactResultsByNode maps Node name to result",
"description": "ArtifactsByNode maps Node name to information pertaining to Artifacts on that Node",
"type": "object"
}
},
"type": "object"
},
"io.argoproj.workflow.v1alpha1.ArtifactGCSpec": {
"description": "ArtifactGCSpec specifies the Artifacts that need to be deleted",
"io.argoproj.workflow.v1alpha1.ArtifactGCStatus": {
"description": "ArtifactGCStatus describes the result of the deletion",
"properties": {
"artifactsByNode": {
"artifactResultsByNode": {
"additionalProperties": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactNodeSpec"
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactResultNodeStatus"
},
"description": "ArtifactsByNode maps Node name to information pertaining to Artifacts on that Node",
"description": "ArtifactResultsByNode maps Node name to result",
"type": "object"
}
},
Expand Down Expand Up @@ -7250,6 +7282,10 @@
"io.argoproj.workflow.v1alpha1.WorkflowStatus": {
"description": "WorkflowStatus contains overall status information about a workflow",
"properties": {
"artifactGCStatus": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtGCStatus",
"description": "ArtifactGCStatus maintains the status of Artifact Garbage Collection"
},
"artifactRepositoryRef": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactRepositoryRefStatus",
"description": "ArtifactRepositoryRef is used to cache the repository to use so we do not need to determine it everytime we reconcile."
Expand Down
58 changes: 47 additions & 11 deletions api/openapi-spec/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -7240,6 +7240,30 @@
}
}
},
"io.argoproj.workflow.v1alpha1.ArtGCStatus": {
"description": "ArtGCStatus maintains state related to ArtifactGC",
"type": "object",
"properties": {
"notSpecified": {
"description": "if this is true, we already checked to see if we need to do it and we don't",
"type": "boolean"
},
"podsRecouped": {
"description": "have completed Pods been processed? (mapped by Pod name) used to prevent re-processing the Status of a Pod more than once",
"type": "object",
"additionalProperties": {
"type": "boolean"
}
},
"strategiesProcessed": {
"description": "have Pods been started to perform this strategy? (enables us not to re-process what we've already done)",
"type": "object",
"additionalProperties": {
"type": "boolean"
}
}
}
},
"io.argoproj.workflow.v1alpha1.Artifact": {
"description": "Artifact indicates an artifact to place at a specified path",
"type": "object",
Expand Down Expand Up @@ -7341,34 +7365,42 @@
"description": "ArtifactGC describes how to delete artifacts from completed Workflows",
"type": "object",
"properties": {
"podMetadata": {
"description": "PodMetadata is an optional field for specifying the Labels and Annotations that should be assigned to the Pod doing the deletion",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.Metadata"
},
"serviceAccountName": {
"description": "ServiceAccountName is an optional field for specifying the Service Account that should be assigned to the Pod doing the deletion",
"type": "string"
},
"strategy": {
"description": "Strategy is the strategy to use. One of \"OnWorkflowCompletion\", \"OnWorkflowDeletion\"",
"description": "Strategy is the strategy to use.",
"type": "string"
}
}
},
"io.argoproj.workflow.v1alpha1.ArtifactGCSStatus": {
"description": "ArtifactGCSStatus describes the result of the deletion",
"io.argoproj.workflow.v1alpha1.ArtifactGCSpec": {
"description": "ArtifactGCSpec specifies the Artifacts that need to be deleted",
"type": "object",
"properties": {
"artifactResultsByNode": {
"description": "ArtifactResultsByNode maps Node name to result",
"artifactsByNode": {
"description": "ArtifactsByNode maps Node name to information pertaining to Artifacts on that Node",
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactResultNodeStatus"
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactNodeSpec"
}
}
}
},
"io.argoproj.workflow.v1alpha1.ArtifactGCSpec": {
"description": "ArtifactGCSpec specifies the Artifacts that need to be deleted",
"io.argoproj.workflow.v1alpha1.ArtifactGCStatus": {
"description": "ArtifactGCStatus describes the result of the deletion",
"type": "object",
"properties": {
"artifactsByNode": {
"description": "ArtifactsByNode maps Node name to information pertaining to Artifacts on that Node",
"artifactResultsByNode": {
"description": "ArtifactResultsByNode maps Node name to result",
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactNodeSpec"
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactResultNodeStatus"
}
}
}
Expand Down Expand Up @@ -11131,6 +11163,10 @@
"description": "WorkflowStatus contains overall status information about a workflow",
"type": "object",
"properties": {
"artifactGCStatus": {
"description": "ArtifactGCStatus maintains the status of Artifact Garbage Collection",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtGCStatus"
},
"artifactRepositoryRef": {
"description": "ArtifactRepositoryRef is used to cache the repository to use so we do not need to determine it everytime we reconcile.",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ArtifactRepositoryRefStatus"
Expand Down
2 changes: 1 addition & 1 deletion cmd/argoexec/commands/artifact/delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ func deleteArtifacts(labelSelector string, ctx context.Context, artifactGCTaskIn

task.Status.ArtifactResultsByNode[nodeName] = artResultNodeStatus
}
patch, err := json.Marshal(map[string]interface{}{"status": v1alpha1.ArtifactGCSStatus{ArtifactResultsByNode: task.Status.ArtifactResultsByNode}})
patch, err := json.Marshal(map[string]interface{}{"status": v1alpha1.ArtifactGCStatus{ArtifactResultsByNode: task.Status.ArtifactResultsByNode}})
if err != nil {
return err
}
Expand Down
2 changes: 2 additions & 0 deletions docs/executor_swagger.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,8 @@ set when loading input artifacts. | |

| Name | Type | Go type | Required | Default | Description | Example |
|------|------|---------|:--------:| ------- |-------------|---------|
| podMetadata | [Metadata](#metadata)| `Metadata` | | | | |
| serviceAccountName | string| `string` | | | ServiceAccountName is an optional field for specifying the Service Account that should be assigned to the Pod doing the deletion | |
| strategy | [ArtifactGCStrategy](#artifact-g-c-strategy)| `ArtifactGCStrategy` | | | | |


Expand Down
39 changes: 38 additions & 1 deletion docs/fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Workflow is the definition of a workflow resource

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -437,6 +439,8 @@ WorkflowSpec is the specification of a Workflow.

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -831,6 +835,7 @@ WorkflowStatus contains overall status information about a workflow
### Fields
| Field Name | Field Type | Description |
|:----------:|:----------:|---------------|
|`artifactGCStatus`|[`ArtGCStatus`](#artgcstatus)|ArtifactGCStatus maintains the status of Artifact Garbage Collection|
|`artifactRepositoryRef`|[`ArtifactRepositoryRefStatus`](#artifactrepositoryrefstatus)|ArtifactRepositoryRef is used to cache the repository to use so we do not need to determine it everytime we reconcile.|
|`compressedNodes`|`string`|Compressed and base64 decoded Nodes map|
|`conditions`|`Array<`[`Condition`](#condition)`>`|Conditions is a list of conditions the Workflow may have|
Expand Down Expand Up @@ -867,6 +872,8 @@ CronWorkflowSpec is the specification of a CronWorkflow

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -1427,10 +1434,19 @@ Arguments to a template

ArtifactGC describes how to delete artifacts from completed Workflows

<details>
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)
</details>

### Fields
| Field Name | Field Type | Description |
|:----------:|:----------:|---------------|
|`strategy`|`string`|Strategy is the strategy to use. One of "OnWorkflowCompletion", "OnWorkflowDeletion"|
|`podMetadata`|[`Metadata`](#metadata)|PodMetadata is an optional field for specifying the Labels and Annotations that should be assigned to the Pod doing the deletion|
|`serviceAccountName`|`string`|ServiceAccountName is an optional field for specifying the Service Account that should be assigned to the Pod doing the deletion|
|`strategy`|`string`|Strategy is the strategy to use.|

## ArtifactRepositoryRef

Expand Down Expand Up @@ -1739,6 +1755,17 @@ WorkflowTemplateRef is a reference to a WorkflowTemplate resource.
|`clusterScope`|`boolean`|ClusterScope indicates the referred template is cluster scoped (i.e. a ClusterWorkflowTemplate).|
|`name`|`string`|Name is the resource name of the workflow template.|

## ArtGCStatus

ArtGCStatus maintains state related to ArtifactGC

### Fields
| Field Name | Field Type | Description |
|:----------:|:----------:|---------------|
|`notSpecified`|`boolean`|if this is true, we already checked to see if we need to do it and we don't|
|`podsRecouped`|`Map< boolean , string >`|have completed Pods been processed? (mapped by Pod name) used to prevent re-processing the Status of a Pod more than once|
|`strategiesProcessed`|`Map< boolean , string >`|have Pods been started to perform this strategy? (enables us not to re-process what we've already done)|

## ArtifactRepositoryRefStatus

_No description available_
Expand Down Expand Up @@ -1812,6 +1839,8 @@ Outputs hold parameters, artifacts, and results from a step

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -1928,6 +1957,8 @@ Artifact indicates an artifact to place at a specified path

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -4500,6 +4531,8 @@ ObjectMeta is metadata that all persisted resources must have, which includes al

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -5144,6 +5177,8 @@ A single application container that you want to run within a pod.

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down Expand Up @@ -5817,6 +5852,8 @@ PersistentVolumeClaimSpec describes the common attributes of storage devices and

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-gc-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-gc-workflow.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifact-passing.yaml)
Expand Down
36 changes: 36 additions & 0 deletions examples/artifact-gc-workflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# This example shows how you can configure Artifact Garbage Collection for your Workflow.
# Here there are two artifacts - one is automatically deleted when the Workflow completes, and the other
# is deleted when the Workflow gets deleted.
# In this case, "OnWorkflowDeletion" is defined for all artifacts by default, but the "on-completion" artifact
# overrides that default strategy.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: artifact-gc-
spec:
entrypoint: main
artifactGC:
strategy: OnWorkflowDeletion # the overall strategy, which can be overridden
templates:
- name: main
container:
image: argoproj/argosay:v2
command:
- sh
- -c
args:
- |
echo "hello world" > /tmp/on-completion.txt
echo "hello world" > /tmp/on-deletion.txt
outputs:
artifacts:
- name: on-completion
path: /tmp/on-completion.txt
s3:
key: on-completion.txt
artifactGC:
strategy: OnWorkflowCompletion # overriding the default strategy for this artifact
- name: on-deletion
path: /tmp/on-deletion.txt
s3:
key: on-deletion.txt
Loading

0 comments on commit b356cb5

Please sign in to comment.