Skip to content

Commit

Permalink
Remove Git and Storage PipelineResources
Browse files Browse the repository at this point in the history 
This commit removes the Git and Storage Resources:
- removes the Storage resources support at `pkg/apis/resource/v1alpha1/storage`
- removes the `pkg/artifacts`
- removes the Git resources support at `pkg/apis/resource/v1alpha1/git`
- their corresponding unit test cases, examples and integration tests
- the generic logics for input and output resources since `storage` and
  `git` resources are the only allowed types
  • Loading branch information
@JeromeJu
JeromeJu committed Feb 13, 2023
1 parent 2da10fc commit 4d876da
Show file tree
Hide file tree
Showing 82 changed files with 233 additions and 11,212 deletions.
33 changes: 0 additions & 33 deletions config/config-artifact-bucket.yaml
View file

This file was deleted.

278 changes: 0 additions & 278 deletions docs/resources.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,7 @@ For example:
- [Resource Status](#resource-status)
- [Optional Resources](#optional-resources)
- [Resource types](#resource-types)
- [Git Resource](#git-resource)
- [Image Resource](#image-resource)
- [Storage Resource](#storage-resource)
- [GCS Storage Resource](#gcs-storage-resource)
- [Why Aren't PipelineResources in Beta?](#why-aren-t-pipelineresources-in-beta)

## Syntax
Expand Down Expand Up @@ -224,22 +221,6 @@ spec:
emptyDir: {}
```

### Resource Status

When resources are bound inside a `TaskRun`, they can include extra information
in the `TaskRun` Status.ResourcesResult field. This information can be useful
for auditing the exact resources used by a `TaskRun` later. Currently the Image
and Git resources use this mechanism.

For an example of what this output looks like:

```yaml
resourcesResult:
- key: digest
value: sha256:a08412a4164b85ae521b0c00cf328e3aab30ba94a526821367534b81e51cb1cb
resourceName: skaffold-image-leeroy-web
```

### Description

The `description` field is an optional field and can be used to provide description of the Resource.
Expand Down Expand Up @@ -289,171 +270,6 @@ You can refer to different examples demonstrating usage of optional resources in

## Resource Types

### Git Resource

The `git` resource represents a [git](https://git-scm.com/) repository, that
contains the source code to be built by the pipeline. Adding the `git` resource
as an input to a `Task` will clone this repository and allow the `Task` to
perform the required actions on the contents of the repo.

To create a git resource using the `PipelineResource` CRD:

```yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: wizzbang-git
namespace: default
spec:
type: git
params:
- name: url
value: https://github.com/wizzbangcorp/wizzbang.git
- name: revision
value: master
```

Params that can be added are the following:

1. `url`: represents the location of the git repository, you can use this to
change the repo, e.g. [to use a fork](#using-a-fork)
1. `revision`: Git [revision][git-rev] (branch, tag, commit SHA or ref) to
clone. You can use this to control what commit [or branch](#using-a-branch)
is used. [git checkout][git-checkout] is used to switch to the
revision, and will result in a detached HEAD in most cases. Use refspec
along with revision if you want to checkout a particular branch without a
detached HEAD. _If no revision is specified, the resource inspects remote repository to determine the correct default branch._
1. `refspec`: (Optional) specify a git [refspec][git-refspec] to pass to git-fetch.
Note that if this field is specified, it must specify all refs, branches, tags,
or commits required to checkout the specified `revision`. An additional fetch
will not be run to obtain the contents of the revision field. If no refspec
is specified, the value of the `revision` field will be fetched directly.
The refspec is useful in manipulating the repository in several cases:
* when the server does not support fetches via the commit SHA (i.e. does
not have `uploadpack.allowReachableSHA1InWant` enabled) and you want
to fetch and checkout a specific commit hash from a ref chain.
* when you want to fetch several other refs alongside your revision
(for instance, tags)
* when you want to checkout a specific branch, the revision and refspec
fields can work together to be able to set the destination of the incoming
branch and switch to the branch.

Examples:
- Check out a specified revision commit SHA1 after fetching ref (detached) <br>
&nbsp;&nbsp;`revision`: cb17eba165fe7973ef9afec20e7c6971565bd72f <br>
&nbsp;&nbsp;`refspec`: refs/smoke/myref <br>
- Fetch all tags alongside refs/heads/master and switch to the master branch
(not detached) <br>
&nbsp;&nbsp;`revision`: master <br>
&nbsp;&nbsp;`refspec`: "refs/tags/\*:refs/tags/\* +refs/heads/master:refs/heads/master"<br>
- Fetch the develop branch and switch to it (not detached) <br>
&nbsp;&nbsp;`revision`: develop <br>
&nbsp;&nbsp;`refspec`: refs/heads/develop:refs/heads/develop <br>
- Fetch refs/pull/1009/head into the master branch and switch to it (not detached) <br>
&nbsp;&nbsp;`revision`: master <br>
&nbsp;&nbsp;`refspec`: refs/pull/1009/head:refs/heads/master <br>

1. `submodules`: defines if the resource should initialize and fetch the
submodules, value is either `true` or `false`. _If not specified, this will
default to true_
1. `depth`: performs a [shallow clone][git-depth] where only the most recent
commit(s) will be fetched. This setting also applies to submodules. If set to
`'0'`, all commits will be fetched. _If not specified, the default depth is 1._
1. `sslVerify`: defines if [http.sslVerify][git-http.sslVerify] should be set
to `true` or `false` in the global git config. _Defaults to `true` if
omitted._

[git-rev]: https://git-scm.com/docs/gitrevisions#_specifying_revisions
[git-checkout]: https://git-scm.com/docs/git-checkout
[git-refspec]: https://git-scm.com/book/en/v2/Git-Internals-The-Refspec
[git-depth]: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---depthltdepthgt
[git-http.sslVerify]: https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslVerify

When used as an input, the Git resource includes the exact commit fetched in the
`resourceResults` section of the `taskRun`'s status object:

```yaml
resourceResults:
- key: commit
value: 6ed7aad5e8a36052ee5f6079fc91368e362121f7
resourceName: skaffold-git
```

#### Using a fork

The `Url` parameter can be used to point at any git repository, for example to
use a GitHub fork at master:

```yaml
spec:
type: git
params:
- name: url
value: https://github.com/bobcatfish/wizzbang.git
```

#### Using a branch

The `revision` can be any
[git commit-ish (revision)](https://git-scm.com/docs/gitrevisions#_specifying_revisions).
You can use this to create a git `PipelineResource` that points at a branch, for
example:

```yaml
spec:
type: git
params:
- name: url
value: https://github.com/wizzbangcorp/wizzbang.git
- name: revision
value: some_awesome_feature
```

To point at a pull request, you can use
[the pull requests's branch](https://help.github.com/articles/checking-out-pull-requests-locally/):

```yaml
spec:
type: git
params:
- name: url
value: https://github.com/wizzbangcorp/wizzbang.git
- name: revision
value: refs/pull/52525/head
```

#### Using HTTP/HTTPS Proxy

The `httpProxy` and `httpsProxy` parameter can be used to proxy non-SSL/SSL requests, for example to use an enterprise
proxy server for SSL requests:

```yaml
spec:
type: git
params:
- name: url
value: https://github.com/bobcatfish/wizzbang.git
- name: httpsProxy
value: "my-enterprise.proxy.com"
```

#### Using No Proxy

The `noProxy` parameter can be used to opt out of proxying, for example, to not proxy HTTP/HTTPS requests to
`no.proxy.com`:

```yaml
spec:
type: git
params:
- name: url
value: https://github.com/bobcatfish/wizzbang.git
- name: noProxy
value: "no.proxy.com"
```

Note: `httpProxy`, `httpsProxy`, and `noProxy` are all optional but no validation done if all three are specified.

### Image Resource

An `image` resource represents an image that lives in a remote repository. It is
Expand Down Expand Up @@ -540,100 +356,6 @@ status:
If the `index.json` file is not produced, the image digest will not be included
in the `taskRun` output.

### Storage Resource

The `storage` resource represents blob storage, that contains either an object
or directory. Adding the storage resource as an input to a `Task` will download
the blob and allow the `Task` to perform the required actions on the contents of
the blob.

Only blob storage type
[Google Cloud Storage](https://cloud.google.com/storage/)(gcs) is supported as
of now via [GCS storage resource](#gcs-storage-resource).

#### GCS Storage Resource

The `gcs` storage resource points to
[Google Cloud Storage](https://cloud.google.com/storage/) blob.

To create a GCS type of storage resource using the `PipelineResource` CRD:

```yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: wizzbang-storage
namespace: default
spec:
type: storage
params:
- name: type
value: gcs
- name: location
value: gs://some-bucket
- name: dir
value: "y" # This can have any value to be considered "true"
```

Params that can be added are the following:

1. `location`: represents the location of the blob storage.
1. `type`: represents the type of blob storage. For GCS storage resource this
value should be set to `gcs`.
1. `dir`: represents whether the blob storage is a directory or not. By default
a storage artifact is not considered a directory.

- If the artifact is a directory then `-r`(recursive) flag is used, to
copy all files under the source directory to a GCS bucket. Eg: `gsutil
cp -r source_dir/* gs://some-bucket`
- If an artifact is a single file like a zip or tar, then the copy will be
only 1 level deep(not recursive). It will not trigger a copy of sub
directories in the source directory. Eg: `gsutil cp source.tar
gs://some-bucket.tar`.

Private buckets can also be configured as storage resources. To access GCS
private buckets, service accounts with correct permissions are required. The
`secrets` field on the storage resource is used for configuring this
information. Below is an example on how to create a storage resource with a
service account.

1. Refer to the
[official documentation](https://cloud.google.com/compute/docs/access/service-accounts)
on how to create service accounts and configuring
[IAM permissions](https://cloud.google.com/storage/docs/access-control/iam-permissions)
to access buckets.

1. Create a Kubernetes secret from a downloaded service account json key

```bash
kubectl create secret generic bucket-sa --from-file=./service_account.json
```

1. To access the GCS private bucket environment variable
[`GOOGLE_APPLICATION_CREDENTIALS`](https://cloud.google.com/docs/authentication/production)
should be set, so apply the above created secret to the GCS storage resource
under the `fieldName` key.

```yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: wizzbang-storage
namespace: default
spec:
type: storage
params:
- name: type
value: gcs
- name: location
value: gs://some-private-bucket
- name: dir
value: "y"
secrets:
- fieldName: GOOGLE_APPLICATION_CREDENTIALS
secretName: bucket-sa
secretKey: service_account.json
```

--------------------------------------------------------------------------------

Expand Down
23 changes: 0 additions & 23 deletions docs/variables.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,21 +84,6 @@ of variables. This section lists the variables exposed by each type. You can acc
variable via `resources.inputs.<resourceName>.<variableName>` or
`resources.outputs.<resourceName>.<variableName>`.

#### Variables for the `Git` type

| Variable | Description |
| -------- | ----------- |
| `name` | The name of the resource. |
| `type` | Type value of `"git"`. |
| `url` | The URL of the Git repository. |
| `revision` | The revision to check out. |
| `refspec` | The value of the resource's `refspec` parameter. |
| `depth` | The integer value of the resource's `depth` parameter. |
| `sslVerify` | The value of the resource's `sslVerify` parameter, either `"true"` or `"false"`. |
| `httpProxy` | The value of the resource's `httpProxy` parameter. |
| `httpsProxy` | The value of the resource's `httpsProxy` parameter. |
| `noProxy` | The value of the resource's `noProxy` parameter. |

#### Variables for the `Image` type

| Variable | Description |
Expand All @@ -108,14 +93,6 @@ variable via `resources.inputs.<resourceName>.<variableName>` or
| `url` | The complete path to the image. |
| `digest` | The digest of the image. |

#### Variables for the `GCS` type

| Variable | Description |
| -------- | ----------- |
| `name` | The name of the resource. |
| `type` | Type value of `"gcs"`. |
| `location` | The fully qualified address of the blob storage. |

#### Variables for the `Cluster` type

| Variable | Description |
Expand Down
Loading

0 comments on commit 4d876da

Please sign in to comment.