Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add option to use amend when updating state to reduce size #46

Merged
merged 6 commits into from
Oct 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.19'
go-version: '1.23'
- name: Checkout Code
uses: actions/checkout@v3
- name: Build Code
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
on:
push:
tags:
- 'v*'
- "v*"

name: Create Release

Expand All @@ -26,7 +26,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.19'
go-version: "1.23"
- name: Set RELEASE_VERSION
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/v}" >> $GITHUB_ENV
- name: Set RELEASE_STRING
Expand Down Expand Up @@ -105,4 +105,4 @@ jobs:
ALLOW_EMPTY_CHANGELOG: "false"
with:
args: |
${{ env.ARTIFACTS }}
${{ env.ARTIFACTS }}
9 changes: 7 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- New `--amend` flag to use commit amend + force push when updating state to maintain repo size

## [0.1.6] - 2023-09-21

### Fixed
Expand Down Expand Up @@ -34,7 +39,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Go to 1.19
- Debian image to `bullseye`
- Various Go dependencies (except for go-git due to backward incompatible changes in https://github.com/go-git/go-git/pull/548)
- Various Go dependencies (except for go-git due to backward incompatible changes in <https://github.com/go-git/go-git/pull/548>)

## [0.1.2] - 2022-05-15

Expand Down Expand Up @@ -83,7 +88,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- GitHub deprecated `set-env`; replaced with https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#environment-files
- GitHub deprecated `set-env`; replaced with <https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#environment-files>

## [0.0.14] - 2020-05-30

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.19 AS build
FROM golang:1.23 AS build

ARG RELEASE_STRING=dev
ENV IMPORT_PATH="github.com/plumber-cd/terraform-backend-git/cmd"
Expand All @@ -8,7 +8,7 @@ RUN mkdir bin && go build \
-ldflags "-X ${IMPORT_PATH}.Version=${RELEASE_STRING}" \
-o ./bin ./...

FROM debian:bullseye
FROM debian:bookworm

# Include CA Certs to resolve TLS handshakes
RUN DEBIAN_FRONTEND="noninteractive" apt-get update && apt-get install -y ca-certificates && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,15 @@ Don't forget to add it to your `PATH`.

#### As Docker Image

See https://github.com/plumber-cd/terraform-backend-git/pkgs/container/terraform-backend-git.
See <https://github.com/plumber-cd/terraform-backend-git/pkgs/container/terraform-backend-git>.

```bash
docker pull ghcr.io/plumber-cd/terraform-backend-git:latest
```

#### As Github Action

See https://github.com/marketplace/actions/setup-terraform-backend-git.
See <https://github.com/marketplace/actions/setup-terraform-backend-git>.

```yaml
steps:
Expand Down Expand Up @@ -189,6 +189,7 @@ CLI | `terraform-backend-git.hcl` | Environment Variable | TF HTTP backend confi
`--repository` | `git.repository` | `TF_BACKEND_GIT_GIT_REPOSITORY` |`repository` | Required; Which repository to use for storing TF state?
`--ref` | `git.ref` | `TF_BACKEND_GIT_GIT_REF` |`ref` | Optional; Which branch to use in that `repository`? Default: `master`.
`--state` | `git.state` | `TF_BACKEND_GIT_GIT_STATE` | `state` | Required; Path to the state file in that `repository`.
`--amend` | `git.amend` | `TF_BACKEND_GIT_GIT_AMEND` | `amend` | Optional; whether to use git amend + force push to update state file.
`--config` | - | - | - | Optional; Path to the `hcl` config file.
`--address` | `address` | `TF_BACKEND_GIT_ADDRESS` | - | Optional; Local binding address and port to listen for HTTP requests. Only change the port, **do not change the address to `0.0.0.0` before you read [Running backend remotely](#running-backend-remotely)**. Default: `127.0.0.1:6061`.
`--access-logs` | `accessLogs` | `TF_BACKEND_GIT_ACCESSLOGS` | - | Optional; Set to `true` to enable HTTP access logs on backend. Default: `false`.
Expand Down Expand Up @@ -283,9 +284,9 @@ Note that if either username or password changes - Terraform will consider this

### Why not native Terraform Backend

Unfortunately, Terraform Backends is not pluggable like Providers are, see https://github.com/hashicorp/terraform/issues/5877.
Unfortunately, Terraform Backends is not pluggable like Providers are, see <https://github.com/hashicorp/terraform/issues/5877>.

Due to this, I couldn't make a proper native Terraform backend implementation for Git, it should have been implemented and added to https://github.com/hashicorp/terraform code base. There is an open ticket to do it https://github.com/hashicorp/terraform/issues/24603, but it is unclear when this would happen ([if it will at all](https://github.com/hashicorp/terraform/issues/24603#issuecomment-613533258)). That said I figured this HTTP backend implementation might be useful for the time being.
Due to this, I couldn't make a proper native Terraform backend implementation for Git, it should have been implemented and added to <https://github.com/hashicorp/terraform> code base. There is an open ticket to do it <https://github.com/hashicorp/terraform/issues/24603>, but it is unclear when this would happen ([if it will at all](https://github.com/hashicorp/terraform/issues/24603#issuecomment-613533258)). That said I figured this HTTP backend implementation might be useful for the time being.

## Why storing state in Git

Expand Down
6 changes: 5 additions & 1 deletion cmd/git_backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ var gitBackendCmd = &cobra.Command{
t, err := template.New(gitHTTPBackendConfigPath).Parse(`
terraform {
backend "http" {
address = "{{ .protocol }}://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
address = "{{ .protocol }}://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}{{ if eq .amend "true" }}&amend=true{{ end }}&state={{ .state }}"
dee-kryvenko marked this conversation as resolved.
Show resolved Hide resolved
lock_address = "{{ .protocol }}://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
unlock_address = "{{ .protocol }}://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
skip_cert_verification = {{ .skipHttpsVerification }}
Expand Down Expand Up @@ -76,6 +76,7 @@ terraform {
log.Fatalf("%s must be set", flag)
}
}
p["amend"] = viper.GetString("git.amend")

backendConfig, err := os.OpenFile(gitHTTPBackendConfigPath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
Expand Down Expand Up @@ -107,6 +108,9 @@ func init() {
gitBackendCmd.PersistentFlags().StringP("state", "s", "", "Ref (branch) to use")
viper.BindPFlag("git.state", gitBackendCmd.PersistentFlags().Lookup("state"))

gitBackendCmd.PersistentFlags().Bool("amend", false, "Use git amend to store updated state")
viper.BindPFlag("git.amend", gitBackendCmd.PersistentFlags().Lookup("amend"))

gitBackendCmd.PersistentFlags().StringP("dir", "d", "", "Change current working directory")
viper.BindPFlag("git.dir", gitBackendCmd.PersistentFlags().Lookup("dir"))

Expand Down
37 changes: 18 additions & 19 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
module github.com/plumber-cd/terraform-backend-git

go 1.19
go 1.23

require (
github.com/go-git/go-billy/v5 v5.4.1
github.com/go-git/go-git/v5 v5.4.2
github.com/go-git/go-billy/v5 v5.5.0
github.com/go-git/go-git/v5 v5.12.0
github.com/gorilla/handlers v1.5.1
github.com/mitchellh/go-homedir v1.1.0
github.com/spf13/cobra v1.6.1
github.com/spf13/viper v1.15.0
github.com/xanzy/ssh-agent v0.3.3
go.mozilla.org/sops/v3 v3.7.3
golang.org/x/crypto v0.6.0
golang.org/x/crypto v0.27.0
golang.org/x/exp v0.0.0-20230213192124-5e25df0256eb
golang.org/x/sys v0.5.0
golang.org/x/sys v0.25.0
)

require (
cloud.google.com/go/compute v1.18.0 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
dario.cat/mergo v1.0.1 // indirect
filippo.io/age v1.1.1 // indirect
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
Expand All @@ -31,20 +32,20 @@ require (
github.com/Azure/go-autorest/autorest/validation v0.3.1 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230214155104-81033d7f4442 // indirect
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/go-crypto v1.0.0 // indirect
github.com/aws/aws-sdk-go v1.44.202 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/cenkalti/backoff/v3 v3.2.2 // indirect
github.com/cloudflare/circl v1.3.2 // indirect
github.com/cloudflare/circl v1.4.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/cyphar/filepath-securejoin v0.3.3 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fatih/color v1.14.1 // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-git/gcfg v1.5.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/golang-jwt/jwt/v4 v4.4.3 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
Expand All @@ -64,7 +65,6 @@ require (
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/vault/api v1.9.0 // indirect
github.com/howeyc/gopass v0.0.0-20210920133722-c8aef6fb66ef // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
Expand All @@ -76,26 +76,25 @@ require (
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/pelletier/go-toml/v2 v2.0.6 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/ryanuber/go-glob v1.0.0 // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.3.0 // indirect
github.com/spf13/afero v1.9.3 // indirect
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
go.mozilla.org/gopgagent v0.0.0-20170926210634-4d7ea76ff71a // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/net v0.29.0 // indirect
golang.org/x/oauth2 v0.5.0 // indirect
golang.org/x/term v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/term v0.24.0 // indirect
golang.org/x/text v0.18.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.6.0 // indirect
google.golang.org/api v0.110.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230209215440-0dfe4f8abfcc // indirect
Expand Down
Loading
Loading