Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make drift check and apply with custom runner code #2

Merged
merged 42 commits into from
Dec 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
afbe55e
chore(wip): wip
Alan-pad Nov 24, 2022
15f13a8
chore(evaluate): change function signature
Alan-pad Nov 24, 2022
87c3d6d
feat: add compute hash func
spoukke Nov 24, 2022
5ceba86
chore(signature): again changing function signature
Alan-pad Nov 24, 2022
30d2d16
feat: add evaluate for isTerraformRunning and isPlanArtifactUpToDDate
spoukke Nov 24, 2022
8645fe2
feat(conditions): terraformFailure TerraformApplyUpToDate
Alan-pad Nov 24, 2022
b6852fc
feat: use const prefixes for cahce keys
spoukke Nov 24, 2022
026bdaf
chore(tests): add some units tests for conditions
Alan-pad Dec 5, 2022
f19aaf4
build: generate deepcopy and manifests
spoukke Dec 5, 2022
99469c1
feat: fix timestamp conversion
spoukke Dec 5, 2022
8ceb75b
test: implement testing for all evaluate methods of terraform conditions
spoukke Dec 5, 2022
b4cea0f
chore: rename structs
spoukke Dec 5, 2022
ba4b83d
chore: uniformize every sub evaluate mehtod
spoukke Dec 5, 2022
4a993ae
test: start implementing terraform layer conditions tests
spoukke Dec 5, 2022
6a44b2b
test: implement terraform layer conditions testing
spoukke Dec 5, 2022
47ca78c
chore: remove unused struct
spoukke Dec 5, 2022
e873c28
feat: start working on job
spoukke Dec 8, 2022
89a4c2f
feat: fix init default pod spec
spoukke Dec 12, 2022
49be385
feat: fix init default pod spec
spoukke Dec 15, 2022
5a25c7b
chore(wip): runner
Alan-pad Dec 15, 2022
c05749f
feat: finish cache logic for plan pod creation
spoukke Dec 15, 2022
420ed36
feat: add command for apply pod
spoukke Dec 15, 2022
811e165
feat: implement redis cache
spoukke Dec 16, 2022
8e71e38
improvement(runner): add runner code, init cobra, move to cache package
Alan-pad Dec 19, 2022
29204d7
feat: remove applied bin cahce key
spoukke Dec 21, 2022
3faf6d9
feat(cobra): implement cobra launch
Alan-pad Dec 21, 2022
3774fc5
chore(runner): modify pod creation to use burrito binary
Alan-pad Dec 21, 2022
92634b5
chore(docker): and crd generation
Alan-pad Dec 21, 2022
53415ab
feat: use config to setup cahce in layer controller
spoukke Dec 21, 2022
1275c76
feat: run exec on cotnrollers start
spoukke Dec 21, 2022
f0c96dc
feat: use custom image for runeer and generate random name
spoukke Dec 21, 2022
aa75fdb
fix: do not inpu error 2 times to logger
spoukke Dec 21, 2022
a78b505
fix(runner): pod generation
Alan-pad Dec 21, 2022
4536cbc
fix(runner): pod generation remove command
Alan-pad Dec 21, 2022
7ebd8a5
fix: change prefix in pod env vars
spoukke Dec 21, 2022
47af2fe
feat: catch set lock error
spoukke Dec 21, 2022
ead33c9
feat: use commen generate key function for cahce keys
spoukke Dec 21, 2022
155c17c
fix(all): add some logs on controller and runner, fix some cache dele…
Alan-pad Dec 21, 2022
b3d51f4
feat(redis): implement delete func
Alan-pad Dec 21, 2022
f48eee3
improvement(runner): no sé
Alan-pad Dec 21, 2022
fd3b920
feat: make runner put last plan date in cache
spoukke Dec 22, 2022
2452f8b
fix: use unix timestamp as value for last plan date key in cache
spoukke Dec 22, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,24 @@ RUN go mod download
COPY main.go main.go
COPY api/ api/
COPY controllers/ controllers/
COPY burrito/ burrito/
COPY cmd/ cmd/
COPY runner/ runner/
COPY cache/ cache

# Build
# the GOARCH has not a default value to allow the binary be built according to the host where the command
# was called. For example, if we call make docker-build in a local env which has the Apple Silicon M1 SO
# the docker BUILDPLATFORM arg will be linux/arm64 when for Apple x86 it will be linux/amd64. Therefore,
# by leaving it empty we can ensure that the container and binary shipped on it will have the same platform.
RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go
RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o bin/burrito main.go

FROM golang:alpine

RUN apk add --update git bash openssh

# Use distroless as minimal base image to package the manager binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM gcr.io/distroless/static:nonroot
WORKDIR /
COPY --from=builder /workspace/manager .
COPY --from=builder /workspace/bin/burrito .
USER 65532:65532

ENTRYPOINT ["/manager"]
ENTRYPOINT ["/burrito"]
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ test: manifests generate fmt vet envtest ## Run tests.

.PHONY: build
build: generate fmt vet ## Build manager binary.
go build -o bin/manager main.go
go build -o bin/burrito main.go

.PHONY: run
run: manifests generate fmt vet ## Run a controller from your host.
Expand Down
24 changes: 20 additions & 4 deletions api/v1alpha1/terraformlayer_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,30 @@ type TerraformLayerSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file

// Foo is an example field of TerraformLayer. Edit terraformlayer_types.go to remove/update
Foo string `json:"foo,omitempty"`
Path string `json:"path,omitempty"`
Branch string `json:"branch,omitempty"`
TerraformVersion string `json:"terraformVersion,omitempty"`
Repository TerraformLayerRepository `json:"repository,omitempty"`
RemediationStrategy TerraformLayerRemediationStrategy `json:"remediationStrategy,omitempty"`
PlanOnPullRequest bool `json:"planOnPullRequest,omitempty"`
// RunnerPodTemplate corev1.PodSpec `json:"template,omitempty"`
}

type TerraformLayerRemediationStrategy struct {
PlanOnDrift bool `json:"planOnDrift,omitempty"`
ApplyOnDrift bool `json:"applyOnDrift,omitempty"`
ApplyOnPush bool `json:"applyOnPush,omitempty"`
}

type TerraformLayerRepository struct {
Kind string `json:"kind,omitempty"`
Name string `json:"name,omitempty"`
Namespace string `json:"namespace,omitempty"`
}

// TerraformLayerStatus defines the observed state of TerraformLayer
type TerraformLayerStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
}

//+kubebuilder:object:root=true
Expand Down
12 changes: 8 additions & 4 deletions api/v1alpha1/terraformrepository_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ limitations under the License.
package v1alpha1

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

Expand All @@ -28,14 +29,17 @@ type TerraformRepositorySpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file

// Foo is an example field of TerraformRepository. Edit terraformrepository_types.go to remove/update
Foo string `json:"foo,omitempty"`
Repository TerraformRepositoryRepository `json:"repository,omitempty"`
}

type TerraformRepositoryRepository struct {
Url string `json:"url,omitempty"`
SecretRef corev1.SecretReference `json:"secretRef,omitempty"`
}

// TerraformRepositoryStatus defines the observed state of TerraformRepository
type TerraformRepositoryStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
}

//+kubebuilder:object:root=true
Expand Down
68 changes: 66 additions & 2 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions burrito/burrito.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package burrito

import (
"io"
"os"

"github.com/padok-team/burrito/burrito/config"
"github.com/padok-team/burrito/controllers"
"github.com/padok-team/burrito/runner"
)

type App struct {
Config *config.Config

Runner Runner
Controllers Controllers

Out io.Writer
Err io.Writer
}

type Runner interface {
Exec()
}

type Controllers interface {
Exec()
}

func New() (*App, error) {
c := &config.Config{}
app := &App{
Config: c,
Runner: runner.New(c),
Controllers: controllers.New(c),
Out: os.Stdout,
Err: os.Stderr,
}
return app, nil
}
112 changes: 112 additions & 0 deletions burrito/config/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
package config

import (
"fmt"
"os"
"reflect"
"strings"

"github.com/spf13/pflag"
"github.com/spf13/viper"
)

type Config struct {
Runner RunnerConfig `yaml:"runner"`
Controller ControllerConfig `yaml:"controller"`
Redis Redis `yaml:"redis"`
}

type ControllerConfig struct {
WatchedNamespaces []string `yaml:"namespaces"`
}

type RepositoryConfig struct {
URL string `yaml:"url"`
SSH string `yaml:"ssh"`
Username string `yaml:"username"`
Password string `yaml:"password"`
}

type RunnerConfig struct {
Path string `yaml:"path"`
Branch string `yaml:"branch"`
Version string `yaml:"version"`
Action string `yaml:"action"`
Repository RepositoryConfig `yaml:"repository"`
Layer LayerConfig `yaml:"layer"`
}

type LayerConfig struct {
Lock string `yaml:"lock"`
PlanSum string `yaml:"planSum"`
PlanBin string `yaml:"planBin"`
ApplySum string `yaml:"applySum"`
PlanDate string `yaml:"planDate"`
}

type Redis struct {
URL string `yaml:"url"`
Password string `yaml:"password"`
Database int `yaml:"database"`
}

func (c *Config) Load(flags *pflag.FlagSet) error {
v := viper.New()

// burrito looks for configuration files called config.yaml, config.json,
// config.toml, config.hcl, etc.
v.SetConfigName("config")

// burrito looks for configuration files in the common configuration
// directories.
v.AddConfigPath("/etc/burrito/")
v.AddConfigPath("$HOME/.burrito/")

// Viper logs the configuration file it uses, if any.
if err := v.ReadInConfig(); err == nil {
fmt.Fprintf(os.Stderr, "Using config file: %s\n", v.ConfigFileUsed())
}

// burrito can be configured with environment variables that start with
// burrito_.
v.SetEnvPrefix("burrito")
v.AutomaticEnv()

// Options with dashes in flag names have underscores when set inside a
// configuration file or with environment variables.
flags.SetNormalizeFunc(func(fs *pflag.FlagSet, name string) pflag.NormalizedName {
name = strings.ReplaceAll(name, "-", "_")
return pflag.NormalizedName(name)
})
v.BindPFlags(flags)

// Nested configuration options set with environment variables use an
// underscore as a separator.
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
bindEnvironmentVariables(v, *c)

return v.Unmarshal(c)
}

// bindEnvironmentVariables inspects iface's structure and recursively binds its
// fields to environment variables. This is a workaround to a limitation of
// Viper, found here:
// https://github.com/spf13/viper/issues/188#issuecomment-399884438
func bindEnvironmentVariables(v *viper.Viper, iface interface{}, parts ...string) {
ifv := reflect.ValueOf(iface)
ift := reflect.TypeOf(iface)
for i := 0; i < ift.NumField(); i++ {
val := ifv.Field(i)
typ := ift.Field(i)
tv, ok := typ.Tag.Lookup("yaml")
if !ok {
continue
}
switch val.Kind() {
case reflect.Struct:
bindEnvironmentVariables(v, val.Interface(), append(parts, tv)...)
default:
v.BindEnv(strings.Join(append(parts, tv), "."))
}
}
}
6 changes: 6 additions & 0 deletions burrito/controllers.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package burrito

func (app *App) StartController() error {
app.Controllers.Exec()
return nil
}
5 changes: 5 additions & 0 deletions burrito/runner.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package burrito

func (app *App) StartRunner() {
app.Runner.Exec()
}
Loading