Skip to content

Commit

Permalink
Split into two commands/RPMs
Browse files Browse the repository at this point in the history
  • Loading branch information
sevagh committed Mar 6, 2018
1 parent 52aaa26 commit 8caf23d
Show file tree
Hide file tree
Showing 16 changed files with 193 additions and 88 deletions.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
dist/
goat
bin/
rpm-package/ebs/goat-ebs
rpm-package/eni/goat-eni
52 changes: 26 additions & 26 deletions GNUmakefile
Original file line number Diff line number Diff line change
@@ -1,50 +1,50 @@
VERSION:=0.4.1
VERSION:=0.5.0
GOAT_FILES?=$$(find . -name '*.go' | grep -v vendor)
GOAT_NAME=$(notdir $(shell pwd))
GOAT_CMDS=$(shell find cmd/ -maxdepth 1 -mindepth 1 -type d)

STATIC_ENV:=CGO_ENABLED=0 GOOS=linux GOARCH=amd64
STATIC_FLAGS:=-a -tags netgo -ldflags '-extldflags "-static" -X main.VERSION=$(VERSION)'
RELEASE_FLAGS:=-a -tags netgo -ldflags '-w -extldflags "-static" -X main.VERSION=$(VERSION)'

DIST_DIR=dist/
BIN_DIR=dist/bin

all: build_static

builddir:
@mkdir -p $(DIST_DIR) $(BIN_DIR)

build: builddir deps
@cd cmd/goat && go build $(DYNAMIC_FLAGS) -o ../../$(BIN_DIR)/$(GOAT_NAME)

build_static: builddir deps
@cd cmd/goat && go build $(STATIC_FLAGS) -o ../../$(BIN_DIR)/$(GOAT_NAME)
build: deps
@$(foreach cmd,$(GOAT_CMDS),\
cd $(cmd) &&\
$(STATIC_ENV) go build $(STATIC_FLAGS) \
-o ../../bin/$(notdir $(cmd)) &&\
cd - 2>&1 >/dev/null;)

release: builddir deps
@cd cmd/goat && go build $(RELEASE_FLAGS) -o ../../$(BIN_DIR)/$(GOAT_NAME)

deps:
@go get -u github.com/golang/dep/cmd/dep
@dep ensure
release: deps
@$(foreach cmd,$(GOAT_CMDS),\
cd $(cmd) &&\
$(STATIC_ENV) go build $(RELEASE_FLAGS) \
-o ../../bin/$(notdir $(cmd)) &&\
cd - 2>&1 >/dev/null;)

test:
@$(foreach cmd,$(GOAT_CMDS),\
go vet ./$(cmd) &&\
go test -v ./$(cmd);)
@go vet ./pkg/...
@go vet ./cmd/goat/...
@go test -v ./pkg/...
@go test -v ./cmd/goat/...

lint: lintsetup
deps:
@command -v dep 2>&1 >/dev/null || go get -u github.com/golang/dep/cmd/dep
@dep ensure

lint:
@gofmt -s -w $(GOAT_FILES)
@gometalinter.v2 --enable-all $(GOAT_FILES) --exclude=_test.go
-gometalinter.v2 --enable-all $(GOAT_FILES) --exclude=_test.go

lintsetup:
@go get -u gopkg.in/alecthomas/gometalinter.v2
@gometalinter.v2 --install 2>&1 >/dev/null

clean:
-rm -rf build
-rm -rf bin

package:
@GOAT_VERSION=$(VERSION) $(MAKE) -C ./centos-package/
package: release
@GOAT_VERSION=$(VERSION) $(MAKE) -C ./rpm-package/

.PHONY: clean test
40 changes: 11 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
[![ReportCard](http://goreportcard.com/badge/sevagh/goat)](http://goreportcard.com/report/sevagh/goat) [![GitHub tag](https://img.shields.io/github/tag/sevagh/goat.svg)](https://github.com/sevagh/goat/releases)

*VERY EARLY ALPHA - USE AT YOUR OWN RISK*

# goat :goat:

### Attach EBS volumes and ENIs to running EC2 instances
Expand All @@ -12,35 +10,22 @@ By setting your tags correctly, `goat` can discover and attach EBS volumes and E

Furthermore, for EBS volumes, it can perform additional actions such as RAID (with mdadm), mkfs, and mount EBS volumes to the EC2 instance where it's running.

The `goat` package consists on the subcommands [goat-ebs](./cmd/goat-ebs/README.md) and [goat-eni](./cmd/goat-eni/README.md).

### Permission model

It's necessary for the instance to have an IAM Role with _at least_ access to the EBS and ENI resources that it will be attaching - see [here](./hcl-example/iam_role.tf). Your roles can be even more permissive (i.e. full EC2 access) but that comes with its own risks.

Unfortunately, resource-level permissions are [currently not supported](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/ec2-api-permissions.html#ec2-api-unsupported-resource-permissions) for attaching network interfaces. This means that to use `goat@eni`, your instances must have full permissions for __all__ ENIs.

### Motivation

The Terraform resource `aws_volume_attachment` isn't handled well when destroying a stack. See [here](https://github.com/hashicorp/terraform/issues/9000) for some discussion on the matter. We initially wrote instance-specific user-data shell scripts with hardcoded values (e.g. `mkfs.ext4 /dev/xvdb`, `mount /dev/xvdb /var/kafka_data`). With `goat` we can avoid needing to pass parameters or hardcoding values. All the required information comes from the EC2 instance and EBS volume tags.

### Subcommands

`goat` for now supports the subcommands `goat ebs` for EBS volumes and `goat eni` for ENIs.

Docs:

* [ebs](./pkg/commands/ebs/README.md)
* [eni](./pkg/commands/eni/README.md)

### RPM-based install

Goat is systemd-based and has been developed for CentOS. Install the rpm from the releases page:

```
$ sudo yum install -y https://github.com/sevagh/goat/releases/download/0.4.1/goat-0.4.1-1.fc25.x86_64.rpm
$ sudo systemctl enable goat@ebs
$ sudo systemctl start goat@ebs
$ ...
$ journalctl -u goat@ebs
$ sudo yum install -y https://github.com/sevagh/goat/releases/download/0.4.1/goat-ebs-0.4.2-1.fc25.x86_64.rpm
$ sudo systemctl enable goat-ebs
$ sudo systemctl start goat-ebs
```

### Additional dependencies for ENI
Expand All @@ -52,15 +37,12 @@ Refer to [this](./commands/eni#setting-up-the-eni---ec2-net-utils) document. It
A fully working chunk of `ec2 user-data` with `goat` looks like [this](./hcl-example/blob/master/bootstrap.tpl#L8):

```
yum install -y wget mdadm
yum install -y https://github.com/sevagh/goat/releases/download/0.4.0/goat-0.4.0-1.fc25.x86_64.rpm
yum install -y https://github.com/sevagh/ec2-utils/releases/download/v0.5.3/ec2-net-utils-0.5-2.fc25.noarch.rpm
systemctl enable elastic-network-interfaces
systemctl start elastic-network-interfaces
systemctl enable goat@ebs
systemctl enable goat@eni
systemctl start goat@ebs
systemctl start goat@eni
$ sudo yum install -y https://github.com/sevagh/goat/releases/download/0.4.0/goat-eni-0.4.2-1.fc25.x86_64.rpm
$ sudo yum install -y https://github.com/sevagh/ec2-utils/releases/download/v0.5.3/ec2-net-utils-0.5-2.fc25.noarch.rpm
$ sudo systemctl enable elastic-network-interfaces
$ sudo systemctl start elastic-network-interfaces
$ sudo systemctl enable goat-eni
$ sudo systemctl start goat-eni
```

### Examples
Expand Down
1 change: 0 additions & 1 deletion centos-package/.gitignore

This file was deleted.

4 changes: 0 additions & 4 deletions centos-package/GNUmakefile

This file was deleted.

File renamed without changes.
2 changes: 1 addition & 1 deletion pkg/commands/ebs/ebs.go → cmd/goat-ebs/ebs.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package ebs
package main

import (
log "github.com/sirupsen/logrus"
Expand Down
17 changes: 4 additions & 13 deletions cmd/goat/main.go → cmd/goat-ebs/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,16 @@ import (
"github.com/docopt/docopt-go"
log "github.com/sirupsen/logrus"
"os"

"github.com/sevagh/goat/pkg/commands/ebs"
"github.com/sevagh/goat/pkg/commands/eni"
)

//Goat version substituted by the Makefile
var VERSION string

func main() {
usage := `goat - EC2/EBS utility
usage := `goat - EBS attach utility
Usage:
goat ebs [--log-level=<log-level>] [--dry] [--debug]
goat eni [--log-level=<log-level>] [--dry] [--debug]
goat [--log-level=<log-level>] [--dry] [--debug]
goat -h | --help
goat --version
Expand All @@ -43,11 +39,6 @@ Options:
dryRun := arguments["--dry"].(bool)
debug := arguments["--debug"].(bool)

if arguments["ebs"].(bool) {
log.Printf("Running goat for EBS")
ebs.GoatEbs(dryRun, debug)
} else if arguments["eni"].(bool) {
log.Printf("Running goat for ENI")
eni.GoatEni(dryRun, debug)
}
log.Printf("Running goat for EBS")
GoatEbs(dryRun, debug)
}
File renamed without changes.
2 changes: 1 addition & 1 deletion pkg/commands/eni/eni.go → cmd/goat-eni/eni.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package eni
package main

import (
log "github.com/sirupsen/logrus"
Expand Down
44 changes: 44 additions & 0 deletions cmd/goat-eni/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package main

import (
"fmt"
"github.com/docopt/docopt-go"
log "github.com/sirupsen/logrus"
"os"
)

//Goat version substituted by the Makefile
var VERSION string

func main() {
usage := `goat - ENI attach utility
Usage:
goat [--log-level=<log-level>] [--dry] [--debug]
goat -h | --help
goat --version
Options:
--log-level=<level> Log level (debug, info, warn, error, fatal) [default: info]
--dry Dry run
--debug Interactive prompts to continue between phases
-h --help Show this screen.
--version Show version.`
arguments, _ := docopt.Parse(usage, nil, true, fmt.Sprintf("goat %s", VERSION), false)

log.SetOutput(os.Stderr)
logLevel := arguments["--log-level"].(string)
if level, err := log.ParseLevel(logLevel); err != nil {
log.Fatalf("%v", err)
} else {
log.SetLevel(level)
}

log.SetFormatter(&log.TextFormatter{})

dryRun := arguments["--dry"].(bool)
debug := arguments["--debug"].(bool)

log.Printf("Running goat for ENI")
GoatEni(dryRun, debug)
}
13 changes: 13 additions & 0 deletions rpm-package/GNUmakefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
all: ebs eni

ebs:
-rpmlint ebs/specfile.spec
@cd ebs && cp ../../bin/goat-ebs . &&\
rpmbuild -ba specfile.spec --define "_sourcedir $$PWD" --define "_version $$GOAT_VERSION"

eni:
-rpmlint eni/specfile.spec
@cd eni && cp ../../bin/goat-eni . &&\
rpmbuild -ba specfile.spec --define "_sourcedir $$PWD" --define "_version $$GOAT_VERSION"

.PHONY: ebs eni
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
[Unit]
Description=GOAT: EC2-%i attach utility
Description=GOAT: EC2-EBS attach utility
Documentation=https://github.com/sevagh/goat
Requires=network.target remote-fs.target
After=network.target remote-fs.target
ConditionPathExists=/usr/bin/goat
ConditionPathExists=/usr/bin/goat-ebs

[Service]
Type=oneshot
User=root
Group=root
ExecStart=/usr/bin/goat "%i" --log-level=info
SyslogIdentifier=goat
ExecStart=/usr/bin/goat-ebs --log-level=info
SyslogIdentifier=goat-ebs

[Install]
WantedBy=multi-user.target
61 changes: 61 additions & 0 deletions rpm-package/ebs/specfile.spec
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
%define pkgname goat-ebs

Name: %{pkgname}
Version: %{_version}
Release: 1%{?dist}
Summary: Attach and mount elastic block store volumes

License: BSD 3-clause
URL: https://github.com/sevagh/goat
Source0: %{pkgname}
Source1: %{pkgname}.service

Requires: systemd mdadm


%description
Automatically attach AWS resources to a running EC2 instance.


#%prep
#%setup
#%build


%install
%{__mkdir} -p %{buildroot}/%{_bindir}
%{__mkdir} -p %{buildroot}/%{_unitdir}
%{__install} -m0775 %{SOURCE0} %{buildroot}/%{_bindir}/%{pkgname}
%{__install} -m0777 %{SOURCE1} %{buildroot}/%{_unitdir}/%{pkgname}.service


%files
%{_bindir}/%{pkgname}
%{_unitdir}/%{pkgname}.service


%post
if [ $1 -eq 1 ]; then
/bin/systemctl daemon-reload >/dev/null 2>&1 || :
fi


%preun
if [ $1 -eq 0 ] ; then
# Package removal, not upgrade
/bin/systemctl disable %{pkgname}.service >/dev/null 2>&1 || :
/bin/systemctl stop %{pkgname}.service >/dev/null 2>&1 || :
fi


%postun
/bin/systemctl daemon-reload >/dev/null 2>&1 || :


%changelog
* Tue Mar 06 2018 Sevag Hanssian <sevag.hanssian@gmail.com>
- Split subcommands into two binaries
* Thu Aug 10 2017 Sevag Hanssian <sevag.hanssian@gmail.com>
- Goat subcommands
* Tue Jul 11 2017 Sevag Hanssian <sevag.hanssian@gmail.com>
- First RPM package for goat
16 changes: 16 additions & 0 deletions rpm-package/eni/goat-eni.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[Unit]
Description=GOAT: EC2-ENI attach utility
Documentation=https://github.com/sevagh/goat
Requires=network.target remote-fs.target
After=network.target remote-fs.target
ConditionPathExists=/usr/bin/goat-eni

[Service]
Type=oneshot
User=root
Group=root
ExecStart=/usr/bin/goat-eni --log-level=info
SyslogIdentifier=goat-eni

[Install]
WantedBy=multi-user.target
Loading

0 comments on commit 8caf23d

Please sign in to comment.