Skip to content
This repository has been archived by the owner on Aug 29, 2023. It is now read-only.

Commit

Permalink
Merge pull request #44 from luxas/sample-readme
Browse files Browse the repository at this point in the history
Split sample-app to that and sample-watch
  • Loading branch information
twelho authored Aug 21, 2020
2 parents 91096ca + 1ccaade commit 69889c6
Show file tree
Hide file tree
Showing 5 changed files with 203 additions and 60 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ SRC_PKGS := cmd pkg
DOCKER_ARGS := --rm
CACHE_DIR := $(shell pwd)/bin/cache
API_DOCS := api/sample-app.md api/runtime.md
BINARIES := bin/sample-app bin/sample-gitops
BINARIES := bin/sample-app bin/sample-gitops bin/sample-watch

# If we're not running in CI, run Docker interactively
ifndef CI
Expand Down
105 changes: 105 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,111 @@ See the [`pkg/gitdir`](pkg/gitdir) package for details.
This package contains utilities used by the rest of the library. The most interesting thing here is the `Patcher`
under [`pkg/util/patch`](pkg/util/patch), which can be used to apply patches to `pkg/runtime.Object` compliant types.

## Sample implementations

All sample binaries in this repo are operating on a sample type called `Car`, that looks something like this:

```yaml
apiVersion: sample-app.weave.works/v1alpha1
kind: Car
metadata:
creationTimestamp: "2020-08-17T14:33:16Z"
name: foo
namespace: default
spec:
brand: SAAB
engine: The best one
yearModel: "2008"
status:
acceleration: 0
distance: 12176941420362965433
persons: 0
speed: 53.37474583162469
```
All binaries let you access the data and fake a modification event using a sample webserver running on `localhost:8888`.

### sample-gitops

This is a sample binary that:

a) clones a Git repo of your choice to a temporary directory, and authenticates using given `id_rsa` and `known_hosts` files. Create a Git repo with e.g. the sample file above, and set up SSH credentials.
b) exposes all `Car`s in your Git repository at URL `GET localhost:8888/git/`
c) lets you fake a "reconciler spec/status write" event at path `PUT localhost:8888/git/<name>`, where `name` is the name of the `Car` in your repo you want to modify
d) re-syncs every 10 seconds, and tries to pull the git repo
e) has an inotify watch on the temporary Git clone, so it will log all objects that have been changed as they happen in Git (e.g. from new commits)

When you modify the "desired state/current status" using e.g. `curl -sSL -X PUT localhost:8888/git/foo`, the following will happen:

a) a `Transaction` will be started, which means `git pull` and `git checkout -b <name>-update-<random_sha>` will be executed
b) `Storage.Get` for the `Car` with the given name will be requested
c) the Car's `.status.distance` and `.status.speed` fields are updated to random numbers, and `Storage.Update` is run
d) the transaction is "committed" by returning a `transaction.PullRequestResult`
e) when the transaction ends, `git commit -A -m <message>`, `git push` and `git checkout <main>` will be executed. The `git pull` loop is resumed.
f) as a `transaction.PullRequestResult` was returned (and not `transaction.CommitResult`), the code will also use a `transaction.PullRequestProvider` to create a PR towards the repo. The configured provider is for now GitHub-only, and configured through passing the `GITHUB_TOKEN` environment variable.
g) the PR will be created for the given branch, with the given assignees, labels and milestone
h) once the PR is merged, the `git pull` loop will eventually download the new commit, and the inotify watch will tell which files were changed.

#### sample-gitops Usage

```console
$ make
...
$ bin/sample-gitops --help
Usage of bin/sample-gitops:
--author-email string Author email for Git commits (default "support@weave.works")
--author-name string Author name for Git commits (default "Weave libgitops")
--git-url string HTTPS Git URL; where the Git repository is, e.g. https://github.com/luxas/ignite-gitops
--identity-file string Path to where the SSH private key is
--pr-assignees strings What user logins to assign for the created PR. The user must have pull access to the repo.
--pr-milestone string What milestone to tag the PR with
--version Show version information and exit
```

You also need to set `GITHUB_TOKEN` in order to be able to create the PR.

### sample-watch

sample-watch demonstrates use of the inotify `GenericWatchStorage` on a customizable directory.

When running it, create a file (e.g. the example above) anywhere in the folder you're watching.
You'll see it being noticed in the log. Once that's done, you can curl it like this: `curl -sSL localhost:8888/watch/<name>`, where name equals `.metadata.name` of the object you just put in the dir.

You can also write a new status using `curl -sSL -X PUT localhost:8888/watch/foo`, so that the next time you get it as per above, you can see the status has changed.

#### sample-watch Usage

```console
$ make
...
$ bin/sample-watch --help
Usage of bin/sample-watch:
--version Show version information and exit
--watch-dir string Where to watch for YAML/JSON manifests (default "/tmp/libgitops/watch")
```

### sample-app

sample-app is using the `GenericStorage` and `GenericRawStorage` on the directory of your choice. The path where the objects are stored are of the form `<top-level-dir>/<kind>/<identifier>/metadata.json`.

Use it as follows:

a) create a new `Car` using `curl -sSL -X POST localhost:8888/plain/foo`
b) see the object in e.g. using `cat /tmp/libgitops/manifest/Car/default/foo/metadata.yaml`
c) get it through the webserver using `curl -sSL localhost:8888/plain/foo`
d) update status through `curl -sSL -X PUT localhost:8888/plain/foo`

#### sample-app Usage

```console
$ make
...
$ bin/sample-app --help
Usage of bin/sample-app:
--data-dir string Where to store the YAML files (default "/tmp/libgitops/manifest")
--version Show version information and exit
```

## Getting Help

If you have any questions about, feedback for or problems with `libgitops`:
Expand Down
63 changes: 6 additions & 57 deletions cmd/sample-app/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@ package main
import (
"bytes"
"fmt"
"github.com/weaveworks/libgitops/pkg/storage/watch"
"github.com/weaveworks/libgitops/pkg/storage/watch/update"
"net/http"
"os"

"github.com/labstack/echo"
"github.com/sirupsen/logrus"
"github.com/spf13/pflag"
"github.com/weaveworks/libgitops/cmd/common"
"github.com/weaveworks/libgitops/cmd/sample-app/apis/sample/scheme"
"github.com/weaveworks/libgitops/cmd/sample-app/apis/sample/v1alpha1"
Expand All @@ -19,10 +18,7 @@ import (
"github.com/weaveworks/libgitops/pkg/storage"
)

const (
ManifestDir = "/tmp/libgitops/manifest"
WatchDir = "/tmp/libgitops/watch"
)
var manifestDirFlag = pflag.String("data-dir", "/tmp/libgitops/manifest", "Where to store the YAML files")

func main() {
// Parse the version flag
Expand All @@ -36,39 +32,21 @@ func main() {
}

func run() error {
// Create the manifest and watch directories
if err := os.MkdirAll(ManifestDir, 0755); err != nil {
return err
}
if err := os.MkdirAll(WatchDir, 0755); err != nil {
// Create the manifest directory
if err := os.MkdirAll(*manifestDirFlag, 0755); err != nil {
return err
}

// Set the log level
logs.Logger.SetLevel(logrus.TraceLevel)
logs.Logger.SetLevel(logrus.InfoLevel)

plainStorage := storage.NewGenericStorage(
storage.NewGenericRawStorage(ManifestDir, v1alpha1.SchemeGroupVersion, serializer.ContentTypeYAML),
storage.NewGenericRawStorage(*manifestDirFlag, v1alpha1.SchemeGroupVersion, serializer.ContentTypeYAML),
scheme.Serializer,
[]runtime.IdentifierFactory{runtime.Metav1NameIdentifier},
)
defer func() { _ = plainStorage.Close() }()

watchStorage, err := watch.NewManifestStorage(WatchDir, scheme.Serializer)
if err != nil {
return err
}
defer func() { _ = watchStorage.Close() }()

updates := make(chan update.Update, 4096)
watchStorage.SetUpdateStream(updates)

go func() {
for upd := range updates {
logrus.Infof("Got %s update for: %v %v", upd.Event, upd.PartialObject.GetObjectKind().GroupVersionKind(), upd.PartialObject.GetObjectMeta())
}
}()

e := common.NewEcho()

e.GET("/plain/:name", func(c echo.Context) error {
Expand Down Expand Up @@ -112,34 +90,5 @@ func run() error {
return c.String(200, "OK!")
})

e.GET("/watch/:name", func(c echo.Context) error {
name := c.Param("name")
if len(name) == 0 {
return echo.NewHTTPError(http.StatusBadRequest, "Please set name")
}

obj, err := watchStorage.Get(common.CarKeyForName(name))
if err != nil {
return err
}
var content bytes.Buffer
if err := scheme.Serializer.Encoder().Encode(serializer.NewJSONFrameWriter(&content), obj); err != nil {
return err
}
return c.JSONBlob(http.StatusOK, content.Bytes())
})

e.PUT("/watch/:name", func(c echo.Context) error {
name := c.Param("name")
if len(name) == 0 {
return echo.NewHTTPError(http.StatusBadRequest, "Please set name")
}

if err := common.SetNewCarStatus(plainStorage, common.CarKeyForName(name)); err != nil {
return err
}
return c.String(200, "OK!")
})

return common.StartEcho(e)
}
4 changes: 2 additions & 2 deletions cmd/sample-gitops/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ package main
import (
"context"
"fmt"
"github.com/weaveworks/libgitops/pkg/storage/watch"
"github.com/weaveworks/libgitops/pkg/storage/watch/update"
"io/ioutil"
"net/http"
"os"
Expand All @@ -23,6 +21,8 @@ import (
"github.com/weaveworks/libgitops/pkg/storage"
"github.com/weaveworks/libgitops/pkg/storage/transaction"
githubpr "github.com/weaveworks/libgitops/pkg/storage/transaction/pullrequest/github"
"github.com/weaveworks/libgitops/pkg/storage/watch"
"github.com/weaveworks/libgitops/pkg/storage/watch/update"
)

var (
Expand Down
89 changes: 89 additions & 0 deletions cmd/sample-watch/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package main

import (
"bytes"
"fmt"
"net/http"
"os"

"github.com/labstack/echo"
"github.com/sirupsen/logrus"
"github.com/spf13/pflag"
"github.com/weaveworks/libgitops/cmd/common"
"github.com/weaveworks/libgitops/cmd/sample-app/apis/sample/scheme"
"github.com/weaveworks/libgitops/pkg/logs"
"github.com/weaveworks/libgitops/pkg/serializer"
"github.com/weaveworks/libgitops/pkg/storage/watch"
"github.com/weaveworks/libgitops/pkg/storage/watch/update"
)

var watchDirFlag = pflag.String("watch-dir", "/tmp/libgitops/watch", "Where to watch for YAML/JSON manifests")

func main() {
// Parse the version flag
common.ParseVersionFlag()

// Run the application
if err := run(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}

func run() error {
// Create the watch directory
if err := os.MkdirAll(*watchDirFlag, 0755); err != nil {
return err
}

// Set the log level
logs.Logger.SetLevel(logrus.InfoLevel)

watchStorage, err := watch.NewManifestStorage(*watchDirFlag, scheme.Serializer)
if err != nil {
return err
}
defer func() { _ = watchStorage.Close() }()

updates := make(chan update.Update, 4096)
watchStorage.SetUpdateStream(updates)

go func() {
for upd := range updates {
logrus.Infof("Got %s update for: %v %v", upd.Event, upd.PartialObject.GetObjectKind().GroupVersionKind(), upd.PartialObject.GetObjectMeta())
}
}()

e := common.NewEcho()

e.GET("/watch/:name", func(c echo.Context) error {
name := c.Param("name")
if len(name) == 0 {
return echo.NewHTTPError(http.StatusBadRequest, "Please set name")
}

obj, err := watchStorage.Get(common.CarKeyForName(name))
if err != nil {
return err
}
var content bytes.Buffer
if err := scheme.Serializer.Encoder().Encode(serializer.NewJSONFrameWriter(&content), obj); err != nil {
return err
}
return c.JSONBlob(http.StatusOK, content.Bytes())
})

e.PUT("/watch/:name", func(c echo.Context) error {
name := c.Param("name")
if len(name) == 0 {
return echo.NewHTTPError(http.StatusBadRequest, "Please set name")
}

if err := common.SetNewCarStatus(watchStorage, common.CarKeyForName(name)); err != nil {
return err
}
return c.String(200, "OK!")
})

return common.StartEcho(e)
}

0 comments on commit 69889c6

Please sign in to comment.