Turn consumer guides into executable tests

kptdev · Sep 18, 2020 · a7418e7 · a7418e7
1 parent 30817eb
commit a7418e7
Show file tree

Hide file tree

Showing 7 changed files with 165 additions and 4 deletions.
diff --git a/.github/workflows/verifyGuides.yml b/.github/workflows/verifyGuides.yml
@@ -31,4 +31,6 @@ jobs:
           GOPATH=$(go env GOPATH) go get github.com/monopole/mdrip
           GOPATH=$(go env GOPATH) go get sigs.k8s.io/kind@v0.8.1
       - run: |
+          git config --global user.email "you@example.com"
+          git config --global user.name "Your Name"
           make verify-guides
diff --git a/scripts/verifyGuides.sh b/scripts/verifyGuides.sh
@@ -4,7 +4,7 @@ set -o nounset
 set -o errexit
 set -o pipefail
 
-mdrip --blockTimeOut 6m0s --mode test \
+mdrip -alsologtostderr -v 10 --blockTimeOut 20m0s --mode test \
     --label verifyGuides site/content/en/guides
 
 echo "Success"
diff --git a/site/content/en/guides/consumer/apply/_index.md b/site/content/en/guides/consumer/apply/_index.md
@@ -7,6 +7,17 @@ description: >
     Apply the contents of a local package to a remote cluster.
 ---
 
+{{% hide %}}
+
+<!-- @makeWorkplace @verifyGuides-->
+```
+# Set up workspace for the test.
+TEST_HOME=$(mktemp -d)
+cd $TEST_HOME
+```
+
+{{% /hide %}}
+
 ## Topics
 
 [kpt live apply]
@@ -50,6 +61,7 @@ resources for deleted configuration.
 
 ### Command
 
+<!-- @fetchPackage @verifyGuides-->
 ```sh
 export SRC_REPO=https://github.com/GoogleContainerTools/kpt.git
 kpt pkg get $SRC_REPO/package-examples/helloworld-set@v0.5.0 helloworld
@@ -79,6 +91,7 @@ The inventory template must be created for a package to be applied using
 
 ### Init Command
 
+<!-- @liveInit @verifyGuides-->
 ```sh
 kpt live init helloworld
 ```
@@ -113,8 +126,18 @@ rest of the package, but otherwise ignored by users.
 
 ## Apply to a cluster
 
+{{% hide %}}
+
+<!-- @createKindCluster @verifyGuides-->
+```
+kind delete cluster && kind create cluster
+```
+
+{{% /hide %}}
+
 ### Apply Command
 
+<!-- @liveApply @verifyGuides-->
 ```sh
 kpt live apply helloworld --reconcile-timeout=2m
 ```
@@ -165,6 +188,16 @@ service/helloworld-gke   NodePort    10.48.2.143   <none>        80:32442/TCP
 service/kubernetes       ClusterIP   10.48.0.1     <none>        443/TCP        19m
 ```
 
+{{% hide %}}
+
+<!-- @ @verifyApply-->
+```
+# Verify that apply was successful
+kubectl get deployments | tr -s ' ' | grep "deployment.apps/helloworld-gke 5/5"
+```
+
+{{% /hide %}}
+
 ### Command: `tree`
 
 ```sh

diff --git a/site/content/en/guides/consumer/display/_index.md b/site/content/en/guides/consumer/display/_index.md
@@ -7,6 +7,17 @@ description: >
     Display the contents of a local package using kpt cfg for rendering.
 ---
 
+{{% hide %}}
+
+<!-- @makeWorkplace @verifyGuides-->
+```
+# Set up workspace for the test.
+TEST_HOME=$(mktemp -d)
+cd $TEST_HOME
+```
+
+{{% /hide %}}
+
 *Tools can parse and render configuration so it is easier for humans to read.*
 
 ## Topics
@@ -30,6 +41,7 @@ as a public package catalogue.
 
 ### Fetch Command
 
+<!-- @fetchPackage @verifyGuides-->
 ```sh
 kpt pkg get https://github.com/kubernetes/examples/staging/ examples
 ```
@@ -64,6 +76,7 @@ to work with using tools such as `less`.
 
 ### Count Example Command 1
 
+<!-- @countExamples @verifyGuides-->
 ```sh
 kpt cfg count examples/
 ```
@@ -85,6 +98,7 @@ Pod: 45
 
 ### Count Example Command 2
 
+<!-- @countCockroachdb @verifyGuides-->
 ```sh
 kpt cfg count examples/cockroachdb/
 ```
@@ -102,6 +116,7 @@ StatefulSet: 1
 
 ### Count Example Command 3
 
+<!-- @countAll @verifyGuides-->
 ```sh
 kpt cfg count examples/ --kind=false
 ```
@@ -118,6 +133,7 @@ The total aggregate resource count can be shown with `--kind=false`
 
 ### Display Command
 
+<!-- @treeCockroachdb @verifyGuides-->
 ```sh
 kpt cfg tree examples/cockroachdb/ --image --replicas
 ```
@@ -147,6 +163,7 @@ will print arbitrary fields by providing the `--field` flag.
 
 ### Filter Command
 
+<!-- @filterExamples @verifyGuides-->
 ```sh
 kpt cfg grep "spec.replicas>3" examples | kpt cfg tree --replicas
 ```
@@ -174,8 +191,11 @@ may be piped to tree for rendering.
 
 ### Dump Command
 
+<!-- @catCockroachdb @verifyGuides-->
 ```sh
 kpt cfg cat examples/cockroachdb
+# Temporary workaround for https://github.com/GoogleContainerTools/kpt/issues/1050
+echo "\n"
 ```
 
 The raw YAML configuration may be dumped using [`kpt cfg cat`][kpt cfg cat].

diff --git a/site/content/en/guides/consumer/set/_index.md b/site/content/en/guides/consumer/set/_index.md
@@ -7,6 +7,17 @@ description: >
     Customize a local package by setting field values.
 ---
 
+{{% hide %}}
+
+<!-- @makeWorkplace @verifyGuides-->
+```
+# Set up workspace for the test.
+TEST_HOME=$(mktemp -d)
+cd $TEST_HOME
+```
+
+{{% /hide %}}
+
 *Dynamic needs for packages are built into tools which read and write
 configuration data.*
 
@@ -67,6 +78,7 @@ in this guide.
 
 ### Command
 
+<!-- @fetchPackage @verifyGuides-->
 ```sh
 export SRC_REPO=https://github.com/GoogleContainerTools/kpt.git
 kpt pkg get $SRC_REPO/package-examples/helloworld-set@v0.6.0 helloworld
@@ -103,6 +115,18 @@ Print the list of setters included in the package.
 The package contains 3 setters which may be used to modify the configuration
 using `kpt set`.
 
+{{% hide %}}
+
+<!-- @verifyListSetters @verifyGuides-->
+```
+# Verify that we find the expected setters.
+kpt cfg list-setters helloworld/ | tr -s ' ' | grep "http-port 80 package-default helloworld port 3 No"
+kpt cfg list-setters helloworld/ | tr -s ' ' | grep "image-tag v0.1.0 package-default helloworld image tag 1 No"
+kpt cfg list-setters helloworld/ | tr -s ' ' | grep "replicas 5 package-default helloworld replicas 1 No"
+```
+
+{{% /hide %}}
+
 ## Set a field
 
 Setters **modify the resource configuration in place by reading the resources,
@@ -122,6 +146,7 @@ spec:
 
 ### Set Command
 
+<!-- @setReplicas @verifyGuides-->
 ```sh
 kpt cfg set helloworld/ replicas 3
 ```
@@ -146,6 +171,16 @@ spec:
 ...
 ```
 
+{{% hide %}}
+
+<!-- @verifySet @verifyGuides-->
+```
+# Verify that the setter updated the value
+grep "replicas: 3" helloworld/deploy.yaml
+```
+
+{{% /hide %}}
+
 [Kptfile]: ../../../api-reference/kptfile/
 [kpt cfg set]: ../../../reference/cfg/set/
 [setters]: ../../../reference/cfg/create-setter/

diff --git a/site/content/en/guides/consumer/substitute/_index.md b/site/content/en/guides/consumer/substitute/_index.md
@@ -7,6 +7,17 @@ description: >
     Customize a local package by substituting values into fields.
 ---
 
+{{% hide %}}
+
+<!-- @makeWorkplace @verifyGuides-->
+```
+# Set up workspace for the test.
+TEST_HOME=$(mktemp -d)
+cd $TEST_HOME
+```
+
+{{% /hide %}}
+
 *Dynamic needs for packages are built into tools which read and write
 configuration data.*
 
@@ -66,6 +77,7 @@ in this guide.
 
 ### Fetch Command
 
+<!-- @fetchPackage @verifyGuides-->
 ```sh
 export SRC_REPO=https://github.com/GoogleContainerTools/kpt.git
 kpt pkg get $SRC_REPO/package-examples/helloworld-set@v0.3.0 helloworld
@@ -106,15 +118,25 @@ kpt cfg list-setters helloworld/ --include-subst
 ### List Output
 
 ```sh
-    NAME      VALUE       SET BY             DESCRIPTION        COUNT  
+    NAME      VALUE       SET BY             DESCRIPTION        COUNT
   http-port   80      package-default   helloworld port         3
   image-tag   0.1.0   package-default   hello-world image tag   1
   replicas    5       package-default   helloworld replicas     1
   ------------   ------------------------------------------   ----------
-  SUBSTITUTION                    PATTERN                     REFERENCES
-  image          gcr.io/kpt-dev/helloworld-gke:${image-tag}   [image-tag]
+  SUBSTITUTION                      PATTERN                       REFERENCES
+  image-tag      gcr.io/kpt-dev/helloworld-gke:IMAGE_TAG_SETTER   [image-tag]
 ```
 
+{{% hide %}}
+
+<!-- @verifyListSubst @verifyGuides-->
+```
+# Verify that we get the expected output
+kpt cfg list-setters helloworld/ --include-subst | tr -s ' ' | grep "image-tag gcr.io/kpt-dev/helloworld-gke:IMAGE_TAG_SETTER \[image-tag\]"
+```
+
+{{% /hide %}}
+
 ## Substitute a value
 
 ### Package contents
@@ -134,6 +156,7 @@ metadata:
 
 ### Command
 
+<!-- @setImageTag @verifyGuides-->
 ```sh
  kpt cfg set helloworld/ image-tag v0.2.0
 ```
@@ -160,6 +183,16 @@ metadata:
 ...
 ```
 
+{{% hide %}}
+
+<!-- @verifySubst @verifyGuides-->
+```
+# Verify that the sustitution was updated
+grep "image: gcr.io/kpt-dev/helloworld-gke:v0.2.0" helloworld/deploy.yaml
+```
+
+{{% /hide %}}
+
 ## Customizing setters
 
 See [setters] and [substitutions] for how to add or update them in the

diff --git a/site/content/en/guides/consumer/update/_index.md b/site/content/en/guides/consumer/update/_index.md
@@ -7,6 +7,17 @@ description: >
     Update a customized local package with upstream (remote) package changes.
 ---
 
+{{% hide %}}
+
+<!-- @makeWorkplace @verifyGuides-->
+```
+# Set up workspace for the test.
+TEST_HOME=$(mktemp -d)
+cd $TEST_HOME
+```
+
+{{% /hide %}}
+
 *Packages can be arbitrarily customized and later merge updates from
 upstream.*
 
@@ -68,6 +79,7 @@ updated to later versions to merge in upstream changes.
 
 ### Fetch Command
 
+<!-- @fetchPackage @verifyGuides-->
 ```sh
 export REPO=https://github.com/GoogleContainerTools/kpt.git
 kpt pkg get $REPO/package-examples/helloworld-set@v0.3.0 helloworld
@@ -114,6 +126,19 @@ The old package contents without local modifications.
 vi helloworld/deploy.yaml
 ```
 
+{{% hide %}}
+
+<!-- @updateLocalPackage @verifyGuides-->
+```
+# Update the local package
+cat <<EOF >> helloworld/deploy.yaml
+        - name: NEW_ENV # This is a local package addition
+          value: "local package edits"
+EOF
+```
+
+{{% /hide %}}
+
 ### New local configuration
 
 ```yaml
@@ -141,6 +166,7 @@ kpt will throw an error if trying to update a package and the git repo
 has uncommitted changes.
 {{% /pageinfo %}}
 
+<!-- @commitLocalChanges @verifyGuides-->
 ```sh
 git init
 git add .
@@ -154,6 +180,7 @@ specified version and applying the upstream changes to the local package.
 
 ### Merge Command
 
+<!-- @mergeUpdates @verifyGuides-->
 ```sh
 kpt pkg update helloworld@v0.5.0 --strategy=resource-merge
 ```
@@ -243,6 +270,17 @@ The Kptfile was updated with the new upstream metadata.
 ...
 ```
 
+{{% hide %}}
+
+<!-- @verifyUpdate @verifyGuides-->
+```
+grep "helloworld-gke:v0.3.0" helloworld/deploy.yaml
+grep "name: NEW_ENV" helloworld/deploy.yaml
+grep "ref: v0.5.0" helloworld/Kptfile
+```
+
+{{% /hide %}}
+
 The updated local package contains *both* the upstream changes (new image tag),
 and local modifications (additional environment variable).