Handle upgrades with an istio-init chart #10562

sdake · 2018-12-18T23:10:53Z

Install istio-init via helm install or helm template
run tiller-hack to remove references to CRDs that are stored in istio release
Install Istio via helm install or helm template

step 1 may be replaced with a direct apply of the istio-init/files/crd files.

sdake · 2018-12-18T23:13:02Z

/hold

sdake · 2018-12-19T04:51:47Z

To reviewers: This PR is incomplete - but it represents the best I have been able to achieve given the numerous requirements people have, coupled with the desire to deliver a correct solution. Missing from this implementation is go code that reads the configmap and installs the crds - will work on that next.

Just wanted an early review if this is headed in the right direction. I am running out of ideas.

Note files in the files dir are not processed during a kubectl apply, but only via helm install or helm template - so they can be used unmodified in a kubectl apply operation.

ostromart · 2018-12-19T09:18:54Z

install/kubernetes/helm/istio-init/README.md

+
+Helm charts expose configuration options which are currently in alpha.  The currently exposed options are explained in the following table:
+
+| Parameter | Description | Values | Default |


What happened to the remaining of the ~200 options?

The istio-init chart has no options - it simply runs and installs the CRDs from the configmaps.

I'll update the readme.md to state this.

correction here - istio-init has 3 options: hub, tag, and image pull policy.

ostromart · 2018-12-19T09:20:03Z

install/kubernetes/helm/istio-init/README.md

@@ -0,0 +1,186 @@
+# Istio


Is the plan to move this to the docs website in 1.1 or is it going to be a parallel track?

this was just cut and paste. Unfortunately, folks want a replication of the README.md files in the charts as to what is on the website. Helm installation tools render the README.md in a standardized way from my understanding. @linsun would know more - she asked me IIRC to revert the removal of this file from the main istio chart.

catching up on this, sorry for the delay. Yes this file is useful when users install helm from a repo directly or from a UI, where this tells user more info about the chart they are going to install.

sbezverk

Curious what is the reason to keep crds in a separate files? Would it not be a more clean to keep all crds in the same configmap?

sdake · 2018-12-19T14:46:36Z

@sbezverk we want to keep the contents of the crd yamls in separate files for management reasons. However, the configmap itself could probably be unified into one configmap. I will try that line of thinking once I get a go implementation of the conversion running.

Cheers
-steve

sdake · 2018-12-19T14:48:10Z

install/kubernetes/helm/istio-init/Chart.yaml

+version: 1.1.0
+appVersion: 1.1.0
+tillerVersion: ">=2.7.2-0"
+description: Helm chart for all istio components


this should be modified

sdake · 2018-12-19T14:48:20Z

install/kubernetes/helm/istio-init/Chart.yaml

@@ -0,0 +1,17 @@
+apiVersion: v1
+name: istio


sdake · 2018-12-19T14:48:36Z

install/kubernetes/helm/istio-init/Chart.yaml

+tillerVersion: ">=2.7.2-0"
+description: Helm chart for all istio components
+keywords:
+  - istio


and the keyword list

sdake · 2018-12-19T22:48:20Z

@ostromart thanks - although not quite done yet- needs the configmap to crd converter code and integration with the CI system. Will need followup reviews once I get it done.

linsun · 2019-01-03T19:34:18Z

install/kubernetes/helm/istio-init/README.md

+
+## Introduction
+
+This chart bootstraps [CRDs](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)


boostraps Istio CRDs, to be specific.

linsun · 2019-01-03T19:36:26Z

install/kubernetes/helm/istio-init/README.md

+which are an internal implementation detail of Istio.  CRDs contain all runtime configuration specified by the human
+operator.
+
+This chart must be run to completion prior to running other Istio charts, or other Istio charts will fail to initialize.


can we build this dependency in other istio charts? https://github.com/helm/helm/blob/master/docs/helm/helm_dependency.md

unfortunately Helm has no ordering. You could build the dependencies, but they wouldn't be honored in any type of order. As an example, the existing charts deploy the addon's vs/gw/dr for each service. If these deploy prior to the CRD being registered from the unordered istio-init chart, the registration would fail. One could argue we shouldn't be registering CustomResources at all in the main charts, but instead have some operator-type thing doing that job...

linsun · 2019-01-03T19:51:37Z

install/kubernetes/helm/istio-init/README.md

+
+1. Install the Istio initializer chart:
+    ```
+    $ helm install install/kubernetes/helm/istio-init --name istio-init


can you confirm namespace isn't needed because it only installs CRDs which are cluster wide?

i am pretty sure, but will verify.

namespace is needed as the jobs will run in a specific namespace. Since the jobs register global objects, I think we can come up with a new namespace rather than re-using istio-system. re-using istio-system may cause problems with chart creation/removal. @costinm had some good ideas for naming here for global objects such as CRDs, but I don't recall what he suggested. I'll see if I can find that and implement that in this PR.

linsun · 2019-01-03T19:53:29Z

install/kubernetes/helm/istio-init/README.md

+To uninstall/delete the `istio-init` release completely and make its name free for later use:
+    ```
+    $ helm delete istio-init --purge
+    ```


should we append instructions on how to remove CRDs at the end?

since you said delete the istio-init chart won't remove the CRDs.

linsun · 2019-01-03T19:58:20Z

tests/istio.mk

+	${HELM} install \
+	  install/kubernetes/helm/istio-init \
+	  --name istio-system-init --namespace istio-system \
+	  --set global.hub=${HUB} \


is hub or tag var relevant? I thought you said earlier no config nob for this chart.

yes, there will be config knobs, as a kubectl container needs to be run to convert configmaps to CRDs. I will modify the docs in a followon.

golangcibot · 2019-01-07T15:43:32Z

tests/e2e/framework/kubernetes.go

 	yamlFileName = filepath.Join(k.ReleaseDir, yamlFileName)

 	// deploy CRDs first
 	if err := util.KubeApply("kube-system", yamlFileName, k.KubeConfig); err != nil {
-		log.Errorf("Failed to apply %s", yamlFileName)
-		return err
+		return fmt.Errorf("Failed to apply %s because %v", yamlFileName, err)


error strings should not be capitalized or end with punctuation or a newline (from golint)

golangcibot · 2019-01-07T15:43:33Z

tests/e2e/framework/kubernetes.go

+	// deploy all CRDs in Istio first
+	for _, yamlFileName := range istioCRDFileNames {
+		if err := k.deployCRDs(yamlFileName); err != nil {
+			return fmt.Errorf("Failed to apply all Istio CRDS: %v", err)


error strings should not be capitalized or end with punctuation or a newline (from golint)

golangcibot · 2019-01-07T15:43:33Z

tests/e2e/framework/kubernetes.go

+	// CRD files that should be installed during testing
+	// NB: these files come from the directory install/kubernetes/helm/istio-init/files/*crd*
+	//     and contain all CRDs used by Istio during runtime
+	CRDFileZero                    = "crd-10.yaml"


File is not goimports-ed (from goimports)

golangcibot · 2019-01-07T15:54:16Z

tests/e2e/framework/kubernetes.go

+	// CRD files that should be installed during testing
+	// NB: these files come from the directory install/kubernetes/helm/istio-init/files/*crd*
+	//     and contain all CRDs used by Istio during runtime
+	CRDZeroInstallFile             = "crd-10.yaml"


File is not goimports-ed (from goimports)

sbezverk · 2019-01-07T16:10:51Z

tests/e2e/framework/kubernetes.go

+	// deploy all CRDs in Istio first
+	for _, yamlFileName := range istioCRDFileNames {
+		if err := k.deployCRDs(yamlFileName); err != nil {
+			return fmt.Errorf("failed to apply all Istio CRDs: %v", err)


Please add file name to the message, something like this:
return fmt.Errorf("failed to apply all Istio CRDs from file: %s with error: %v", yamlFileName, err)
In this case it will be easier to troubleshoot as we know the exact file name which cause the failure.

thanks will do.

sbezverk · 2019-01-07T16:12:38Z

@sdake one small nit, otherwise lgtm

sdake · 2019-01-07T16:20:24Z

@sbezverk if satisfied, can you either dismiss your review or approve.

Cheers
-steve

golangcibot · 2019-01-07T16:41:08Z

tests/e2e/framework/kubernetes.go

+	// CRD files that should be installed during testing
+	// NB: these files come from the directory install/kubernetes/helm/istio-init/files/*crd*
+	//     and contain all CRDs used by Istio during runtime
+	zeroCRDInstallFile             = "crd-10.yaml"


File is not goimports-ed (from goimports)

Have you run gofmt -s on this file? If the lint keeps complaining, please add //nolint: goimports to stop this particular error. There is a known upstream (golang SDK) bug that causes goimports to bark rarely for no reason.

@kyessenov yes I did run it as the linter was complaining. It appears the linters see `//`` as the beginning of a new "section" and renew alignment. Thanks for looking :)

linsun · 2019-01-08T19:57:55Z

/lgtm

istio-testing · 2019-01-08T19:58:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: linsun, sdake

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [linsun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

istio/istio#10562 moved CRD.yaml files from ./istio/templates to ./istio-init/files. This change works with both the old and new paths for backwards compatibility.

After the change istio#10562 Istio Deployment in new test framework started failing. This PR tries to fix that

istio/istio#10562 moved CRD.yaml files from ./istio/templates to ./istio-init/files. This change works with both the old and new paths for backwards compatibility.

* Fix New Test Framework tests running in kubernetes environment After the change #10562 Istio Deployment in new test framework started failing. This PR tries to fix that * Minor fix

* fix the test (#10837) Signed-off-by: Kuat Yessenov <kuat@google.com> * Allow prometheus scraper to fetch port outside of sidecar umbrella (#10492) See issue #10487 - kubernetes-pods job is now keeping all targets without sidecar or with expicit prometheus.io/scheme=http annotation - kubernetes-pods-istio-secure is now discarding targets with expicit prometheus.io/scheme=http annotation * Relax test for kubeenv metric to only error on 'unknowns' (#10787) * Relax test for kubeenv metric to only error on 'unknowns' * Add check to ensure that at least one metric is found * Address lint issues * Fix Citadel Kube JWT authentication result (#10836) * Fix Citadel Kube JWT authentication. * Small fix. * Fix unittest. * Add unit test for coverage. * Adding Sidecar CRD and renaming Sidecar role (#10852) * Sidecar config implementation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * adding CRD template Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * model.Sidecar to model.SidecarProxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits * gen files in galley Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nit Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * e2e tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix yaml path * typo * bad file name * future work Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix bad namespace * assorted fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fixing CDS Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * formatting Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * vendor update Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * validation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * new crd yaml Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nix listener port Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * kubernetes hack for parsing namespace Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * some code cleanups and more TODOs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * spell check Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * leftovers Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undo tests * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * del Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * sidecarproxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * run log Configure before running server and validation (#10643) * run log Configure before running server and validation * remove p.logConfigure func from patchTable * fix lint * fix rebase error * fix rebase error * fix lint * add domain parameter to proxy of istio-policy. (#10857) * Use strings.EqualFold to compare strings (#10859) * Call check licenses only once (#10866) * add sample httpbin service in nodeport type (#10833) * Skip prow e2e test cleanup (#10878) * Use 128bit traceids in envoy (#10811) * Use 128bit traceids in envoy * Update unit test golden files for bootstrap config * Update to latest istio/api changes with MCP enhancements (#10628) * sync with latest istio.io/api This PR syncs to the latest changes from istio.io/api. Notably, this PR includes the enhanced MCP service definitions and protos (ResourceSink and ResourceSource) along with several API cleanups. Minimal changes have been made to fix the build and tests so that subsequent istio.io/api changes can be merged into istio/istio. An additional PR will be introduced to implement the enhanced MCP service layer. * address review comments * remove bad find/replace * Add a newline at the end of each certificate returned by Vault (#10879) * Add a newline at the end of a certificate * Fix the mock test * Fix a lint error * Filter flaky query from galley dashboard test (#10176) * IPv4 forwarding off for some CircleCI builds (#10777) * Log additional information about build machine * Attempt to enable IPv4 forwarding * tabs to spaces * stop mcpclient when mixer stops (#10772) * stop mcpclient when mixer stops * fix test * pushLds should not verify versions (#10861) * add integration test that mTLS through identity provisioned by SDS flow (#10887) * add integration test that mTLS through identity provisioned by SDS flow * format * remove unused files (#10890) * fix pilot goroutine leak (#10892) * fix pilot goroutine leak * remove done channel * Add missing copyright header (#10841) * Do not fail envoy health probe if a config was rejected (#9786) (#10154) * Do not fail envoy health probe if a config was rejected (#9786) * Adjust so that rejection is also an allowed state of health probe for envoy. Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add unit tests for envoy health probe Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fixed linting Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fix another linting problem Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add new stats to String() method * Use better wording in log message * Fix linting Co-authored-by: Ulrich Kramer <u.kramer@sap.com> * Move everything related to spiffe URIs to package spiffe (#9090) * Move everything related to spiffe URIs to package spiffe Co-authored-by: Ulrich Kramer <u.kramer@sap.com> * Fix end-to-end tests after merge Co-authored-by: Julia Plachetka <julia.plachetka@sap.com> * Adapt and fix unit tests. Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Adapt and fix unit tests. * Fix lint errors and unit tests * Fix lint errors * Fix lint errors * Fix lint errors. Exit integration test in case of nonexisting secret * Remove duplicate trustDomain * Fixed compile errors * Fixed lint errors * Fixed lint errors * Do not panic and small fixes * Do not panic when spiffe uri is missing some configuration values * Remove environment variable ISTIO_SA_DOMAIN_CANONICAL * Fix SNA typo * Comment why testing for a kube registry Co-authored-by: Holger Oehm <holger.oehm@sap.com> * goimports-ed Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Adapt test to getSpiffeId no longer panicing Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix formatting Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix lint errors and unit tests Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix double declared imports Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fix more import related linting Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add retry to metrics check in TestTcpMetrics (#10816) * Add retry to metrics check in TestTcpMetrics * Small cleanup * Fix typo * set trust domain (#10905) * Fix New Test Framework tests running in kubernetes environment (#10889) * Fix New Test Framework tests running in kubernetes environment After the change #10562 Istio Deployment in new test framework started failing. This PR tries to fix that * Minor fix * Add Pod and Node sources to Galley. (#10846) * Add Pod and Node sources to Galley. Also plumbing annotations and labels through from the source. * adding access for pods/nodes to deployment. * plumbing labels/annotations through Pilot * implement empty header value expression (#10885) Signed-off-by: Kuat Yessenov <kuat@google.com> * provide some context on bootstrap errors (#10696) - rebased on release-1.1 * fix(#10911): add namespace for crd installation jobs (#10912) * restore MCP registry (#10921) * fix a typo to get familiar with the PR process (#10853) Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn> * Mixer route cache (#10539) * rebase * add test * fix lint * Revert "Mixer route cache (#10539)" (#10936) This reverts commit 024adb0. * Clean up the Helm readiness checking in test cases (#10929) * Clean up the Helm readiness checking in test cases The e2e test cases are often flakey because of the logic of Helm readiness checking in the test cases. Instead of checking of the Pod is in the "RUNNING" state, check that Tiller is able to provide service via the `helm version` operation. If the server is not ready, this will return 1, otherwise 0 will be returned. * Fix CLI call error We have an older version of helm which lacks the proper flag. Instead we rely on the retry with a 10 second context timer. * Test for PERMISSIVE mode, checks Pilot LDS output. (#10614) * injector changes for health check, pilot agent take over app readiness check. (#9266) * WIP injector change to modify istio-proxy. * move out to app_probe.go * Iterating sidecartmpl to find the statusPort. * use the same name for ready path. * Get rewrite work, almost. * Some clean up on test and check one container criteria. * fix the injected test file. * Add inject test for readiness probe itself. * Add missing added test file. * fix helm test. * fix lint. * update header based finding the port. * return to previous injected file status. * fixing TestIntoResource test. * sed fixing all remaining injecting files. * handling named port. * fixing merginge failure. * remove the debug print. * lint fixing. * Apply the suggestions for finding statusPort arg. * Address comments, regex support more port value format. * add app_probe_test.go * add more test. * merge fix the test. * WIP adding test not working. * change k8s env applycontents. * pilot_test.go working adding the policy. * adding authn in the setup. * progress, app is in istio-system. * simplify the pilot_test.go * get config dump for app a. * config is dumped and testhttp pass. * WIP need to figure out why config dump is different than lds output. * finally hacked to get lds output. * almost ready to verify the listener config * get test working, remove some debugging print. * move to permissive_test.go * clean up on test file. * add back auth_permissive_test.go * add some doc and remove infolog. * refine comments. * goimports fix. * bin/fmt.sh * apply comments. * add one more test case. * rename the ConstructDiscoveryRequest. * comment out unimplemented test. * change back logging level. * Sidecar config implementation (#10717) * Sidecar config implementation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * adding CRD template Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * model.Sidecar to model.SidecarProxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits * gen files in galley Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nit Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * e2e tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix yaml path * typo * bad file name * future work Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix bad namespace * assorted fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fixing CDS Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * formatting Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * vendor update Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * validation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * new crd yaml Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nix listener port Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * kubernetes hack for parsing namespace Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * some code cleanups and more TODOs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * spell check Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * leftovers Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undo lint fix * temp undo * ingress and egress listeners on ports Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * if-else Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * missing inbound port fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove constants Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lints Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix http host header Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more if-elses Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more lint and code cov Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * simplifications * remove GetSidecarScope Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * coverage Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * missing configs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * 80 Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove invalid test case * fixing rds bug Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove comment Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * RDS unit tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint again Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * Filter Nodes/Pods in Galley temporarily until custom sources land. (#10938) This is due to the fact that Pod yaml cannot currently be parsed into unstructured types. See: #10891. * fix concurrent map read/write (#10895) * fix concurrent map read/write * simplify EndpointShardsByService * Update integration test job (#10888) * Fix integration test scripts * Making TestMain exit with the proper return code * Update local env references to native * Fix linter errors * Skipping integration tests in codecov since they fail * grant execute permission to e2e_pilotv2_auth_sds.sh (#10908) * grant execute permission to e2e_pilotv2_auth_sds.sh * fix typo * fix typo * typo * coredump * remove deprecated plugin from nodeagent (#10952) * Fix flaky test by reducing poll interval. (#10962) * Add interceptor to create noop spans when sampling is false (#10826) * Add interceptor to create noop spans when sampling is false * Add tests using mocktracer to determine whether span is created * Update dependencies to include OpenTracing mocktracer * Minor change * Updated dependencies again * Add support for ErrSpanContextNotFound error * Fix test and add one for x-b3-sampled=true * Fix lint error * set cluster.LoadAssignment only when service discovery type equals Cluster_STATIC Cluster_STRICT_DNS or Cluster_LOGICAL_DNS (#10926) * Remove Envoy's deprecated --v2-config-only (release-1.1). (#10960) Signed-off-by: Piotr Sikora <piotrsikora@google.com> * update check proxy version (#10769) * Add AWS CloudwatchLogs Adapter (code from #10400) (#10882) * Add AWS CloudwatchLogs Adapter (code from #10400) * Improve codecov * Even moar coverage * remove duplicate LoadAssignment set (#10977) * Enable server side control over maximum connection age (#10870) * add server side maximum connection age control to keepalive options * add server maximum connection age to the gRPC server keepalive options * missing space between concatenated strings * added tests for default values and setting via command line * fix golangci unconvert comment * add helm value file to google ca param (#10563) * add helm value file to preconfig param for googleca * cleanup * Allow pulling images from private repository (#10763) * Only compute diff for ServiceEntry (#10446) * Only compute diff for ServiceEntry This change prevents coredatamodel controller to compute the diff for all the types and it narrows it down to only ServiceEntry. * Add a dummy event for other config types - this dummy event allows DiscoveryServer to purge it's cache * Trigger a single clear cache event * add exponential backoff for retryable CSR error in nodeagent (#10969) * backoff * add unit test * clean up * lint * lint * address comment * typo * Fix flakiness in redisquota tests (#10906) * Fix flakiness in redisquota tests by adding retry for getting requests reported by prometheus One of the things I observed in flaky tests is that total number of requests reported by prometheus was not equal to traffic sent by Fortio. Thus adding a retry to make sure prometheus is queries till we get all requests reported. * Add a buffer for 5 requests to be allowed to be not reported. This buffer is within the error we allow for 200s and 429s reporting. * Fix based on reviews * Fix lint errors * Adding make sync to integ test script (#10984) * Removing Galley pod and node datasets from tests (#10953) * Use common image for node agent (#10949) * Use comment image for node agent * Revert node-agent-k8s * Sort the package * fix MCP server goroutine leak (#10893) * fix MCP server goroutine leak * fix race condition * fix race condition between reqChannel blocking and stream context done (#10998) * add default namespce for istio-init namespace. (#11012) * Handle outbound traffic policy (#10869) * add passthru listener only for mesh config outbound traffic policy ALLOW_ANY * add outbound traffic policy to configmap template and values * add the listener and blackhole cluster in case of outbound policy REGISTRY_ONLY * update DefaultMeshConfig with OutboundTrafficPolicy * use ALLOW_ANY outbound policy by default in tests * add OutboundTrafficPolicy to the default meshconfig of galley * Revert "use ALLOW_ANY outbound policy by default in tests" This reverts commit 9045789. * use REGISTRY_ONLY OutboundTrafficPolicy for galley tests * adopt notion of collections throughout galley/mcp (#10963) * adopt notion of collections throughout galley/mcp * add missing 's/TypeURLs()/Collections()' * fix linter errors and missing dep * linter fixes * another linter fix * address review comments * use correct collection name in copilot test * fix TestConversion/config.istio.io_v1alpha2_circonus * update copilot e2e tests * fix pilot/pkg/config/coredatamodel/controller_test.go unit test * re-add TypeURL and remove typeurl from collections * add Bearer prefix in oauth token that passed to GoogleCA (#11018) * Add bionic and deb_slim base images, optimize size for xenial (#10992) * Remove redundant pieces of code (#11014) * Increase timeout (#11019) * mixer: gateway regression (#10966) * gateway test Signed-off-by: Kuat Yessenov <kuat@google.com> * prepare a test Signed-off-by: Kuat Yessenov <kuat@google.com> * Merge the new tests for isolation=none, some fixes (#10958) * Merge the new tests for isolation=none, some fixes * Add a local directory with certs, can be used with the basedir for local tests * If a BaseDir meta is specified, use it as prefix for the certs - so tests don't need / access * Add the pilot constant and doc * Fix mangled sidecarByNamespace, scope issue * Fix binding inbound listeners to 0.0.0.0, test * Format * Lint * Add back the validation * Reduce flakiness, golden diff reported as warning * Manual format, make fmt doesn't seem to help * Fix authn test * Fix authn test * Reduce parallel to avoid flakiness, fix copilot test * format * remove 'crds' option in relevant manifests (#11013) * remove crds option in istio chart. * delete crds option in values*.yaml * add istio-init as prerequisite of istio chart. * Delete this superfluous script. (#11028) * Refactor in preparation for reverse and incremental MCP (#11005) This PR refactors the MCP client, server, and monitoring packages in preparation for introducing reverse MCP. This includes the following changes: * Structs/Interfaces common to MCP sinks are moved into the sink package. * Structs/Interfaces common to MCP sources are moved into source packages. * The client and server metrics reporting logic is merged into a single reporter interface and implementation, since the majority of code is duplicated. This makes it easier to use a single reporter interface across all source/sink and client/server combinations. * Plumb through source/sink options * Port Mixer's TestTcpMetricTest in new Test framework (#10844) * Port Mixer's TestTcpMetricTest in new Test framework * Look at values file too to determine if mtls is enabled for the test or not. * Add unix domain socket client and server to pilot test apps (#10874) * Add unix domain socket client and server to pilot test apps Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * appends Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * template fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more gotpl Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos * undo Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * Fixing new framework integration test (#11038) Fixes are as follows: 1) PolicyBackend close is failing when closing the listener in natice environment. Thus ignoring it's error and making policy backend a system component, so that it is just reset between the tests and not really closed. 2) Skipping conversion test in local environment as it requires kubernetes environment. 3) Increasing timeout of tests in kubernetes environment 4) Adding test namespace in mixer check test. * Use proxyLabels that were collected earlier (#11016) * Fix comment on defaultNodeSelector comment (#10980) * tracing: Provide default configuration when no host specified for k8s ingress (#10914) * tracing: Provide default configuration when no host specified for k8s ingress * Remove jaeger ingress in favour of one ingress with context based on provider * Updated to remove $ from .Values * Add ymesika to pilot owners (#11053) * Restart Galley in native test fw. component to avoid race. (#11048) There is a race between Galley reading the updated mesh config file and processing of input config files. This change restarts Galley every time mesh config is updated, to avoid race. * Update Istio API to include selector changes in AuthN/AuthZ. (#11046) The following changes are included from istio.io/api: aec9db9 Add option to select worload using lables for authn policy. (#755) 2dadb9e add optional incremental flag to ResponseSink and ResourceSource services (#762) d341fc8 assorted doc updates (#757) 48ad354 Update RBAC for Authorization v2 API. (#748) f818794 add optional header operations (#753) Signed-off-by: Yangmin Zhu <ymzhu@google.com> * update proxy SHA (#11036) * update proxy SHA * Update Proxy SHA to d2d0c62a045d12924180082e8e4b6fbe0a20de1d * Add an example helm values yaml for Vault integration user guide (#11024) * Add an example helm values yaml for Vault integration user guide * Add a comment * Add retry logic to the SDS grpc server of Node Agent (#11063) * Quick fix for #10779 (#11061) * Basic fix to Ingress conversion. * Makes changes based on Ingress changes. * Linter fix. * Remove labels as well. * session affinity (#10730) * handle special char in trustdomain (to construct sa for secure naming) (#11066) * replace special char * update comment * enabled customized cluster domain for chart. (#11050) * enabled customized cluster domain for chart. * update webhook unit test data. * Restructure Galley sources (#11062) * Restructure Galley sources This is a series of simple moves in preparation for #10995 * addressing comments * assign back to s.mesh when reload the mesh config file (#11000) Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn> * Moving Galley source to dynamic package. (#11081) This is in preparation for #10995. Trying to do this move in order to preserve history. * Add reasonable default retry policy. (#10566) Partially addresses #7665. * Reduce flakiness in metrics test in new test framework (#11070) * Reduce flakiness in metrics test in new test framework * Fix based on review * Fix merge

* fix the test (#10837) Signed-off-by: Kuat Yessenov <kuat@google.com> * Allow prometheus scraper to fetch port outside of sidecar umbrella (#10492) See issue #10487 - kubernetes-pods job is now keeping all targets without sidecar or with expicit prometheus.io/scheme=http annotation - kubernetes-pods-istio-secure is now discarding targets with expicit prometheus.io/scheme=http annotation * Relax test for kubeenv metric to only error on 'unknowns' (#10787) * Relax test for kubeenv metric to only error on 'unknowns' * Add check to ensure that at least one metric is found * Address lint issues * Fix Citadel Kube JWT authentication result (#10836) * Fix Citadel Kube JWT authentication. * Small fix. * Fix unittest. * Add unit test for coverage. * Adding Sidecar CRD and renaming Sidecar role (#10852) * Sidecar config implementation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * adding CRD template Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * model.Sidecar to model.SidecarProxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits * gen files in galley Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nit Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * e2e tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix yaml path * typo * bad file name * future work Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix bad namespace * assorted fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fixing CDS Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * formatting Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * vendor update Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * validation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * new crd yaml Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nix listener port Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * kubernetes hack for parsing namespace Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * some code cleanups and more TODOs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * spell check Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * leftovers Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undo tests * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * del Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * sidecarproxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * run log Configure before running server and validation (#10643) * run log Configure before running server and validation * remove p.logConfigure func from patchTable * fix lint * fix rebase error * fix rebase error * fix lint * add domain parameter to proxy of istio-policy. (#10857) * Use strings.EqualFold to compare strings (#10859) * Call check licenses only once (#10866) * add sample httpbin service in nodeport type (#10833) * Skip prow e2e test cleanup (#10878) * Use 128bit traceids in envoy (#10811) * Use 128bit traceids in envoy * Update unit test golden files for bootstrap config * Update to latest istio/api changes with MCP enhancements (#10628) * sync with latest istio.io/api This PR syncs to the latest changes from istio.io/api. Notably, this PR includes the enhanced MCP service definitions and protos (ResourceSink and ResourceSource) along with several API cleanups. Minimal changes have been made to fix the build and tests so that subsequent istio.io/api changes can be merged into istio/istio. An additional PR will be introduced to implement the enhanced MCP service layer. * address review comments * remove bad find/replace * Add a newline at the end of each certificate returned by Vault (#10879) * Add a newline at the end of a certificate * Fix the mock test * Fix a lint error * Filter flaky query from galley dashboard test (#10176) * IPv4 forwarding off for some CircleCI builds (#10777) * Log additional information about build machine * Attempt to enable IPv4 forwarding * tabs to spaces * stop mcpclient when mixer stops (#10772) * stop mcpclient when mixer stops * fix test * pushLds should not verify versions (#10861) * add integration test that mTLS through identity provisioned by SDS flow (#10887) * add integration test that mTLS through identity provisioned by SDS flow * format * remove unused files (#10890) * fix pilot goroutine leak (#10892) * fix pilot goroutine leak * remove done channel * Add missing copyright header (#10841) * Do not fail envoy health probe if a config was rejected (#9786) (#10154) * Do not fail envoy health probe if a config was rejected (#9786) * Adjust so that rejection is also an allowed state of health probe for envoy. Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add unit tests for envoy health probe Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fixed linting Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fix another linting problem Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add new stats to String() method * Use better wording in log message * Fix linting Co-authored-by: Ulrich Kramer <u.kramer@sap.com> * Move everything related to spiffe URIs to package spiffe (#9090) * Move everything related to spiffe URIs to package spiffe Co-authored-by: Ulrich Kramer <u.kramer@sap.com> * Fix end-to-end tests after merge Co-authored-by: Julia Plachetka <julia.plachetka@sap.com> * Adapt and fix unit tests. Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Adapt and fix unit tests. * Fix lint errors and unit tests * Fix lint errors * Fix lint errors * Fix lint errors. Exit integration test in case of nonexisting secret * Remove duplicate trustDomain * Fixed compile errors * Fixed lint errors * Fixed lint errors * Do not panic and small fixes * Do not panic when spiffe uri is missing some configuration values * Remove environment variable ISTIO_SA_DOMAIN_CANONICAL * Fix SNA typo * Comment why testing for a kube registry Co-authored-by: Holger Oehm <holger.oehm@sap.com> * goimports-ed Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Adapt test to getSpiffeId no longer panicing Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix formatting Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix lint errors and unit tests Co-authored-by: Holger Oehm <holger.oehm@sap.com> * Fix double declared imports Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Fix more import related linting Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com> * Add retry to metrics check in TestTcpMetrics (#10816) * Add retry to metrics check in TestTcpMetrics * Small cleanup * Fix typo * set trust domain (#10905) * Fix New Test Framework tests running in kubernetes environment (#10889) * Fix New Test Framework tests running in kubernetes environment After the change istio/istio#10562 Istio Deployment in new test framework started failing. This PR tries to fix that * Minor fix * Add Pod and Node sources to Galley. (#10846) * Add Pod and Node sources to Galley. Also plumbing annotations and labels through from the source. * adding access for pods/nodes to deployment. * plumbing labels/annotations through Pilot * implement empty header value expression (#10885) Signed-off-by: Kuat Yessenov <kuat@google.com> * provide some context on bootstrap errors (#10696) - rebased on release-1.1 * fix(#10911): add namespace for crd installation jobs (#10912) * restore MCP registry (#10921) * fix a typo to get familiar with the PR process (#10853) Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn> * Mixer route cache (#10539) * rebase * add test * fix lint * Revert "Mixer route cache (#10539)" (#10936) This reverts commit 024adb0e5edfd902939211d321e5459758046905. * Clean up the Helm readiness checking in test cases (#10929) * Clean up the Helm readiness checking in test cases The e2e test cases are often flakey because of the logic of Helm readiness checking in the test cases. Instead of checking of the Pod is in the "RUNNING" state, check that Tiller is able to provide service via the `helm version` operation. If the server is not ready, this will return 1, otherwise 0 will be returned. * Fix CLI call error We have an older version of helm which lacks the proper flag. Instead we rely on the retry with a 10 second context timer. * Test for PERMISSIVE mode, checks Pilot LDS output. (#10614) * injector changes for health check, pilot agent take over app readiness check. (#9266) * WIP injector change to modify istio-proxy. * move out to app_probe.go * Iterating sidecartmpl to find the statusPort. * use the same name for ready path. * Get rewrite work, almost. * Some clean up on test and check one container criteria. * fix the injected test file. * Add inject test for readiness probe itself. * Add missing added test file. * fix helm test. * fix lint. * update header based finding the port. * return to previous injected file status. * fixing TestIntoResource test. * sed fixing all remaining injecting files. * handling named port. * fixing merginge failure. * remove the debug print. * lint fixing. * Apply the suggestions for finding statusPort arg. * Address comments, regex support more port value format. * add app_probe_test.go * add more test. * merge fix the test. * WIP adding test not working. * change k8s env applycontents. * pilot_test.go working adding the policy. * adding authn in the setup. * progress, app is in istio-system. * simplify the pilot_test.go * get config dump for app a. * config is dumped and testhttp pass. * WIP need to figure out why config dump is different than lds output. * finally hacked to get lds output. * almost ready to verify the listener config * get test working, remove some debugging print. * move to permissive_test.go * clean up on test file. * add back auth_permissive_test.go * add some doc and remove infolog. * refine comments. * goimports fix. * bin/fmt.sh * apply comments. * add one more test case. * rename the ConstructDiscoveryRequest. * comment out unimplemented test. * change back logging level. * Sidecar config implementation (#10717) * Sidecar config implementation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * adding CRD template Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * model.Sidecar to model.SidecarProxy Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits * gen files in galley Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nit Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * e2e tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix yaml path * typo * bad file name * future work Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix bad namespace * assorted fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fixing CDS Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * formatting Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * vendor update Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * build fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * test fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * validation Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * comments Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * new crd yaml Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nix listener port Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * kubernetes hack for parsing namespace Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * some code cleanups and more TODOs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more undos Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * spell check Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * leftovers Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undo lint fix * temp undo * ingress and egress listeners on ports Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * nits Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * if-else Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * missing inbound port fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove constants Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * final fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lints Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * fix http host header Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more if-elses Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more lint Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more lint and code cov Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * simplifications * remove GetSidecarScope Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * coverage Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * missing configs Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * 80 Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove invalid test case * fixing rds bug Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * remove comment Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * RDS unit tests Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * format Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * lint again Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * Filter Nodes/Pods in Galley temporarily until custom sources land. (#10938) This is due to the fact that Pod yaml cannot currently be parsed into unstructured types. See: #10891. * fix concurrent map read/write (#10895) * fix concurrent map read/write * simplify EndpointShardsByService * Update integration test job (#10888) * Fix integration test scripts * Making TestMain exit with the proper return code * Update local env references to native * Fix linter errors * Skipping integration tests in codecov since they fail * grant execute permission to e2e_pilotv2_auth_sds.sh (#10908) * grant execute permission to e2e_pilotv2_auth_sds.sh * fix typo * fix typo * typo * coredump * remove deprecated plugin from nodeagent (#10952) * Fix flaky test by reducing poll interval. (#10962) * Add interceptor to create noop spans when sampling is false (#10826) * Add interceptor to create noop spans when sampling is false * Add tests using mocktracer to determine whether span is created * Update dependencies to include OpenTracing mocktracer * Minor change * Updated dependencies again * Add support for ErrSpanContextNotFound error * Fix test and add one for x-b3-sampled=true * Fix lint error * set cluster.LoadAssignment only when service discovery type equals Cluster_STATIC Cluster_STRICT_DNS or Cluster_LOGICAL_DNS (#10926) * Remove Envoy's deprecated --v2-config-only (release-1.1). (#10960) Signed-off-by: Piotr Sikora <piotrsikora@google.com> * update check proxy version (#10769) * Add AWS CloudwatchLogs Adapter (code from #10400) (#10882) * Add AWS CloudwatchLogs Adapter (code from #10400) * Improve codecov * Even moar coverage * remove duplicate LoadAssignment set (#10977) * Enable server side control over maximum connection age (#10870) * add server side maximum connection age control to keepalive options * add server maximum connection age to the gRPC server keepalive options * missing space between concatenated strings * added tests for default values and setting via command line * fix golangci unconvert comment * add helm value file to google ca param (#10563) * add helm value file to preconfig param for googleca * cleanup * Allow pulling images from private repository (#10763) * Only compute diff for ServiceEntry (#10446) * Only compute diff for ServiceEntry This change prevents coredatamodel controller to compute the diff for all the types and it narrows it down to only ServiceEntry. * Add a dummy event for other config types - this dummy event allows DiscoveryServer to purge it's cache * Trigger a single clear cache event * add exponential backoff for retryable CSR error in nodeagent (#10969) * backoff * add unit test * clean up * lint * lint * address comment * typo * Fix flakiness in redisquota tests (#10906) * Fix flakiness in redisquota tests by adding retry for getting requests reported by prometheus One of the things I observed in flaky tests is that total number of requests reported by prometheus was not equal to traffic sent by Fortio. Thus adding a retry to make sure prometheus is queries till we get all requests reported. * Add a buffer for 5 requests to be allowed to be not reported. This buffer is within the error we allow for 200s and 429s reporting. * Fix based on reviews * Fix lint errors * Adding make sync to integ test script (#10984) * Removing Galley pod and node datasets from tests (#10953) * Use common image for node agent (#10949) * Use comment image for node agent * Revert node-agent-k8s * Sort the package * fix MCP server goroutine leak (#10893) * fix MCP server goroutine leak * fix race condition * fix race condition between reqChannel blocking and stream context done (#10998) * add default namespce for istio-init namespace. (#11012) * Handle outbound traffic policy (#10869) * add passthru listener only for mesh config outbound traffic policy ALLOW_ANY * add outbound traffic policy to configmap template and values * add the listener and blackhole cluster in case of outbound policy REGISTRY_ONLY * update DefaultMeshConfig with OutboundTrafficPolicy * use ALLOW_ANY outbound policy by default in tests * add OutboundTrafficPolicy to the default meshconfig of galley * Revert "use ALLOW_ANY outbound policy by default in tests" This reverts commit 90457899dc27d4e8cae8016520b1e320a77b2bb1. * use REGISTRY_ONLY OutboundTrafficPolicy for galley tests * adopt notion of collections throughout galley/mcp (#10963) * adopt notion of collections throughout galley/mcp * add missing 's/TypeURLs()/Collections()' * fix linter errors and missing dep * linter fixes * another linter fix * address review comments * use correct collection name in copilot test * fix TestConversion/config.istio.io_v1alpha2_circonus * update copilot e2e tests * fix pilot/pkg/config/coredatamodel/controller_test.go unit test * re-add TypeURL and remove typeurl from collections * add Bearer prefix in oauth token that passed to GoogleCA (#11018) * Add bionic and deb_slim base images, optimize size for xenial (#10992) * Remove redundant pieces of code (#11014) * Increase timeout (#11019) * mixer: gateway regression (#10966) * gateway test Signed-off-by: Kuat Yessenov <kuat@google.com> * prepare a test Signed-off-by: Kuat Yessenov <kuat@google.com> * Merge the new tests for isolation=none, some fixes (#10958) * Merge the new tests for isolation=none, some fixes * Add a local directory with certs, can be used with the basedir for local tests * If a BaseDir meta is specified, use it as prefix for the certs - so tests don't need / access * Add the pilot constant and doc * Fix mangled sidecarByNamespace, scope issue * Fix binding inbound listeners to 0.0.0.0, test * Format * Lint * Add back the validation * Reduce flakiness, golden diff reported as warning * Manual format, make fmt doesn't seem to help * Fix authn test * Fix authn test * Reduce parallel to avoid flakiness, fix copilot test * format * remove 'crds' option in relevant manifests (#11013) * remove crds option in istio chart. * delete crds option in values*.yaml * add istio-init as prerequisite of istio chart. * Delete this superfluous script. (#11028) * Refactor in preparation for reverse and incremental MCP (#11005) This PR refactors the MCP client, server, and monitoring packages in preparation for introducing reverse MCP. This includes the following changes: * Structs/Interfaces common to MCP sinks are moved into the sink package. * Structs/Interfaces common to MCP sources are moved into source packages. * The client and server metrics reporting logic is merged into a single reporter interface and implementation, since the majority of code is duplicated. This makes it easier to use a single reporter interface across all source/sink and client/server combinations. * Plumb through source/sink options * Port Mixer's TestTcpMetricTest in new Test framework (#10844) * Port Mixer's TestTcpMetricTest in new Test framework * Look at values file too to determine if mtls is enabled for the test or not. * Add unix domain socket client and server to pilot test apps (#10874) * Add unix domain socket client and server to pilot test apps Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * snafu Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * appends Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * compile fix Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * template fixes Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * more gotpl Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * undos * undo Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com> * Fixing new framework integration test (#11038) Fixes are as follows: 1) PolicyBackend close is failing when closing the listener in natice environment. Thus ignoring it's error and making policy backend a system component, so that it is just reset between the tests and not really closed. 2) Skipping conversion test in local environment as it requires kubernetes environment. 3) Increasing timeout of tests in kubernetes environment 4) Adding test namespace in mixer check test. * Use proxyLabels that were collected earlier (#11016) * Fix comment on defaultNodeSelector comment (#10980) * tracing: Provide default configuration when no host specified for k8s ingress (#10914) * tracing: Provide default configuration when no host specified for k8s ingress * Remove jaeger ingress in favour of one ingress with context based on provider * Updated to remove $ from .Values * Add ymesika to pilot owners (#11053) * Restart Galley in native test fw. component to avoid race. (#11048) There is a race between Galley reading the updated mesh config file and processing of input config files. This change restarts Galley every time mesh config is updated, to avoid race. * Update Istio API to include selector changes in AuthN/AuthZ. (#11046) The following changes are included from istio.io/api: aec9db9 Add option to select worload using lables for authn policy. (#755) 2dadb9e add optional incremental flag to ResponseSink and ResourceSource services (#762) d341fc8 assorted doc updates (#757) 48ad354 Update RBAC for Authorization v2 API. (#748) f818794 add optional header operations (#753) Signed-off-by: Yangmin Zhu <ymzhu@google.com> * update proxy SHA (#11036) * update proxy SHA * Update Proxy SHA to d2d0c62a045d12924180082e8e4b6fbe0a20de1d * Add an example helm values yaml for Vault integration user guide (#11024) * Add an example helm values yaml for Vault integration user guide * Add a comment * Add retry logic to the SDS grpc server of Node Agent (#11063) * Quick fix for istio/istio#10779 (#11061) * Basic fix to Ingress conversion. * Makes changes based on Ingress changes. * Linter fix. * Remove labels as well. * session affinity (#10730) * handle special char in trustdomain (to construct sa for secure naming) (#11066) * replace special char * update comment * enabled customized cluster domain for chart. (#11050) * enabled customized cluster domain for chart. * update webhook unit test data. * Restructure Galley sources (#11062) * Restructure Galley sources This is a series of simple moves in preparation for #10995 * addressing comments * assign back to s.mesh when reload the mesh config file (#11000) Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn> * Moving Galley source to dynamic package. (#11081) This is in preparation for #10995. Trying to do this move in order to preserve history. * Add reasonable default retry policy. (#10566) Partially addresses #7665. * Reduce flakiness in metrics test in new test framework (#11070) * Reduce flakiness in metrics test in new test framework * Fix based on review * Fix merge

istio/istio#10562 moved CRD.yaml files from ./istio/templates to ./istio-init/files. This change works with both the old and new paths for backwards compatibility.

googlebot added the cla: yes label Dec 18, 2018

istio-testing added the do-not-merge/hold Block automatic merging of a PR. label Dec 18, 2018

sdake requested review from costinm and ostromart December 19, 2018 04:49

sdake requested a review from linsun December 19, 2018 05:20

This was referenced Dec 19, 2018

Helm upgrades are broken related to CRDs #9884

Closed

Custom modifications we make when deploying istio #6037

Closed

Operator #10015

Closed

Initial implementation of Galley registers the CRDs #10120

Closed

ostromart reviewed Dec 19, 2018

View reviewed changes

sbezverk reviewed Dec 19, 2018

View reviewed changes

sdake commented Dec 19, 2018

View reviewed changes

install/kubernetes/helm/istio-init/Chart.yaml Outdated

@@ -0,0 +1,17 @@

apiVersion: v1

name: istio

Copy link

Member Author

sdake Dec 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this.

sdake commented Dec 19, 2018

View reviewed changes

istio-testing added the approved label Dec 19, 2018

ostromart approved these changes Dec 19, 2018

View reviewed changes

sdake mentioned this pull request Dec 26, 2018

WIP: tracing: Change tracing helm chart to use jaeger-operator #9508

Closed

istio-testing removed the approved label Dec 31, 2018

sdake force-pushed the istio-init branch 3 times, most recently from 6895c8e to 9017eed Compare December 31, 2018 20:08

linsun reviewed Jan 3, 2019

View reviewed changes

Address review comments.

73ba659

golangcibot reviewed Jan 7, 2019

View reviewed changes

Fix golint-ci errors

655a7f7

golangcibot reviewed Jan 7, 2019

View reviewed changes

sbezverk reviewed Jan 7, 2019

View reviewed changes

Address reviewer comments.

047ac22

sbezverk approved these changes Jan 7, 2019

View reviewed changes

Fix golintci failure

8e72244

golangcibot reviewed Jan 7, 2019

View reviewed changes

go linter error cleanup

4655b31

istio-testing assigned linsun Jan 8, 2019

istio-testing added the lgtm label Jan 8, 2019

istio-testing added the approved label Jan 8, 2019

istio-testing merged commit 45b6a79 into istio:release-1.1 Jan 8, 2019

dtzar mentioned this pull request Jan 8, 2019

remove helm chart global.crds setting #10813

Closed

therealmitchconnors mentioned this pull request Jan 11, 2019

Update setup.sh to new location for Istio CRDS istio/tools#34

Merged

gargnupur added a commit to gargnupur/istio that referenced this pull request Jan 11, 2019

Fix New Test Framework tests running in kubernetes environment

9636b78

After the change istio#10562 Istio Deployment in new test framework started failing. This PR tries to fix that

gargnupur mentioned this pull request Jan 11, 2019

Fix New Test Framework tests running in kubernetes environment #10889

Merged

This was referenced Jan 12, 2019

Fail to create CRDs by istio-init #10911

Closed

Add short names for crds #10296

Closed


		Helm charts expose configuration options which are currently in alpha. The currently exposed options are explained in the following table:

		\| Parameter \| Description \| Values \| Default \|


		## Introduction

		This chart bootstraps [CRDs](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)

Handle upgrades with an istio-init chart #10562

Handle upgrades with an istio-init chart #10562

Conversation

sdake commented Dec 18, 2018 • edited Loading

sdake commented Dec 18, 2018

sdake commented Dec 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbezverk left a comment

Choose a reason for hiding this comment

sdake commented Dec 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdake commented Dec 19, 2018 • edited Loading

linsun Jan 3, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdake Jan 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbezverk Jan 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbezverk commented Jan 7, 2019

sdake commented Jan 7, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linsun commented Jan 8, 2019

istio-testing commented Jan 8, 2019

sdake commented Dec 18, 2018 •

edited

Loading

sdake commented Dec 19, 2018 •

edited

Loading

sdake commented Dec 19, 2018 •

edited

Loading

linsun Jan 3, 2019 •

edited

Loading

sdake Jan 4, 2019 •

edited

Loading

sbezverk Jan 7, 2019 •

edited

Loading