Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Google KMS transformer support with envelope encryption #2

Closed
wants to merge 39 commits into from

Conversation

sakshamsharma
Copy link
Owner

@sakshamsharma sakshamsharma commented Jun 19, 2017

Implements an envelope encryption scheme to allow Google KMS transformer as an encryption provider.

@sakshamsharma
Copy link
Owner Author

sakshamsharma commented Jun 19, 2017

@jcbsmpsn These changes were growing to be too big, and thus, I thought it may be good to get a review / suggestions, before I merge the KEK DEK logic too.

@sakshamsharma sakshamsharma force-pushed the kmstransformer branch 4 times, most recently from cc2d6e7 to 796fc6a Compare June 21, 2017 00:04
yiqinguo and others added 8 commits June 21, 2017 15:15
If unspecified in probe definition, User-Agent will be set to
`kube-probe/<version major.minor>` on httpGet probe types
instead of the default Go User-Agent.
Support vmodule (via `LOG_SPEC`) for all local cluster components.
@sakshamsharma
Copy link
Owner Author

sakshamsharma commented Jun 26, 2017

@jcbsmpsn @destijl @cjcullen I have a working implementation of envelope encryption using Google KMS included in this PR now. Requesting initial review before opening a PR on the main branch.

Update:
Of interest, the documentation for encrypting secrets is up as a PR. It also contains a hacky way to rotate keys in the AES transformer (which requires rebooting the cluster and re-reading the configuration file).

Update 2:
Would also like to discuss how to write tests for this feature. Does it have to be an e2e test? It does require Google credentials, but can be tested without GKE. Update: I am planning to write a unit test, with mocked up versions of the KMS API and etcd API.

@sakshamsharma sakshamsharma force-pushed the kmstransformer branch 2 times, most recently from 69c83a1 to 834ea57 Compare June 26, 2017 20:18
@sakshamsharma sakshamsharma changed the title Add elementary KMS transformer support Add Google KMS transformer support with envelope encryption Jun 26, 2017
@sakshamsharma sakshamsharma force-pushed the kmstransformer branch 6 times, most recently from f342e82 to 32a7ba6 Compare June 26, 2017 22:12
danwinship and others added 3 commits June 27, 2017 09:26
The delete tests were copy+pasted from the create tests, and the names
not fully updated to match.
@sakshamsharma sakshamsharma force-pushed the kmstransformer branch 2 times, most recently from 51190c0 to 835cfde Compare June 28, 2017 22:36
Kubernetes Submit Queue and others added 25 commits June 28, 2017 17:11
Automatic merge from submit-queue (batch tested with PRs 48214, 48154)

Allow creating special node for running heapster in GCE

This should enable scheduling heapster pod(s) in our large cluster tests, where the minions are very small (n1-standard-1) and heapster can't schedule on them due to high memory requests.
We need this for running heapster dependent correctness tests in 5k-node cluster.

cc @kubernetes/sig-scalability-misc @kubernetes/heapster-maintainers @gmarek
Automatic merge from submit-queue (batch tested with PRs 48214, 48154)

Adding a retry and traceroute to the master version checking

This is hitting a lot of connection refused errors in the e2e upgrade tests. We should make this more robust in case this is intermittent network errors. In the event of an error, attempt to log a traceroute to the master.

cc @kubernetes/sig-cluster-lifecycle-bugs @dchen1107 

kubernetes#47379
…e_coversions

Automatic merge from submit-queue

Remove repeat type conversions

Here is the type of conversion for the variable is repeated.

**Release note**:
```release-note
NONE
```
Implement GetCapacity in container_manager_unsupported
Automatic merge from submit-queue

fixes to two pkg/util unit tests

Spun out from kubernetes#46537. Trivial unit test fixes:
- `pkg/util/ebtables/ebtables_test.go`'s tests are incorrectly named and so weren't getting run by "go test". (And one of them had a typo in it so it failed after the name was fixed)
- In `pkg/util/iptables/iptables_test.go`, the deletion tests were apparently written by copying+pasting+editing the creation tests, but they forgot to change the names of the tests to match the behavior that was being tested in the new version
Automatic merge from submit-queue (batch tested with PRs 48137, 48172)

Used const variable in scheduler test.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
none
```
…ure_cloud_provider

Automatic merge from submit-queue (batch tested with PRs 47286, 47729)

Add client certificate authentication to Azure cloud provider

This adds support for client cert authentication in Azure cloud provider. The certificate can be provided in PKCS kubernetes#12 format with password protection. Not that this authentication will be active only when no client secret is configured.

cc @brendandburns @colemickens
Automatic merge from submit-queue (batch tested with PRs 47286, 47729)

Set default User-Agent on http probe

**Set a default User-Agent on `httpGet` probes**:

Currently the default Go HTTP client sets a `User-Agent` specific to the language and version, but every Go client has the same one.  In Kubernetes, users can override the User-Agent by setting a header in their probe definition, but its tedious to do this everywhere.

This PR changes the default User-Agent only if not set (or removed) in the probe definition.

**Which issue this PR fixes** 
fixes kubernetes#29025

**Special notes for your reviewer**:

**Release note**:

```release-note
Change default `httpGet` probe `User-Agent` to `kube-probe/<version major.minor>` if none specified, overriding the default Go `User-Agent`.
```
Also, remove the flags that have been deprecated in the new version.
…udgment

Automatic merge from submit-queue

Add type conversion judgment

If do not type conversion judgment, there may be panic.

**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue

Allow log-dumping only N randomly-chosen nodes in the cluster

This should let us save "lots" (~3-4 hours) of time in our 5000-node cluster scale tests as we copy logs from all the nodes to jenkins worker and then upload all of them to gcs (while we don't need too many).
This will also prevent the jenkins container facing "No space left on device" error while dumping logs, that we saw in runs 12-13 of gce-enormous-cluster.

The longterm fix will be to enable [logexporter](https://github.com/kubernetes/test-infra/tree/master/logexporter) for our tests.

cc @kubernetes/sig-scalability-misc @kubernetes/test-infra-maintainers @gmarek @fejta
Automatic merge from submit-queue (batch tested with PRs 46850, 47984)

Update addon-resizer version

Update addon-resizer version and remove the flags that have been deprecated in the new version.

**What this PR does / why we need it**:
ref kubernetes-retired/contrib#2623

**Special notes for your reviewer**:
Need to wait for merging kubernetes-retired/contrib#2623 first.

**Release note**:
```release-note
addon-resizer flapping behavior was removed.
```
…odule

Automatic merge from submit-queue (batch tested with PRs 46850, 47984)

Enable vmodule support for all components

Support vmodule (via `LOG_SPEC`) for all local cluster components.

```release-note
NONE
```
sakshamsharma pushed a commit that referenced this pull request Oct 28, 2017
This is part of the move to external cloud providers. Please see
plan detail in issue 50986. This PR covers step #2:
 v1.9 - set no cloud provider as the default in kubelet but still allow
 opt in for auto-detect
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants