Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(provider/docker): Added insecure registry support #1887

Merged
merged 25 commits into from
Sep 29, 2017

Conversation

podnov
Copy link
Contributor

@podnov podnov commented Sep 11, 2017

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.
@podnov
Copy link
Contributor Author

podnov commented Sep 11, 2017

I'd love to be able to write a better test for this case, but I can't think of a good way short of setting up a publicly accessible insecure registry.

@edwinavalos
Copy link
Contributor

@lwander wander says, "Tell him @lwanders is very happy to see this"

We did run into a similar issue at Target, in the end we decided to create custom containers and specified them via artifactId, https://www.spinnaker.io/reference/halyard/custom/#tweakable-service-settings

podnov pushed a commit to podnov/halyard that referenced this pull request Sep 11, 2017
Added --insecure-registry=[true|false] as a command line flag
when adding a docker-registry account.

This is dependent on spinnaker/clouddriver#1887 and will
require a dependency version bump once merged and released.

This fixes spinnaker/halyard#680.
podnov added a commit to podnov/halyard that referenced this pull request Sep 11, 2017
Added --insecure-registry=[true|false] as a command line flag
when adding a docker-registry account.

This is dependent on spinnaker/clouddriver#1887 and will
require a dependency version bump once merged and released.

This fixes spinnaker/halyard#680.
podnov added a commit to podnov/halyard that referenced this pull request Sep 11, 2017
Added --insecure-registry=[true|false] as a command line flag
when adding a docker-registry account.

This is dependent on spinnaker/clouddriver#1887 and will
require a dependency version bump once merged and released.

This fixes spinnaker/halyard#680.
podnov added a commit to podnov/halyard that referenced this pull request Sep 11, 2017
Added --insecure-registry=[true|false] as a command line flag
when adding a docker-registry account.

This is dependent on spinnaker/clouddriver#1887 and will
require a dependency version bump once merged and released.

This fixes spinnaker/halyard#680.
podnov added a commit to podnov/halyard that referenced this pull request Sep 11, 2017
Added --insecure-registry=[true|false] as a command line flag
when adding a docker-registry account.

This is dependent on spinnaker/clouddriver#1887 and will
require a dependency version bump once merged and released.

This fixes spinnaker/halyard#680.
Copy link
Member

@lwander lwander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - but this isn't my area of expertise - could @ttomsu and @tomaslin take a look?

@@ -92,16 +97,21 @@ class DockerRegistryClient {
return this
}

Builder insecureRegistry(boolean insecureRegistry) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see why the halyard build failed - we need to release this change to clouddriver once it's merged - I'll push it through

@lwander
Copy link
Member

lwander commented Sep 11, 2017

BTW thanks for this :)

@podnov
Copy link
Contributor Author

podnov commented Sep 11, 2017

My pleasure :)

DockerRegistryClient(String address, String email, String username, String password, long clientTimeoutMillis, int paginateSize, String catalogFile) {
this(address, clientTimeoutMillis, paginateSize)
DockerRegistryClient(String address, String email, String username, String password, long clientTimeoutMillis, int paginateSize, String catalogFile, boolean insecureRegistry) {
this(address, clientTimeoutMillis, paginateSize, catalogFile, insecureRegistry)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think f70d6ea introduces a potential bug as it's not passing catalogFile to the underlying constructor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@podnov I just merged this last night. I'll double check that. If it's missing, it was an oversight! Thanks for the heads up!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@podnov fixed in #1903

DockerRegistryClient(String address, String email, String username, File passwordFile, long clientTimeoutMillis, int paginateSize, String catalogFile) {
this(address, clientTimeoutMillis, paginateSize)
DockerRegistryClient(String address, String email, String username, File passwordFile, long clientTimeoutMillis, int paginateSize, String catalogFile, boolean insecureRegistry) {
this(address, clientTimeoutMillis, paginateSize, catalogFile, insecureRegistry)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think f70d6ea introduces a potential bug as it's not passing catalogFile to the underlying constructor.

@podnov
Copy link
Contributor Author

podnov commented Sep 29, 2017

Is there anything I can do to help get this merged?

@lwander
Copy link
Member

lwander commented Sep 29, 2017

Sorry for the wait, I took another read & looked at the trust manager docs. LGTM

@lwander lwander merged commit de6fea0 into spinnaker:master Sep 29, 2017
@lwander
Copy link
Member

lwander commented Sep 29, 2017

I will cut a release of clouddriver, update spinnaker-dependencies & then show you how to integrate this change into halyard.

@lwander
Copy link
Member

lwander commented Sep 29, 2017

Actually... there's a bug in clouddriver k8s initialization that I don't want to pull into halyard - let's get #1933 resolved first.

@podnov podnov deleted the clouddriver-docker-insecure-registry branch September 29, 2017 13:33
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Oct 26, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* Merged upstream, updated AbstractEcsCachingAgent and Keys.java
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Oct 26, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* target group is now a string, instead of an array of strigns
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Nov 6, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* Updating API for IAM role dropdown to work nicely with Deck
lwander pushed a commit to lwander/clouddriver that referenced this pull request Aug 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants