Configurable resource requirements for APIManager components

[miguelsorianod]

This PR adds configurable resource requirements for APIManager components.

Summary of the implemented behaviour

• Configurable resource requirements per component has been added
• When resource requirements are not specified at component level the global defaults for the component are used for it (if enabled)
• When resource requirements are specified at component level, they take over preference over the global default and they follow a REPLACE behaviour (instead of merge behaviour) with the defaults.

Detailed implemented behaviour

• global-level defaults for resource limits and requests for the components of 3scale are provided. This global-level defaults can be enabled or disabled through a CR attribute that already existed before this PR, named 'resourceRequirementsEnabled'.
• component-level resources attribute is provided at component level (for example spec.backend.listenerSpec.resources attribute):
• If it is not set by the user or set to nil and the global-level CR attribute is not set to true then no resource limits and/or requests are enforced
• If it is not set by the user or set to nil and the global-level CR attribute is set to true then the defaults for that given component are set
• If it is set by the user to a non-nil value then the resources requests and limits set are the ones specified by the user with a REPLACE behaviour, not a merge behavior. Global defaults are not used for that component then, independently of the value of the global-level CR attribute. In this case, omitting a limit, request or cpu,memory inside a limit o request is equivalent to not enforcing it for that component. This is the same behavior that ResourceLimits have in Kubernetes.

Possible use case scenarios

1. The user wants to have default limits and requests for all components enabled and wants to modify them for a specific DeploymentConfig

• The user is able to do that by setting the global resourceRequirementsEnabled attribute to true (default behavior) and then specifying the desired
  values on the corresponding resources attribute provided at component level in the APIManager CR

2. The user wants to have default limits and requests for all components disabled and wants to set them with custom values for a specific DeploymentConfig

• The user is able to do that by setting the global resourceRequirementsEnabled attribute to false and then specifying the desired values on the
  corresponding resources attribute provided at component level in the APIManager CR
• Limitation: The user has no way to specify not having a default limit and having a custom request for that same component or viceversa

3. The user wants to have default limits and requests for all components enabled and wants to disable them for a specific DeploymentConfig

• The user is able to do that by setting the global resourceRequirementsEnabled attribute to true (default behavior) and then setting the corresponding
  resources attribute provided at component level in the APIManager CR as the empty map value {}

Unsupported scenarios

Basically the ones that involve a potentially desired merge behavior between global defaults and custom values:

1. On a given component, having a global default resource request and a custom resource limit for that same component or viceversa
2. On a given component, not having a global default resource request and having a custom resource limit for the same component or viceversa
3. On a given component, having a global default CPU and having custom Memory, or having global default Memory and having custom CPU for resource requirements
4. On a given component, having a global default CPU and having custom Memory, or having global default Memory and using custom CPU for resource limits

Allowing those unsupported scenarios would require us to stop using the v1.ResourceRequirements type provided by Kubernetes in our CR and provide our own type where we can differentiate between setting resource requests to nil or empty, as the Kubernetes implementation treats those values with the same behavior, which is not enforcing them in that case.
The alternative implementation:

• Increases complexity
• May not be intuitive for users as it differs from how v1.ResourceRequirements field works (nil, vs empty, ...)
• Removes flexibility. Right now by reusing v1.ResourceRequirements other possible additional resources can be specified potentially, we would not be limited to CPU and requests. We would not be able to take advantage on changes/improvements on that type
• Allows more configuration combinations between defaults and custom resources at both requests/limits level and cpu/memory inside requests/limits
• Opens up new additional questions about how to specify cpus and memory and other potential resources (provide a map as kubernetes does, attribute fields? use Quantity attribute? etc...)

• Documentation

[eguzki]

Configurable resource requirements per component has been added

👍 per component is per container.

When resource requirements are not specified at component level the global defaults for the component are used for it (if enabled)

Good. Intuitive and common sense.

When resource requirements are specified at component level, they take over preference over the global default and they follow a REPLACE behaviour (instead of merge behaviour) with the defaults.

Agree. REPLACE behavior is easier and enough. Merge behavior (with defaults) can be confusing and requires not standard data structure type in the CRD to define resource requirements. REPLACE behavior requires, though, to know existing defaults. A user may want to update cpu limit and leave cpu requests, memory limits and memory requests "as defaults". So the user needs to create structure with desired cpu limit and copy other values from defaults. For example:

resources:
      requests:
        memory: "64Mi" <- copied from defaults
        cpu: "250m" <- copied from defaults
      limits:
        memory: "128Mi" <- copied from defaults
        cpu: "500m" <- desired cpu value

For that, default values should be documented.

[eguzki]

Overall LGTM.

Left some comments. I also miss some option provider tests.

[miguelsorianod]

Remanining tests added.

Documentation pending.

[miguelsorianod]

Added documentation

[eguzki]

fix the broken link and merge.

[codeclimate]

Code Climate has analyzed commit 7b3bf5a and detected 15 issues on this pull request.

Here's the issue category breakdown:

Category	Count
Complexity	1
Duplication	14

View more on Code Climate.