Skip to content

Commit

Permalink
Merge branch 'main' into issue-10936
Browse files Browse the repository at this point in the history
  • Loading branch information
dishankkalra23 authored Oct 14, 2024
2 parents 4e058b7 + 5315c86 commit c5f775b
Show file tree
Hide file tree
Showing 14 changed files with 321 additions and 21 deletions.
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@ This module implements IBM Event Streams for IBM Cloud with topics, partitions,

The Event Streams service supports payload data encryption that uses a root key CRN of a key management service, such as Key Protect or Hyper Protect Crypto Services. You specify the root key CRN with the `kms_key_crn` input. For more information, see [Managing encryption in Event Streams](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption).

Before you run the module, configure an authorization policy to allow the Event Streams service to access the key management service instance with the reader role. For more information, see [Using authorizations to grant access between services](https://cloud.ibm.com/docs/account?topic=account-serviceauth).

You can't manage the policy in the same Terraform state file as the Event Streams service instance. When you issue a `terraform destroy` command, the instance is only soft deleted and remains as a reclamation resource for a while to support recovery (reclamation). An authorization policy must exist when the instance is hard deleted or reclaimed or else the unregistration of the instance from the root key fails on the backend. If the policy doesn't exist, the only way to unregister the instance, which is a requirement for deletion of the root key, is by opening a support case. For more information, see [Using a customer-managed key](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption#using_encryption).

<!-- Below content is automatically populated via pre-commit hook -->
<!-- BEGIN OVERVIEW HOOK -->
## Overview
Expand All @@ -25,6 +21,7 @@ You can't manage the policy in the same Terraform state file as the Event Stream
* [Examples](./examples)
* [Basic example](./examples/basic)
* [Complete example with topics and schema creation.](./examples/complete)
* [Financial Services Cloud profile example](./examples/fscloud)
* [Contributing](#contributing)
<!-- END OVERVIEW HOOK -->

Expand Down Expand Up @@ -110,6 +107,7 @@ You need the following permissions to run this module.
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3.0 |
| <a name="requirement_ibm"></a> [ibm](#requirement\_ibm) | >= 1.65.0, <2.0.0 |
| <a name="requirement_time"></a> [time](#requirement\_time) | >= 0.9.1 |

### Modules

Expand All @@ -123,8 +121,10 @@ You need the following permissions to run this module.
|------|------|
| [ibm_event_streams_schema.es_schema](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/event_streams_schema) | resource |
| [ibm_event_streams_topic.es_topic](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/event_streams_topic) | resource |
| [ibm_iam_authorization_policy.kms_policy](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/iam_authorization_policy) | resource |
| [ibm_resource_instance.es_instance](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_instance) | resource |
| [ibm_resource_key.service_credentials](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_key) | resource |
| [time_sleep.wait_for_authorization_policy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |

### Inputs

Expand All @@ -134,13 +134,16 @@ You need the following permissions to run this module.
| <a name="input_create_timeout"></a> [create\_timeout](#input\_create\_timeout) | The timeout value for creating an Event Streams instance. Specify `3h` for an Enterprise plan instance. Add 1 h for each level of non-default throughput. Add 30 min for each level of non-default storage size. | `string` | `"3h"` | no |
| <a name="input_delete_timeout"></a> [delete\_timeout](#input\_delete\_timeout) | The timeout value for deleting an Event Streams instance. | `string` | `"15m"` | no |
| <a name="input_es_name"></a> [es\_name](#input\_es\_name) | The name to give the Event Streams instance created by this module. | `string` | n/a | yes |
| <a name="input_kms_key_crn"></a> [kms\_key\_crn](#input\_kms\_key\_crn) | The root key CRN of the key management service (Key Protect or Hyper Protect Crypto Services) to use to encrypt the payload data. [Learn more](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption) about integrating Hyper Protect Crypto Services with Event Streams. Configure an authorization policy to allow the Event Streams service to access the key management service instance with the reader role ([Learn more](https://cloud.ibm.com/docs/account?topic=account-serviceauth)). You can't manage the policy in the same Terraform state file as the Event Streams service instance ([Learn more](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption#using\_encryption)). | `string` | `null` | no |
| <a name="input_existing_kms_instance_guid"></a> [existing\_kms\_instance\_guid](#input\_existing\_kms\_instance\_guid) | The GUID of the Hyper Protect Crypto Services or Key Protect instance in which the key specified in var.kms\_key\_crn is coming from. Required only if var.kms\_encryption\_enabled is set to true, var.skip\_iam\_authorization\_policy is set to false, and you pass a value for var.kms\_key\_crn. | `string` | `null` | no |
| <a name="input_kms_encryption_enabled"></a> [kms\_encryption\_enabled](#input\_kms\_encryption\_enabled) | Set this to true to control the encryption keys used to encrypt the data that you store in IBM Cloud® Databases. If set to false, the data is encrypted by using randomly generated keys. For more info on Key Protect integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect. For more info on HPCS integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs | `bool` | `false` | no |
| <a name="input_kms_key_crn"></a> [kms\_key\_crn](#input\_kms\_key\_crn) | The root key CRN of the key management service (Key Protect or Hyper Protect Crypto Services) to use to encrypt the payload data. [Learn more](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption) about integrating Hyper Protect Crypto Services with Event Streams. | `string` | `null` | no |
| <a name="input_plan"></a> [plan](#input\_plan) | The plan for the Event Streams instance. Possible values: `lite`, `standard`, `enterprise-3nodes-2tb`. | `string` | `"standard"` | no |
| <a name="input_region"></a> [region](#input\_region) | The region where the Event Streams are created. | `string` | `"us-south"` | no |
| <a name="input_resource_group_id"></a> [resource\_group\_id](#input\_resource\_group\_id) | The resource group ID where the Event Streams instance is created. | `string` | n/a | yes |
| <a name="input_schemas"></a> [schemas](#input\_schemas) | The list of schema objects. Include the `schema_id` and the `type` and `name` of the schema in the `schema` object. | <pre>list(object(<br/> {<br/> schema_id = string<br/> schema = object({<br/> type = string<br/> name = string<br/> fields = optional(list(object({<br/> name = string<br/> type = string<br/> })))<br/> })<br/> }<br/> ))</pre> | `[]` | no |
| <a name="input_service_credential_names"></a> [service\_credential\_names](#input\_service\_credential\_names) | The mapping of names and roles for service credentials that you want to create for the Event Notifications streams. | `map(string)` | `{}` | no |
| <a name="input_service_endpoints"></a> [service\_endpoints](#input\_service\_endpoints) | The type of service endpoints. Possible values: 'public', 'private', 'public-and-private'. | `string` | `"public"` | no |
| <a name="input_skip_iam_authorization_policy"></a> [skip\_iam\_authorization\_policy](#input\_skip\_iam\_authorization\_policy) | Set to true to skip the creation of an IAM authorization policy that permits all Event Streams database instances in the resource group to read the encryption key from the KMS instance. If set to false, pass in a value for the KMS instance in the existing\_kms\_instance\_guid variable. In addition, no policy is created if var.kms\_encryption\_enabled is set to false. | `bool` | `false` | no |
| <a name="input_storage_size"></a> [storage\_size](#input\_storage\_size) | Storage size of the Event Streams in GB. Applies only to Enterprise plan instances. Possible values: `2048`, `4096`, `6144`, `8192`, `10240`, `12288`. Storage capacity cannot be reduced after the instance is created. When the `throughput` input variable is set to `300`, storage size starts at 4096. When `throughput` is `450`, storage size starts starts at `6144`. | `number` | `"2048"` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | The list of tags associated with the Event Steams instance. | `list(string)` | `[]` | no |
| <a name="input_throughput"></a> [throughput](#input\_throughput) | Throughput capacity in MB per second. Applies only to Enterprise plan instances. Possible values: `150`, `300`, `450`. | `number` | `"150"` | no |
Expand Down
17 changes: 17 additions & 0 deletions examples/fscloud/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Financial Services Cloud profile example

An end-to-end example that uses the [Profile for IBM Cloud Framework for Financial Services](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/modules/fscloud) to deploy an instance of Event Streams.

The example uses the IBM Cloud Terraform provider to create the following infrastructure:

- A resource group, if one is not passed in.
- An IAM authorization between all Event Stream instances in the given resource group and the Hyper Protect Crypto Services instance that is passed in.
- An Event Streams instance that is encrypted with the Hyper Protect Crypto Services root key that is passed in.
- A sample virtual private cloud (VPC).
- A context-based restriction (CBR) rule to only allow Event Streams to be accessible from within the VPC.

:exclamation: **Important:** In this example, only the Event Streams instance complies with the IBM Cloud Framework for Financial Services. Other parts of the infrastructure do not necessarily comply.

## Before you begin

- You need a Hyper Protect Crypto Services instance and root key available in the region that you want to deploy your Event Streams instance to.
83 changes: 83 additions & 0 deletions examples/fscloud/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
##############################################################################
# Resource Group
##############################################################################

module "resource_group" {
source = "git::https://github.com/terraform-ibm-modules/terraform-ibm-resource-group.git?ref=v1.1.6"
# if an existing resource group is not set (null) create a new one using prefix
resource_group_name = var.resource_group == null ? "${var.prefix}-resource-group" : null
existing_resource_group_name = var.resource_group
}

##############################################################################
# Get Cloud Account ID
##############################################################################

data "ibm_iam_account_settings" "iam_account_settings" {
}

##############################################################################
# VPC
##############################################################################
resource "ibm_is_vpc" "example_vpc" {
name = "${var.prefix}-vpc"
resource_group = module.resource_group.resource_group_id
tags = var.resource_tags
}

resource "ibm_is_subnet" "testacc_subnet" {
name = "${var.prefix}-subnet"
vpc = ibm_is_vpc.example_vpc.id
zone = "${var.region}-1"
total_ipv4_address_count = 256
resource_group = module.resource_group.resource_group_id
}

##############################################################################
# Create CBR Zone
##############################################################################
module "cbr_zone" {
source = "terraform-ibm-modules/cbr/ibm//modules/cbr-zone-module"
version = "1.27.0"
name = "${var.prefix}-VPC-network-zone"
zone_description = "CBR Network zone representing VPC"
account_id = data.ibm_iam_account_settings.iam_account_settings.account_id
addresses = [{
type = "vpc", # to bind a specific vpc to the zone
value = ibm_is_vpc.example_vpc.crn,
}]
}


# #############################################################################
# Events-streams-instance
# #############################################################################

module "event_streams" {
source = "../../modules/fscloud"
resource_group_id = module.resource_group.resource_group_id
es_name = "${var.prefix}-es-fs"
kms_key_crn = var.kms_key_crn
schemas = var.schemas
tags = var.resource_tags
topics = var.topics
existing_kms_instance_guid = var.existing_kms_instance_guid
cbr_rules = [
{
description = "${var.prefix}-event stream access only from vpc"
enforcement_mode = "enabled"
account_id = data.ibm_iam_account_settings.iam_account_settings.account_id
rule_contexts = [{
attributes = [
{
"name" : "endpointType",
"value" : "private"
},
{
name = "networkZoneId"
value = module.cbr_zone.zone_id
}]
}]
}
]
}
33 changes: 33 additions & 0 deletions examples/fscloud/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
##############################################################################
# Outputs
##############################################################################

output "resource_group_name" {
description = "Resource group name"
value = module.resource_group.resource_group_name
}

output "resource_group_id" {
description = "Resource group ID"
value = module.resource_group.resource_group_id
}

output "crn" {
description = "Event Streams instance crn"
value = module.event_streams.crn
}

output "guid" {
description = "Event Streams instance guid"
value = module.event_streams.guid
}

output "kafka_brokers_sasl" {
description = "(Array of Strings) Kafka brokers use for interacting with Kafka native API"
value = module.event_streams.kafka_brokers_sasl
}

output "kafka_http_url" {
description = "The API endpoint to interact with Event Streams REST API"
value = module.event_streams.kafka_http_url
}
4 changes: 4 additions & 0 deletions examples/fscloud/provider.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
provider "ibm" {
ibmcloud_api_key = var.ibmcloud_api_key
region = var.region
}
65 changes: 65 additions & 0 deletions examples/fscloud/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
variable "ibmcloud_api_key" {
type = string
description = "The IBM Cloud API Key"
sensitive = true
}

variable "region" {
type = string
description = "Region to provision all resources created by this example"
default = "us-south"
}

variable "prefix" {
type = string
description = "Prefix to append to all resources created by this example"
default = "fs-cloud"
}

variable "resource_group" {
type = string
description = "An existing resource group name to use for this example, if unset a new resource group will be created"
default = null
}

variable "resource_tags" {
type = list(string)
description = "List of tags associated with the Event Steams instance"
default = []
}

variable "schemas" {
type = list(object(
{
schema_id = string
schema = object({
type = string
name = string
})
}
))
description = "The list of schema object which contains schema id and format of the schema"
default = []
}

variable "topics" {
type = list(object(
{
name = string
partitions = number
config = object({})
}
))
description = "List of topics. For lite plan only one topic is allowed."
default = []
}

variable "existing_kms_instance_guid" {
description = "The GUID of the Hyper Protect Crypto service in which the key specified in var.kms_key_crn is coming from"
type = string
}

variable "kms_key_crn" {
type = string
description = "The root key CRN of a Hyper Protect Crypto Service (HPCS) that you want to use for disk encryption. See https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs&interface=ui for more information on integrating HPCS with Event Streams instance."
}
10 changes: 10 additions & 0 deletions examples/fscloud/version.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
terraform {
required_version = ">= 1.3.0"
required_providers {
# Use latest version of provider in non-basic examples to verify latest version works with module
ibm = {
source = "IBM-Cloud/ibm"
version = ">= 1.56.1"
}
}
}
Loading

0 comments on commit c5f775b

Please sign in to comment.