Skip to content

Commit

Permalink
fix!: moved require_partition_filter to the top (#335)
Browse files Browse the repository at this point in the history
  • Loading branch information
g-awmalik authored Jul 29, 2024
1 parent 470c5cc commit 90f931f
Showing 8 changed files with 82 additions and 40 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -200,7 +200,7 @@ This module provisions a dataset and a list of tables with associated JSON schem
| max\_time\_travel\_hours | Defines the time travel window in hours | `number` | `null` | no |
| project\_id | Project where the dataset and table are created | `string` | n/a | yes |
| routines | A list of objects which include routine\_id, routine\_type, routine\_language, definition\_body, return\_type, routine\_description and arguments. | <pre>list(object({<br> routine_id = string,<br> routine_type = string,<br> language = string,<br> definition_body = string,<br> return_type = string,<br> description = string,<br> arguments = list(object({<br> name = string,<br> data_type = string,<br> argument_kind = string,<br> mode = string,<br> })),<br> }))</pre> | `[]` | no |
| tables | A list of objects which include table\_id, table\_name, schema, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels. | <pre>list(object({<br> table_id = string,<br> description = optional(string),<br> table_name = optional(string),<br> schema = string,<br> clustering = list(string),<br> time_partitioning = object({<br> expiration_ms = string,<br> field = string,<br> type = string,<br> require_partition_filter = bool,<br> }),<br> range_partitioning = object({<br> field = string,<br> range = object({<br> start = string,<br> end = string,<br> interval = string,<br> }),<br> }),<br> expiration_time = string,<br> deletion_protection = optional(bool),<br> labels = map(string),<br> }))</pre> | `[]` | no |
| tables | A list of objects which include table\_id, table\_name, schema, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels. | <pre>list(object({<br> table_id = string,<br> description = optional(string),<br> table_name = optional(string),<br> schema = string,<br> clustering = list(string),<br> require_partition_filter = optional(bool),<br> time_partitioning = object({<br> expiration_ms = string,<br> field = string,<br> type = string,<br> }),<br> range_partitioning = object({<br> field = string,<br> range = object({<br> start = string,<br> end = string,<br> interval = string,<br> }),<br> }),<br> expiration_time = string,<br> deletion_protection = optional(bool),<br> labels = map(string),<br> }))</pre> | `[]` | no |
| views | A list of objects which include view\_id and view query | <pre>list(object({<br> view_id = string,<br> description = optional(string),<br> query = string,<br> use_legacy_sql = bool,<br> labels = map(string),<br> }))</pre> | `[]` | no |

## Outputs
43 changes: 43 additions & 0 deletions docs/upgrading_to_bigquery_v8.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Upgrading to BigQuery v8.0

The v8.0 release of BigQuery is a backwards incompatible release.

- The supported provider has been updated to v5.3
- `require_partition_filter` has been deprecated under the `time_partitioning`
block and can be used at the top level with the same name instead.

## Migration Instructions

1. Upgrade version

```diff
module "bigquery" {
source = "terraform-google-modules/bigquery/google"
- version = "~> 7.0"
+ version = "~> 8.0"
....
}
```

2. Remove `require_partition_filter` from within the `time_partitioning` block
and set it at the top level, if required.

```diff
module "bigquery" {
source = "terraform-google-modules/bigquery/google"
....
tables = [
{
....
+ require_partition_filter = true,
time_partitioning = {
....
- require_partition_filter = true,
....
},
....
}
]
...
}
```
2 changes: 1 addition & 1 deletion examples/basic_view/README.md
Original file line number Diff line number Diff line change
@@ -11,7 +11,7 @@ This is a common practice for providing limited data in a different dataset.
| delete\_contents\_on\_destroy | (Optional) If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present. | `bool` | `null` | no |
| table\_dataset\_labels | A mapping of labels to assign to the table. | `map(string)` | n/a | yes |
| table\_project\_id | Project where the dataset and table are created. | `any` | n/a | yes |
| tables | A list of maps that includes table\_id, schema, clustering, time\_partitioning, range\_partitioning, view, expiration\_time, labels in each element. | <pre>list(object({<br> table_id = string,<br> schema = string,<br> clustering = list(string),<br> time_partitioning = object({<br> expiration_ms = string,<br> field = string,<br> type = string,<br> require_partition_filter = bool,<br> }),<br> range_partitioning = object({<br> field = string,<br> range = object({<br> start = string,<br> end = string,<br> interval = string,<br> }),<br> }),<br> expiration_time = string,<br> labels = map(string),<br> }))</pre> | `[]` | no |
| tables | A list of maps that includes table\_id, schema, clustering, time\_partitioning, range\_partitioning, view, expiration\_time, labels in each element. | <pre>list(object({<br> table_id = string,<br> schema = string,<br> clustering = list(string),<br> time_partitioning = object({<br> expiration_ms = string,<br> field = string,<br> type = string,<br> }),<br> range_partitioning = object({<br> field = string,<br> range = object({<br> start = string,<br> end = string,<br> interval = string,<br> }),<br> }),<br> expiration_time = string,<br> labels = map(string),<br> }))</pre> | `[]` | no |
| view\_dataset\_labels | A mapping of labels to assign to the table. | `map(string)` | n/a | yes |
| view\_project\_id | Project where the dataset and table are created. | `any` | n/a | yes |
| views | A list of objects which include table\_id, which is view id, and view query | <pre>list(object({<br> view_id = string,<br> query = string,<br> use_legacy_sql = bool,<br> labels = map(string),<br> }))</pre> | `[]` | no |
7 changes: 3 additions & 4 deletions examples/basic_view/variables.tf
Original file line number Diff line number Diff line change
@@ -37,10 +37,9 @@ variable "tables" {
schema = string,
clustering = list(string),
time_partitioning = object({
expiration_ms = string,
field = string,
type = string,
require_partition_filter = bool,
expiration_ms = string,
field = string,
type = string,
}),
range_partitioning = object({
field = string,
30 changes: 15 additions & 15 deletions main.tf
Original file line number Diff line number Diff line change
@@ -65,25 +65,25 @@ resource "google_bigquery_dataset" "main" {
}

resource "google_bigquery_table" "main" {
for_each = local.tables
dataset_id = google_bigquery_dataset.main.dataset_id
friendly_name = each.value["table_name"] != null ? each.value["table_name"] : each.key
table_id = each.key
description = each.value["description"]
labels = each.value["labels"]
schema = each.value["schema"]
clustering = each.value["clustering"]
expiration_time = each.value["expiration_time"] != null ? each.value["expiration_time"] : 0
project = var.project_id
deletion_protection = coalesce(each.value["deletion_protection"], var.deletion_protection)
for_each = local.tables
dataset_id = google_bigquery_dataset.main.dataset_id
friendly_name = each.value["table_name"] != null ? each.value["table_name"] : each.key
table_id = each.key
description = each.value["description"]
labels = each.value["labels"]
schema = each.value["schema"]
clustering = each.value["clustering"]
expiration_time = each.value["expiration_time"] != null ? each.value["expiration_time"] : 0
project = var.project_id
deletion_protection = coalesce(each.value["deletion_protection"], var.deletion_protection)
require_partition_filter = each.value["require_partition_filter"]

dynamic "time_partitioning" {
for_each = each.value["time_partitioning"] != null ? [each.value["time_partitioning"]] : []
content {
type = time_partitioning.value["type"]
expiration_ms = time_partitioning.value["expiration_ms"] != null ? time_partitioning.value["expiration_ms"] : 0
field = time_partitioning.value["field"]
require_partition_filter = time_partitioning.value["require_partition_filter"]
type = time_partitioning.value["type"]
expiration_ms = time_partitioning.value["expiration_ms"] != null ? time_partitioning.value["expiration_ms"] : 0
field = time_partitioning.value["field"]
}
}

18 changes: 9 additions & 9 deletions metadata.yaml
Original file line number Diff line number Diff line change
@@ -182,16 +182,16 @@ spec:
description: A list of objects which include table_id, table_name, schema, clustering, time_partitioning, range_partitioning, expiration_time and labels.
varType: |-
list(object({
table_id = string,
description = optional(string),
table_name = optional(string),
schema = string,
clustering = list(string),
table_id = string,
description = optional(string),
table_name = optional(string),
schema = string,
clustering = list(string),
require_partition_filter = optional(bool),
time_partitioning = object({
expiration_ms = string,
field = string,
type = string,
require_partition_filter = bool,
expiration_ms = string,
field = string,
type = string,
}),
range_partitioning = object({
field = string,
18 changes: 9 additions & 9 deletions variables.tf
Original file line number Diff line number Diff line change
@@ -98,16 +98,16 @@ variable "tables" {
description = "A list of objects which include table_id, table_name, schema, clustering, time_partitioning, range_partitioning, expiration_time and labels."
default = []
type = list(object({
table_id = string,
description = optional(string),
table_name = optional(string),
schema = string,
clustering = list(string),
table_id = string,
description = optional(string),
table_name = optional(string),
schema = string,
clustering = list(string),
require_partition_filter = optional(bool),
time_partitioning = object({
expiration_ms = string,
field = string,
type = string,
require_partition_filter = bool,
expiration_ms = string,
field = string,
type = string,
}),
range_partitioning = object({
field = string,
2 changes: 1 addition & 1 deletion versions.tf
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ terraform {

google = {
source = "hashicorp/google"
version = ">= 4.42, < 6"
version = ">= 5.3, < 6"
}
}

0 comments on commit 90f931f

Please sign in to comment.