Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vertex_ai: add google_vertex_ai_index_endpoint_deployed_index resource #764

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
resource "google_vertex_ai_index_endpoint_deployed_index" "automatic_resources" {
depends_on = [ google_vertex_ai_index_endpoint.vertex_endpoint ]
index_endpoint = google_vertex_ai_index_endpoint.vertex_endpoint.id
index = google_vertex_ai_index.index.id // this is the index that will be deployed onto an endpoint
deployed_index_id = "deployed_index_id-${local.name_suffix}"
display_name = "vertex-deployed-index-${local.name_suffix}"
automatic_resources{
max_replica_count = 2
min_replica_count = 1
}
deployment_group = "test"
}

resource "google_storage_bucket" "bucket" {
name = "bucket-name-${local.name_suffix}"
location = "us-central1"
uniform_bucket_level_access = true
}

# The sample data comes from the following link:
# https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#specify-namespaces-tokens
resource "google_storage_bucket_object" "data" {
name = "contents/data.json"
bucket = google_storage_bucket.bucket.name
content = <<EOF
{"id": "42", "embedding": [0.5, 1.0], "restricts": [{"namespace": "class", "allow": ["cat", "pet"]},{"namespace": "category", "allow": ["feline"]}]}
{"id": "43", "embedding": [0.6, 1.0], "restricts": [{"namespace": "class", "allow": ["dog", "pet"]},{"namespace": "category", "allow": ["canine"]}]}
EOF
}

resource "google_vertex_ai_index" "index" {
labels = {
foo = "bar"
}
region = "us-central1"
display_name = "test-index-${local.name_suffix}"
description = "index for test"
metadata {
contents_delta_uri = "gs://${google_storage_bucket.bucket.name}/contents"
config {
dimensions = 2
approximate_neighbors_count = 150
shard_size = "SHARD_SIZE_MEDIUM"
distance_measure_type = "DOT_PRODUCT_DISTANCE"
algorithm_config {
tree_ah_config {
leaf_node_embedding_count = 500
leaf_nodes_to_search_percent = 7
}
}
}
}
index_update_method = "BATCH_UPDATE"
}


resource "google_vertex_ai_index_endpoint" "vertex_endpoint" {
display_name = "sample-endpoint"
description = "A sample vertex endpoint"
region = "us-central1"
labels = {
label-one = "value-one"
}
public_endpoint_enabled = true
}

data "google_project" "project" {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Vertex Ai Index Endpoint Deployed Index Automatic Resources - Terraform

## Setup

<walkthrough-author name="rileykarson@google.com" analyticsId="UA-125550242-1" tutorialName="vertex_ai_index_endpoint_deployed_index_automatic_resources" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
15 changes: 15 additions & 0 deletions vertex_ai_index_endpoint_deployed_index_basic/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
77 changes: 77 additions & 0 deletions vertex_ai_index_endpoint_deployed_index_basic/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
resource "google_service_account" "sa" {
account_id = "vertex-sa-${local.name_suffix}"
}

resource "google_vertex_ai_index_endpoint_deployed_index" "basic_deployed_index" {
depends_on = [ google_vertex_ai_index_endpoint.vertex_index_endpoint_deployed, google_service_account.sa ]
index_endpoint = google_vertex_ai_index_endpoint.vertex_index_endpoint_deployed.id
index = google_vertex_ai_index.index.id // this is the index that will be deployed onto an endpoint
deployed_index_id = "deployed_index_id-${local.name_suffix}"
reserved_ip_ranges = ["vertex-ai-range-${local.name_suffix}"]
enable_access_logging = false
display_name = "vertex-deployed-index-${local.name_suffix}"
deployed_index_auth_config{
auth_provider{
audiences = ["123456-my-app"]
allowed_issuers = ["${google_service_account.sa.email}"]
}
}
}

resource "google_storage_bucket" "bucket" {
name = "bucket-name-${local.name_suffix}"
location = "us-central1"
uniform_bucket_level_access = true
}

# The sample data comes from the following link:
# https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#specify-namespaces-tokens
resource "google_storage_bucket_object" "data" {
name = "contents/data.json"
bucket = google_storage_bucket.bucket.name
content = <<EOF
{"id": "42", "embedding": [0.5, 1.0], "restricts": [{"namespace": "class", "allow": ["cat", "pet"]},{"namespace": "category", "allow": ["feline"]}]}
{"id": "43", "embedding": [0.6, 1.0], "restricts": [{"namespace": "class", "allow": ["dog", "pet"]},{"namespace": "category", "allow": ["canine"]}]}
EOF
}

resource "google_vertex_ai_index" "index" {
labels = {
foo = "bar"
}
region = "us-central1"
display_name = "test-index-${local.name_suffix}"
description = "index for test"
metadata {
contents_delta_uri = "gs://${google_storage_bucket.bucket.name}/contents"
config {
dimensions = 2
approximate_neighbors_count = 150
shard_size = "SHARD_SIZE_SMALL"
distance_measure_type = "DOT_PRODUCT_DISTANCE"
algorithm_config {
tree_ah_config {
leaf_node_embedding_count = 500
leaf_nodes_to_search_percent = 7
}
}
}
}
index_update_method = "BATCH_UPDATE"
}

resource "google_vertex_ai_index_endpoint" "vertex_index_endpoint_deployed" {
display_name = "sample-endpoint"
description = "A sample vertex endpoint"
region = "us-central1"
labels = {
label-one = "value-one"
}
network = "projects/${data.google_project.project.number}/global/networks/${data.google_compute_network.vertex_network.name}"
}

data "google_compute_network" "vertex_network" {
name = "network-name-${local.name_suffix}"
}

data "google_project" "project" {}
7 changes: 7 additions & 0 deletions vertex_ai_index_endpoint_deployed_index_basic/motd
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
79 changes: 79 additions & 0 deletions vertex_ai_index_endpoint_deployed_index_basic/tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Vertex Ai Index Endpoint Deployed Index Basic - Terraform

## Setup

<walkthrough-author name="rileykarson@google.com" analyticsId="UA-125550242-1" tutorialName="vertex_ai_index_endpoint_deployed_index_basic" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
15 changes: 15 additions & 0 deletions vertex_ai_index_endpoint_deployed_index_basic_two/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
Loading