From 04cd9f04f1713b47f50c1931fd8955665ea8cbcc Mon Sep 17 00:00:00 2001 From: Jakub Michalak Date: Thu, 7 Nov 2024 11:51:14 +0100 Subject: [PATCH] feat: Rework config hierarchy (#3166) - add remaining fields to the schema - deprecate `account` - implement and use a helper function for matching provider versions in acceptance tests - use helpers to fill config values - add acceptance tests for all fields in the config - move some code to internal package - improve documentation: describe config hierarchy and provide better config file examples - improve and test sdk.MergeConfig - move mock helper to a separate package because it caused unnecessarily registered `sqlmock` driver in one of the tests ## Test Plan * [x] acceptance tests * [x] unit tests ## References https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/1881 https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/2145 https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/2925 https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/2983 https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3104 ## TODO - acceptance test for fields regarding private keys - will be done in SNOW-1754319 - unskip some tests after creating a compatible config for older versions --- MIGRATION_GUIDE.md | 25 + Makefile | 2 +- README.md | 20 +- docs/index.md | 203 +++++++-- examples/additional/provider_config_tf.MD | 35 ++ examples/additional/provider_config_toml.MD | 34 ++ examples/provider/provider.tf | 33 +- pkg/acceptance/helpers/random/certs.go | 41 +- pkg/acceptance/testing.go | 11 + .../testprofiles/testing_config_profiles.go | 2 + pkg/helpers/helpers.go | 14 - .../collections/collection_helpers.go | 14 + .../snowflake_environment_variables.go | 9 +- pkg/provider/provider.go | 431 ++++++++++-------- pkg/provider/provider_acceptance_test.go | 420 ++++++++++++++++- pkg/provider/provider_helpers.go | 177 +++---- pkg/provider/provider_helpers_test.go | 41 +- pkg/provider/testdata/config.toml | 42 ++ ...tegration_with_authorization_code_grant.go | 2 +- ...ion_integration_with_client_credentials.go | 2 +- ...hentication_integration_with_jwt_bearer.go | 2 +- pkg/resources/database.go | 3 +- pkg/resources/oauth_integration_test.go | 2 +- pkg/resources/saml_integration_test.go | 2 +- pkg/resources/schema.go | 2 +- pkg/resources/schema_parameters.go | 4 +- pkg/resources/secondary_database.go | 3 +- .../secret_with_basic_authentication.go | 3 +- pkg/resources/secret_with_generic_string.go | 3 +- ...ret_with_oauth_authorization_code_grant.go | 3 +- .../secret_with_oauth_client_credentials.go | 3 +- pkg/resources/shared_database.go | 3 +- pkg/resources/stream_on_directory_table.go | 3 +- pkg/resources/stream_on_external_table.go | 3 +- pkg/resources/stream_on_table.go | 3 +- pkg/resources/stream_on_view.go | 3 +- pkg/resources/user.go | 6 +- pkg/schemas/security_integration.go | 3 +- pkg/sdk/client.go | 5 - pkg/sdk/config.go | 365 ++++++++++++++- pkg/sdk/config_test.go | 316 ++++++++++--- pkg/sdk/internal/client/client_test.go | 15 - pkg/sdk/sweepers_test.go | 3 + pkg/testhelpers/helpers.go | 21 +- pkg/testhelpers/mock/mock.go | 25 + templates/index.md.tmpl | 78 +++- 46 files changed, 1890 insertions(+), 550 deletions(-) create mode 100644 examples/additional/provider_config_tf.MD create mode 100644 examples/additional/provider_config_toml.MD create mode 100644 pkg/provider/testdata/config.toml create mode 100644 pkg/testhelpers/mock/mock.go diff --git a/MIGRATION_GUIDE.md b/MIGRATION_GUIDE.md index eea774dd44..5a79d9cc50 100644 --- a/MIGRATION_GUIDE.md +++ b/MIGRATION_GUIDE.md @@ -46,6 +46,31 @@ Please adjust your Terraform configuration files. ### *(behavior change)* Provider configuration rework On our road to v1, we have decided to rework configuration to address the most common issues (see a [roadmap entry](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#providers-configuration-rework)). We have created a list of topics we wanted to address before v1. We will prepare an announcement soon. The following subsections describe the things addressed in the v0.98.0. +#### *(behavior change)* new fields +We have added new fields to match the ones in [the driver](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#Config) and to simplify setting account name. Specifically: +- `include_retry_reason`, `max_retry_count`, `driver_tracing`, `tmp_directory_path` and `disable_console_login` are the new fields that are supported in the driver +- `disable_saml_url_check` will be added to the provider after upgrading the driver +- `account_name` and `organization_name` were added to improve handling account names. Read more in [docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier#using-an-account-name-as-an-identifier). + +#### *(behavior change)* changed configuration of driver log level +To be more consistent with other configuration options, we have decided to add `driver_tracing` to the configuration schema. This value can also be configured by `SNOWFLAKE_DRIVER_TRACING` environmental variable and by `drivertracing` field in the TOML file. The previous `SF_TF_GOSNOWFLAKE_LOG_LEVEL` environmental variable is not supported now, and was removed from the provider. + +#### *(behavior change)* deprecated fields +Because of new fields `account_name` and `organization_name`, `account` is now deprecated. It will be removed before v1. Please adjust your configurations from +```terraform +provider "snowflake" { + account = "ORGANIZATION-ACCOUNT" +} +``` + +to +```terraform +provider "snowflake" { + organization_name = "ORGANIZATION" + account_name = "ACCOUNT" +} +``` + #### *(behavior change)* changed behavior of some fields For the fields that are not deprecated, we focused on improving validations and documentation. Also, we adjusted some fields to match our [driver's](https://github.com/snowflakedb/gosnowflake) defaults. Specifically: - Relaxed validations for enum fields like `protocol` and `authenticator`. Now, the case on such fields is ignored. diff --git a/Makefile b/Makefile index c8feb876b8..66435284af 100644 --- a/Makefile +++ b/Makefile @@ -74,7 +74,7 @@ test-architecture: ## check architecture constraints between packages go test ./pkg/architests/... -v test-client: ## runs test that checks sdk.Client without instrumentedsql - SF_TF_NO_INSTRUMENTED_SQL=1 SF_TF_GOSNOWFLAKE_LOG_LEVEL=debug go test ./pkg/sdk/internal/client/... -v + SF_TF_NO_INSTRUMENTED_SQL=1 go test ./pkg/sdk/internal/client/... -v test-object-renaming: ## runs tests in object_renaming_acceptance_test.go TEST_SF_TF_ENABLE_OBJECT_RENAMING=1 go test ./pkg/resources/object_renaming_acceptace_test.go -v diff --git a/README.md b/README.md index 391118f254..9136352ea3 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ Some links that might help you: - The [issues section](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues) might already have an issue addressing your question. ## Would you like to create an issue? -If you would like to create a GitHub issue, please read our [guide](./CREATING_ISSUES.md) first. +If you would like to create a GitHub issue, please read our [guide](./CREATING_ISSUES.md) first. It contains useful links, FAQ, and commonly known issues with solutions that may already solve your case. ## Additional debug logs for `snowflake_grant_privileges_to_role` resource @@ -89,17 +89,17 @@ Set environment variable `SF_TF_ADDITIONAL_DEBUG_LOGGING` to a non-empty value. ## Additional SQL Client configuration Currently underlying sql [gosnowflake](https://github.com/snowflakedb/gosnowflake) driver is wrapped with [instrumentedsql](https://github.com/luna-duclos/instrumentedsql). In order to use raw [gosnowflake](https://github.com/snowflakedb/gosnowflake) driver, set environment variable `SF_TF_NO_INSTRUMENTED_SQL` to a non-empty value. -By default, the underlying driver is set to error level logging. It can be changed by setting `SF_TF_GOSNOWFLAKE_LOG_LEVEL` to one of: -- `panic` -- `fatal` -- `error` -- `warn` -- `warning` -- `info` -- `debug` +By default, the underlying driver is set to error level logging. It can be changed by setting `driver_tracing` field in the configuration to one of (from most to least verbose): - `trace` +- `debug` +- `info` +- `print` +- `warning` +- `error` +- `fatal` +- `panic` -*note*: It's possible it will be one of the provider config parameters in the future provider versions. +Read more in [provider configuration docs](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs#schema). ## Contributing diff --git a/docs/index.md b/docs/index.md index cc619bb20a..b0e0a17a04 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,6 +17,8 @@ Coverage is focused on part of Snowflake related to access control. ## Example Provider Configuration +This is an example configuration of the provider in `main.tf` in a configuration directory. More examples are provided [below](#order-precedence). + ```terraform terraform { required_providers { @@ -26,31 +28,34 @@ terraform { } } +# A simple configuration of the provider with a default authentication. +# A default value for `authenticator` is `snowflake`, enabling authentication with `user` and `password`. provider "snowflake" { - account = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT env var - username = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var - password = "..." - authenticator = "..." # required if not using password as auth method - oauth_access_token = "..." - private_key_path = "..." - private_key = "..." - private_key_passphrase = "..." - oauth_refresh_token = "..." - oauth_client_id = "..." - oauth_client_secret = "..." - oauth_endpoint = "..." - oauth_redirect_url = "..." + organization_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ORGANIZATION_NAME env var + account_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT_NAME env var + user = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var + password = "..." // optional - region = "..." # required if using legacy format for account identifier role = "..." host = "..." warehouse = "..." - session_params = { + params = { query_tag = "..." } } +# A simple configuration of the provider with private key authentication. +provider "snowflake" { + organization_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ORGANIZATION_NAME env var + account_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT_NAME env var + user = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var + authenticator = "SNOWFLAKE_JWT" + private_key = "-----BEGIN ENCRYPTED PRIVATE KEY-----..." + private_key_passphrase = "passphrase" +} + +# By using the `profile` field, missing fields will be populated from ~/.snowflake/config TOML file provider "snowflake" { profile = "securityadmin" } @@ -65,22 +70,27 @@ provider "snowflake" { ### Optional -- `account` (String) Specifies your Snowflake account identifier assigned, by Snowflake. The [account locator](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-account-locator-in-a-region) format is not supported. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html). Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_ACCOUNT` environment variable. -- `authenticator` (String) Specifies the [authentication type](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#AuthType) to use when connecting to Snowflake. Valid values include: Snowflake, OAuth, ExternalBrowser, Okta, JWT, TokenAccessor, UsernamePasswordMFA. It has to be set explicitly to JWT for private key authentication. Can also be sourced from the `SNOWFLAKE_AUTHENTICATOR` environment variable. +- `account` (String, Deprecated) Use `account_name` and `organization_name` instead. Specifies your Snowflake account identifier assigned, by Snowflake. The [account locator](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-account-locator-in-a-region) format is not supported. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html). Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_ACCOUNT` environment variable. +- `account_name` (String) Specifies your Snowflake account name assigned by Snowflake. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier#account-name). Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_ACCOUNT_NAME` environment variable. +- `authenticator` (String) Specifies the [authentication type](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#AuthType) to use when connecting to Snowflake. Valid options are: `SNOWFLAKE` | `OAUTH` | `EXTERNALBROWSER` | `OKTA` | `JWT` | `SNOWFLAKE_JWT` | `TOKENACCESSOR` | `USERNAMEPASSWORDMFA`. Value `JWT` is deprecated and will be removed in future releases. Can also be sourced from the `SNOWFLAKE_AUTHENTICATOR` environment variable. - `browser_auth` (Boolean, Deprecated) Required when `oauth_refresh_token` is used. Can also be sourced from `SNOWFLAKE_USE_BROWSER_AUTH` environment variable. - `client_ip` (String) IP address for network checks. Can also be sourced from the `SNOWFLAKE_CLIENT_IP` environment variable. - `client_request_mfa_token` (String) When true the MFA token is cached in the credential manager. True by default in Windows/OSX. False for Linux. Can also be sourced from the `SNOWFLAKE_CLIENT_REQUEST_MFA_TOKEN` environment variable. - `client_store_temporary_credential` (String) When true the ID token is cached in the credential manager. True by default in Windows/OSX. False for Linux. Can also be sourced from the `SNOWFLAKE_CLIENT_STORE_TEMPORARY_CREDENTIAL` environment variable. - `client_timeout` (Number) The timeout in seconds for the client to complete the authentication. Can also be sourced from the `SNOWFLAKE_CLIENT_TIMEOUT` environment variable. +- `disable_console_login` (String) Indicates whether console login should be disabled in the driver. Can also be sourced from the `SNOWFLAKE_DISABLE_CONSOLE_LOGIN` environment variable. - `disable_query_context_cache` (Boolean) Disables HTAP query context cache in the driver. Can also be sourced from the `SNOWFLAKE_DISABLE_QUERY_CONTEXT_CACHE` environment variable. - `disable_telemetry` (Boolean) Disables telemetry in the driver. Can also be sourced from the `DISABLE_TELEMETRY` environment variable. +- `driver_tracing` (String) Specifies the logging level to be used by the driver. Valid options are: `trace` | `debug` | `info` | `print` | `warning` | `error` | `fatal` | `panic`. Can also be sourced from the `SNOWFLAKE_DRIVER_TRACING` environment variable. - `external_browser_timeout` (Number) The timeout in seconds for the external browser to complete the authentication. Can also be sourced from the `SNOWFLAKE_EXTERNAL_BROWSER_TIMEOUT` environment variable. - `host` (String) Specifies a custom host value used by the driver for privatelink connections. Can also be sourced from the `SNOWFLAKE_HOST` environment variable. +- `include_retry_reason` (String) Should retried request contain retry reason. Can also be sourced from the `SNOWFLAKE_INCLUDE_RETRY_REASON` environment variable. - `insecure_mode` (Boolean) If true, bypass the Online Certificate Status Protocol (OCSP) certificate revocation check. IMPORTANT: Change the default value for testing or emergency situations only. Can also be sourced from the `SNOWFLAKE_INSECURE_MODE` environment variable. - `jwt_client_timeout` (Number) The timeout in seconds for the JWT client to complete the authentication. Can also be sourced from the `SNOWFLAKE_JWT_CLIENT_TIMEOUT` environment variable. - `jwt_expire_timeout` (Number) JWT expire after timeout in seconds. Can also be sourced from the `SNOWFLAKE_JWT_EXPIRE_TIMEOUT` environment variable. - `keep_session_alive` (Boolean) Enables the session to persist even after the connection is closed. Can also be sourced from the `SNOWFLAKE_KEEP_SESSION_ALIVE` environment variable. - `login_timeout` (Number) Login retry timeout in seconds EXCLUDING network roundtrip and read out http response. Can also be sourced from the `SNOWFLAKE_LOGIN_TIMEOUT` environment variable. +- `max_retry_count` (Number) Specifies how many times non-periodic HTTP request can be retried by the driver. Can also be sourced from the `SNOWFLAKE_MAX_RETRY_COUNT` environment variable. - `oauth_access_token` (String, Sensitive, Deprecated) Token for use with OAuth. Generating the token is left to other tools. Cannot be used with `browser_auth`, `private_key_path`, `oauth_refresh_token` or `password`. Can also be sourced from `SNOWFLAKE_OAUTH_ACCESS_TOKEN` environment variable. - `oauth_client_id` (String, Sensitive, Deprecated) Required when `oauth_refresh_token` is used. Can also be sourced from `SNOWFLAKE_OAUTH_CLIENT_ID` environment variable. - `oauth_client_secret` (String, Sensitive, Deprecated) Required when `oauth_refresh_token` is used. Can also be sourced from `SNOWFLAKE_OAUTH_CLIENT_SECRET` environment variable. @@ -88,25 +98,27 @@ provider "snowflake" { - `oauth_redirect_url` (String, Sensitive, Deprecated) Required when `oauth_refresh_token` is used. Can also be sourced from `SNOWFLAKE_OAUTH_REDIRECT_URL` environment variable. - `oauth_refresh_token` (String, Sensitive, Deprecated) Token for use with OAuth. Setup and generation of the token is left to other tools. Should be used in conjunction with `oauth_client_id`, `oauth_client_secret`, `oauth_endpoint`, `oauth_redirect_url`. Cannot be used with `browser_auth`, `private_key_path`, `oauth_access_token` or `password`. Can also be sourced from `SNOWFLAKE_OAUTH_REFRESH_TOKEN` environment variable. - `ocsp_fail_open` (String) True represents OCSP fail open mode. False represents OCSP fail closed mode. Fail open true by default. Can also be sourced from the `SNOWFLAKE_OCSP_FAIL_OPEN` environment variable. -- `okta_url` (String) The URL of the Okta server. e.g. https://example.okta.com. Can also be sourced from the `SNOWFLAKE_OKTA_URL` environment variable. -- `params` (Map of String) Sets other connection (i.e. session) parameters. [Parameters](https://docs.snowflake.com/en/sql-reference/parameters) +- `okta_url` (String) The URL of the Okta server. e.g. https://example.okta.com. Okta URL host needs to to have a suffix `okta.com`. Read more in Snowflake [docs](https://docs.snowflake.com/en/user-guide/oauth-okta). Can also be sourced from the `SNOWFLAKE_OKTA_URL` environment variable. +- `organization_name` (String) Specifies your Snowflake organization name assigned by Snowflake. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier#organization-name). Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_ORGANIZATION_NAME` environment variable. +- `params` (Map of String) Sets other connection (i.e. session) parameters. [Parameters](https://docs.snowflake.com/en/sql-reference/parameters). This field can not be set with environmental variables. - `passcode` (String) Specifies the passcode provided by Duo when using multi-factor authentication (MFA) for login. Can also be sourced from the `SNOWFLAKE_PASSCODE` environment variable. - `passcode_in_password` (Boolean) False by default. Set to true if the MFA passcode is embedded to the configured password. Can also be sourced from the `SNOWFLAKE_PASSCODE_IN_PASSWORD` environment variable. -- `password` (String, Sensitive) Password for username+password auth. Cannot be used with `browser_auth` or `private_key_path`. Can also be sourced from the `SNOWFLAKE_PASSWORD` environment variable. +- `password` (String, Sensitive) Password for user + password auth. Cannot be used with `browser_auth` or `private_key_path`. Can also be sourced from the `SNOWFLAKE_PASSWORD` environment variable. - `port` (Number) Specifies a custom port value used by the driver for privatelink connections. Can also be sourced from the `SNOWFLAKE_PORT` environment variable. - `private_key` (String, Sensitive) Private Key for username+private-key auth. Cannot be used with `browser_auth` or `password`. Can also be sourced from the `SNOWFLAKE_PRIVATE_KEY` environment variable. - `private_key_passphrase` (String, Sensitive) Supports the encryption ciphers aes-128-cbc, aes-128-gcm, aes-192-cbc, aes-192-gcm, aes-256-cbc, aes-256-gcm, and des-ede3-cbc. Can also be sourced from the `SNOWFLAKE_PRIVATE_KEY_PASSPHRASE` environment variable. - `private_key_path` (String, Sensitive, Deprecated) Path to a private key for using keypair authentication. Cannot be used with `browser_auth`, `oauth_access_token` or `password`. Can also be sourced from `SNOWFLAKE_PRIVATE_KEY_PATH` environment variable. - `profile` (String) Sets the profile to read from ~/.snowflake/config file. Can also be sourced from the `SNOWFLAKE_PROFILE` environment variable. -- `protocol` (String) A protocol used in the connection. Valid options are: `HTTP` | `HTTPS`. Can also be sourced from the `SNOWFLAKE_PROTOCOL` environment variable. +- `protocol` (String) A protocol used in the connection. Valid options are: `http` | `https`. Can also be sourced from the `SNOWFLAKE_PROTOCOL` environment variable. - `region` (String, Deprecated) Snowflake region, such as "eu-central-1", with this parameter. However, since this parameter is deprecated, it is best to specify the region as part of the account parameter. For details, see the description of the account parameter. [Snowflake region](https://docs.snowflake.com/en/user-guide/intro-regions.html) to use. Required if using the [legacy format for the `account` identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html#format-2-legacy-account-locator-in-a-region) in the form of `.`. Can also be sourced from the `SNOWFLAKE_REGION` environment variable. - `request_timeout` (Number) request retry timeout in seconds EXCLUDING network roundtrip and read out http response. Can also be sourced from the `SNOWFLAKE_REQUEST_TIMEOUT` environment variable. - `role` (String) Specifies the role to use by default for accessing Snowflake objects in the client session. Can also be sourced from the `SNOWFLAKE_ROLE` environment variable. - `session_params` (Map of String, Deprecated) Sets session parameters. [Parameters](https://docs.snowflake.com/en/sql-reference/parameters) +- `tmp_directory_path` (String) Sets temporary directory used by the driver for operations like encrypting, compressing etc. Can also be sourced from the `SNOWFLAKE_TMP_DIRECTORY_PATH` environment variable. - `token` (String, Sensitive) Token to use for OAuth and other forms of token based auth. Can also be sourced from the `SNOWFLAKE_TOKEN` environment variable. - `token_accessor` (Block List, Max: 1) (see [below for nested schema](#nestedblock--token_accessor)) - `user` (String) Username. Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_USER` environment variable. -- `username` (String, Deprecated) Username for username+password authentication. Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_USERNAME` environment variable. +- `username` (String, Deprecated) Username for user + password authentication. Required unless using `profile`. Can also be sourced from the `SNOWFLAKE_USERNAME` environment variable. - `validate_default_parameters` (String) True by default. If false, disables the validation checks for Database, Schema, Warehouse and Role at the time a connection is established. Can also be sourced from the `SNOWFLAKE_VALIDATE_DEFAULT_PARAMETERS` environment variable. - `warehouse` (String) Specifies the virtual warehouse to use by default for queries, loading, etc. in the client session. Can also be sourced from the `SNOWFLAKE_WAREHOUSE` environment variable. @@ -132,7 +144,7 @@ The Snowflake provider support multiple ways to authenticate: * Private Key * Config File -In all cases account and username are required. +In all cases `organization_name`, `account_name` and `user` are required. ### Keypair Authentication Environment Variables @@ -209,30 +221,143 @@ export SNOWFLAKE_USER='...' export SNOWFLAKE_PASSWORD='...' ``` -### Config File +## Order Precedence -If you choose to use a config file, the optional `profile` attribute specifies the profile to use from the config file. If no profile is specified, the default profile is used. The Snowflake config file lives at `~/.snowflake/config` and uses [TOML](https://toml.io/) format. You can override this location by setting the `SNOWFLAKE_CONFIG_PATH` environment variable. If no username and account are specified, the provider will fall back to reading the config file. +Currently, the provider can be configured in three ways: +1. In a Terraform file located in the Terraform module with other resources. -```shell +Example content of the Terraform file configuration: + +```terraform +provider "snowflake" { + organization_name = "..." + account_name = "..." + username = "..." + password = "..." +} +``` + +2. In environmental variables (envs). This is mainly used to provide sensitive values. + + +```bash +export SNOWFLAKE_USER="..." +export SNOWFLAKE_PRIVATE_KEY_PATH="~/.ssh/snowflake_key" +``` + +3. In a TOML file (default in ~/.snowflake/config). Notice the use of different profiles. The profile name needs to be specified in the Terraform configuration file in `profile` field. When this is not specified, `default` profile is loaded. +When a `default` profile is not present in the TOML file, it is treated as "empty", without failing. + +Example content of the Terraform file configuration: + +```terraform +provider "snowflake" { + profile = "default" +} +``` + +Example content of the TOML file configuration: + +```toml [default] -account='TESTACCOUNT' -user='TEST_USER' -password='hunter2' +organizationname='organization_name' +accountname='account_name' +user='user' +password='password' role='ACCOUNTADMIN' -[securityadmin] -account='TESTACCOUNT' -user='TEST_USER' -password='hunter2' -role='SECURITYADMIN' +[secondary_test_account] +organizationname='organization_name' +accountname='account2_name' +user='user' +password='password' +role='ACCOUNTADMIN' ``` -## Order Precedence +Not all fields must be configured in one source; users can choose which fields are configured in which source. +Provider uses an established hierarchy of sources. The current behavior is that for each field: +1. Check if it is present in the provider configuration. If yes, use this value. If not, go to step 2. +1. Check if it is present in the environment variables. If yes, use this value. If not, go to step 3. +1. Check if it is present in the TOML config file (specifically, use the profile name configured in one of the steps above). If yes, use this value. If not, the value is considered empty. + +An example TOML file contents: + +```toml +[example] +accountname = 'account_name' +organizationname = 'organization_name' +user = 'user' +password = 'password' +warehouse = 'SNOWFLAKE' +role = 'ACCOUNTADMIN' +clientip = '1.2.3.4' +protocol = 'https' +port = 443 +oktaurl = 'https://example.com' +clienttimeout = 10 +jwtclienttimeout = 20 +logintimeout = 30 +requesttimeout = 40 +jwtexpiretimeout = 50 +externalbrowsertimeout = 60 +maxretrycount = 1 +authenticator = 'snowflake' +insecuremode = true +ocspfailopen = true +keepsessionalive = true +disabletelemetry = true +validatedefaultparameters = true +clientrequestmfatoken = true +clientstoretemporarycredential = true +tracing = 'info' +tmpdirpath = '/tmp/terraform-provider/' +disablequerycontextcache = true +includeretryreason = true +disableconsolelogin = true + +[example.params] +param_key = 'param_value' +``` + +An example terraform configuration file equivalent: -The Snowflake provider will use the following order of precedence when determining which credentials to use: -1) Provider Configuration -2) Environment Variables -3) Config File +```terraform +provider "snowflake" { + organization_name = "organization_name" + account_name = "account_name" + user = "user" + password = "password" + warehouse = "SNOWFLAKE" + protocol = "https" + port = "443" + role = "ACCOUNTADMIN" + validate_default_parameters = true + client_ip = "1.2.3.4" + authenticator = "snowflake" + okta_url = "https://example.com" + login_timeout = 10 + request_timeout = 20 + jwt_expire_timeout = 30 + client_timeout = 40 + jwt_client_timeout = 50 + external_browser_timeout = 60 + insecure_mode = true + ocsp_fail_open = true + keep_session_alive = true + disable_telemetry = true + client_request_mfa_token = true + client_store_temporary_credential = true + disable_query_context_cache = true + include_retry_reason = true + max_retry_count = 3 + driver_tracing = "info" + tmp_directory_path = "/tmp/terraform-provider/" + disable_console_login = true + params = { + param_key = "param_value" + } +} +``` ## Currently deprecated resources diff --git a/examples/additional/provider_config_tf.MD b/examples/additional/provider_config_tf.MD new file mode 100644 index 0000000000..2f1377939c --- /dev/null +++ b/examples/additional/provider_config_tf.MD @@ -0,0 +1,35 @@ +provider "snowflake" { + organization_name = "organization_name" + account_name = "account_name" + user = "user" + password = "password" + warehouse = "SNOWFLAKE" + protocol = "https" + port = "443" + role = "ACCOUNTADMIN" + validate_default_parameters = true + client_ip = "1.2.3.4" + authenticator = "snowflake" + okta_url = "https://example.com" + login_timeout = 10 + request_timeout = 20 + jwt_expire_timeout = 30 + client_timeout = 40 + jwt_client_timeout = 50 + external_browser_timeout = 60 + insecure_mode = true + ocsp_fail_open = true + keep_session_alive = true + disable_telemetry = true + client_request_mfa_token = true + client_store_temporary_credential = true + disable_query_context_cache = true + include_retry_reason = true + max_retry_count = 3 + driver_tracing = "info" + tmp_directory_path = "/tmp/terraform-provider/" + disable_console_login = true + params = { + param_key = "param_value" + } +} diff --git a/examples/additional/provider_config_toml.MD b/examples/additional/provider_config_toml.MD new file mode 100644 index 0000000000..e0f4039014 --- /dev/null +++ b/examples/additional/provider_config_toml.MD @@ -0,0 +1,34 @@ +[example] +accountname = 'account_name' +organizationname = 'organization_name' +user = 'user' +password = 'password' +warehouse = 'SNOWFLAKE' +role = 'ACCOUNTADMIN' +clientip = '1.2.3.4' +protocol = 'https' +port = 443 +oktaurl = 'https://example.com' +clienttimeout = 10 +jwtclienttimeout = 20 +logintimeout = 30 +requesttimeout = 40 +jwtexpiretimeout = 50 +externalbrowsertimeout = 60 +maxretrycount = 1 +authenticator = 'snowflake' +insecuremode = true +ocspfailopen = true +keepsessionalive = true +disabletelemetry = true +validatedefaultparameters = true +clientrequestmfatoken = true +clientstoretemporarycredential = true +tracing = 'info' +tmpdirpath = '/tmp/terraform-provider/' +disablequerycontextcache = true +includeretryreason = true +disableconsolelogin = true + +[example.params] +param_key = 'param_value' diff --git a/examples/provider/provider.tf b/examples/provider/provider.tf index d7452fec32..0264c069ab 100644 --- a/examples/provider/provider.tf +++ b/examples/provider/provider.tf @@ -6,31 +6,34 @@ terraform { } } +# A simple configuration of the provider with a default authentication. +# A default value for `authenticator` is `snowflake`, enabling authentication with `user` and `password`. provider "snowflake" { - account = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT env var - username = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var - password = "..." - authenticator = "..." # required if not using password as auth method - oauth_access_token = "..." - private_key_path = "..." - private_key = "..." - private_key_passphrase = "..." - oauth_refresh_token = "..." - oauth_client_id = "..." - oauth_client_secret = "..." - oauth_endpoint = "..." - oauth_redirect_url = "..." + organization_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ORGANIZATION_NAME env var + account_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT_NAME env var + user = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var + password = "..." // optional - region = "..." # required if using legacy format for account identifier role = "..." host = "..." warehouse = "..." - session_params = { + params = { query_tag = "..." } } +# A simple configuration of the provider with private key authentication. +provider "snowflake" { + organization_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ORGANIZATION_NAME env var + account_name = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT_NAME env var + user = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var + authenticator = "SNOWFLAKE_JWT" + private_key = "-----BEGIN ENCRYPTED PRIVATE KEY-----..." + private_key_passphrase = "passphrase" +} + +# By using the `profile` field, missing fields will be populated from ~/.snowflake/config TOML file provider "snowflake" { profile = "securityadmin" } diff --git a/pkg/acceptance/helpers/random/certs.go b/pkg/acceptance/helpers/random/certs.go index eb0cf4aaf9..e8a7697d70 100644 --- a/pkg/acceptance/helpers/random/certs.go +++ b/pkg/acceptance/helpers/random/certs.go @@ -2,6 +2,7 @@ package random import ( "bytes" + "crypto" "crypto/rand" "crypto/rsa" "crypto/sha256" @@ -16,6 +17,7 @@ import ( "time" "github.com/stretchr/testify/require" + "github.com/youmark/pkcs8" ) // GenerateX509 returns base64 encoded certificate on a single line without the leading -----BEGIN CERTIFICATE----- and ending -----END CERTIFICATE----- markers. @@ -43,8 +45,7 @@ func GenerateX509(t *testing.T) string { // GenerateRSAPublicKey returns an RSA public key without BEGIN and END markers, and key's hash. func GenerateRSAPublicKey(t *testing.T) (string, string) { t.Helper() - key, err := rsa.GenerateKey(rand.Reader, 2048) - require.NoError(t, err) + key := GenerateRSAPrivateKey(t) pub := key.Public() b, err := x509.MarshalPKIXPublicKey(pub.(*rsa.PublicKey)) @@ -52,6 +53,42 @@ func GenerateRSAPublicKey(t *testing.T) (string, string) { return encode(t, "RSA PUBLIC KEY", b), hash(t, b) } +// GenerateRSAPrivateKey returns an RSA private key. +func GenerateRSAPrivateKey(t *testing.T) *rsa.PrivateKey { + t.Helper() + key, err := rsa.GenerateKey(rand.Reader, 2048) + require.NoError(t, err) + return key +} + +// GenerateRSAPrivateKeyEncrypted returns a PEM-encoded pair of unencrypted and encrypted key with a given password +func GenerateRSAPrivateKeyEncrypted(t *testing.T, password string) (unencrypted, encrypted string) { + t.Helper() + rsaPrivateKey := GenerateRSAPrivateKey(t) + unencryptedDer, err := x509.MarshalPKCS8PrivateKey(rsaPrivateKey) + require.NoError(t, err) + privBlock := pem.Block{ + Type: "PRIVATE KEY", + Bytes: unencryptedDer, + } + unencrypted = string(pem.EncodeToMemory(&privBlock)) + + encryptedDer, err := pkcs8.MarshalPrivateKey(rsaPrivateKey, []byte(password), &pkcs8.Opts{ + Cipher: pkcs8.AES256CBC, + KDFOpts: pkcs8.PBKDF2Opts{ + SaltSize: 16, IterationCount: 2000, HMACHash: crypto.SHA256, + }, + }) + require.NoError(t, err) + privEncryptedBlock := pem.Block{ + Type: "ENCRYPTED PRIVATE KEY", + Bytes: encryptedDer, + } + encrypted = string(pem.EncodeToMemory(&privEncryptedBlock)) + + return +} + func hash(t *testing.T, b []byte) string { t.Helper() hash := sha256.Sum256(b) diff --git a/pkg/acceptance/testing.go b/pkg/acceptance/testing.go index 79d52bd35f..bd1c6d299f 100644 --- a/pkg/acceptance/testing.go +++ b/pkg/acceptance/testing.go @@ -21,6 +21,7 @@ import ( "github.com/hashicorp/terraform-plugin-mux/tf5to6server" "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" "github.com/hashicorp/terraform-plugin-testing/config" + "github.com/hashicorp/terraform-plugin-testing/helper/resource" "github.com/snowflakedb/gosnowflake" ) @@ -197,3 +198,13 @@ func TestClient() *helpers.TestClient { func SecondaryTestClient() *helpers.TestClient { return atc.secondaryTestClient } + +// ExternalProviderWithExactVersion returns a map of external providers with an exact version constraint +func ExternalProviderWithExactVersion(version string) map[string]resource.ExternalProvider { + return map[string]resource.ExternalProvider{ + "snowflake": { + VersionConstraint: fmt.Sprintf("=%s", version), + Source: "Snowflake-Labs/snowflake", + }, + } +} diff --git a/pkg/acceptance/testprofiles/testing_config_profiles.go b/pkg/acceptance/testprofiles/testing_config_profiles.go index 6bca3093c1..82770d7381 100644 --- a/pkg/acceptance/testprofiles/testing_config_profiles.go +++ b/pkg/acceptance/testprofiles/testing_config_profiles.go @@ -6,4 +6,6 @@ const ( Third = "third_test_account" Fourth = "fourth_test_account" IncorrectUserAndPassword = "incorrect_test_profile" + CompleteFields = "complete_fields" + CompleteFieldsInvalid = "complete_fields_invalid" ) diff --git a/pkg/helpers/helpers.go b/pkg/helpers/helpers.go index bde76ff381..6b3e39d4cf 100644 --- a/pkg/helpers/helpers.go +++ b/pkg/helpers/helpers.go @@ -142,20 +142,6 @@ func DecodeSnowflakeAccountIdentifier(identifier string) (sdk.AccountIdentifier, } } -// TODO(SNOW-1479870): Test -// MergeMaps takes any number of maps (of the same type) and concatenates them. -// In case of key collision, the value will be selected from the map that is provided -// later in the src function parameter. -func MergeMaps[M ~map[K]V, K comparable, V any](src ...M) M { - merged := make(M) - for _, m := range src { - for k, v := range m { - merged[k] = v - } - } - return merged -} - // TODO: use slices.Concat in Go 1.22 func ConcatSlices[T any](slices ...[]T) []T { var tmp []T diff --git a/pkg/internal/collections/collection_helpers.go b/pkg/internal/collections/collection_helpers.go index b71c44d62d..80375cfba0 100644 --- a/pkg/internal/collections/collection_helpers.go +++ b/pkg/internal/collections/collection_helpers.go @@ -22,3 +22,17 @@ func Map[T any, R any](collection []T, mapper func(T) R) []R { } return result } + +// TODO(SNOW-1479870): Test +// MergeMaps takes any number of maps (of the same type) and concatenates them. +// In case of key collision, the value will be selected from the map that is provided +// later in the src function parameter. +func MergeMaps[M ~map[K]V, K comparable, V any](src ...M) M { + merged := make(M) + for _, m := range src { + for k, v := range m { + merged[k] = v + } + } + return merged +} diff --git a/pkg/internal/snowflakeenvs/snowflake_environment_variables.go b/pkg/internal/snowflakeenvs/snowflake_environment_variables.go index e8443e8fdd..91f817c655 100644 --- a/pkg/internal/snowflakeenvs/snowflake_environment_variables.go +++ b/pkg/internal/snowflakeenvs/snowflake_environment_variables.go @@ -2,6 +2,8 @@ package snowflakeenvs const ( Account = "SNOWFLAKE_ACCOUNT" + AccountName = "SNOWFLAKE_ACCOUNT_NAME" + OrganizationName = "SNOWFLAKE_ORGANIZATION_NAME" User = "SNOWFLAKE_USER" Username = "SNOWFLAKE_USERNAME" Password = "SNOWFLAKE_PASSWORD" @@ -41,9 +43,12 @@ const ( DisableQueryContextCache = "SNOWFLAKE_DISABLE_QUERY_CONTEXT_CACHE" IncludeRetryReason = "SNOWFLAKE_INCLUDE_RETRY_REASON" Profile = "SNOWFLAKE_PROFILE" + MaxRetryCount = "SNOWFLAKE_MAX_RETRY_COUNT" + DriverTracing = "SNOWFLAKE_DRIVER_TRACING" + TmpDirectoryPath = "SNOWFLAKE_TMP_DIRECTORY_PATH" + DisableConsoleLogin = "SNOWFLAKE_DISABLE_CONSOLE_LOGIN" ConfigPath = "SNOWFLAKE_CONFIG_PATH" - NoInstrumentedSql = "SF_TF_NO_INSTRUMENTED_SQL" - GosnowflakeLogLevel = "SF_TF_GOSNOWFLAKE_LOG_LEVEL" + NoInstrumentedSql = "SF_TF_NO_INSTRUMENTED_SQL" ) diff --git a/pkg/provider/provider.go b/pkg/provider/provider.go index 0721189115..725879873a 100644 --- a/pkg/provider/provider.go +++ b/pkg/provider/provider.go @@ -8,10 +8,8 @@ import ( "net/url" "os" "strings" - "time" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/testenvs" - "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/datasources" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider/docs" @@ -22,6 +20,7 @@ import ( "github.com/hashicorp/terraform-plugin-sdk/v2/diag" "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" "github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation" + "github.com/hashicorp/terraform-plugin-testing/helper/resource" "github.com/snowflakedb/gosnowflake" ) @@ -49,11 +48,19 @@ func init() { func Provider() *schema.Provider { return &schema.Provider{ Schema: map[string]*schema.Schema{ - "account": { - Type: schema.TypeString, - Description: envNameFieldDescription("Specifies your Snowflake account identifier assigned, by Snowflake. The [account locator](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-account-locator-in-a-region) format is not supported. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html). Required unless using `profile`.", snowflakeenvs.Account), - Optional: true, - DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Account, nil), + "account_name": { + Type: schema.TypeString, + Description: envNameFieldDescription("Specifies your Snowflake account name assigned by Snowflake. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier#account-name). Required unless using `profile`.", snowflakeenvs.AccountName), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.AccountName, nil), + RequiredWith: []string{"account_name", "organization_name"}, + }, + "organization_name": { + Type: schema.TypeString, + Description: envNameFieldDescription("Specifies your Snowflake organization name assigned by Snowflake. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier#organization-name). Required unless using `profile`.", snowflakeenvs.OrganizationName), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.OrganizationName, nil), + RequiredWith: []string{"account_name", "organization_name"}, }, "user": { Type: schema.TypeString, @@ -64,7 +71,7 @@ func Provider() *schema.Provider { }, "password": { Type: schema.TypeString, - Description: envNameFieldDescription("Password for username+password auth. Cannot be used with `browser_auth` or `private_key_path`.", snowflakeenvs.Password), + Description: envNameFieldDescription("Password for user + password auth. Cannot be used with `browser_auth` or `private_key_path`.", snowflakeenvs.Password), Optional: true, Sensitive: true, DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Password, nil), @@ -91,9 +98,10 @@ func Provider() *schema.Provider { ValidateDiagFunc: validators.ValidateBooleanStringWithDefault, DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.ValidateDefaultParameters, provider.BooleanDefault), }, + // TODO(SNOW-999056): optionally rename to session_params "params": { Type: schema.TypeMap, - Description: "Sets other connection (i.e. session) parameters. [Parameters](https://docs.snowflake.com/en/sql-reference/parameters)", + Description: "Sets other connection (i.e. session) parameters. [Parameters](https://docs.snowflake.com/en/sql-reference/parameters). This field can not be set with environmental variables.", Optional: true, }, "client_ip": { @@ -125,10 +133,10 @@ func Provider() *schema.Provider { }, "authenticator": { Type: schema.TypeString, - Description: envNameFieldDescription("Specifies the [authentication type](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#AuthType) to use when connecting to Snowflake. Valid values include: Snowflake, OAuth, ExternalBrowser, Okta, JWT, TokenAccessor, UsernamePasswordMFA. It has to be set explicitly to JWT for private key authentication.", snowflakeenvs.Authenticator), + Description: envNameFieldDescription(fmt.Sprintf("Specifies the [authentication type](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#AuthType) to use when connecting to Snowflake. Valid options are: %v. Value `JWT` is deprecated and will be removed in future releases.", docs.PossibleValuesListed(sdk.AllAuthenticationTypes)), snowflakeenvs.Authenticator), Optional: true, - DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Authenticator, string(authenticationTypeSnowflake)), - ValidateDiagFunc: validators.NormalizeValidation(toAuthenticatorType), + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Authenticator, string(sdk.AuthenticationTypeSnowflake)), + ValidateDiagFunc: validators.NormalizeValidation(sdk.ToAuthenticatorType), }, "passcode": { Type: schema.TypeString, @@ -146,7 +154,7 @@ func Provider() *schema.Provider { }, "okta_url": { Type: schema.TypeString, - Description: envNameFieldDescription("The URL of the Okta server. e.g. https://example.okta.com.", snowflakeenvs.OktaUrl), + Description: envNameFieldDescription("The URL of the Okta server. e.g. https://example.okta.com. Okta URL host needs to to have a suffix `okta.com`. Read more in Snowflake [docs](https://docs.snowflake.com/en/user-guide/oauth-okta).", snowflakeenvs.OktaUrl), Optional: true, DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.OktaUrl, nil), ValidateDiagFunc: validation.ToDiagFunc(validation.IsURLWithHTTPorHTTPS), @@ -306,6 +314,41 @@ func Provider() *schema.Provider { Optional: true, DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.DisableQueryContextCache, nil), }, + "include_retry_reason": { + Type: schema.TypeString, + Description: envNameFieldDescription("Should retried request contain retry reason.", snowflakeenvs.IncludeRetryReason), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.IncludeRetryReason, resources.BooleanDefault), + ValidateDiagFunc: validators.ValidateBooleanStringWithDefault, + }, + "max_retry_count": { + Type: schema.TypeInt, + Description: envNameFieldDescription("Specifies how many times non-periodic HTTP request can be retried by the driver.", snowflakeenvs.MaxRetryCount), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.MaxRetryCount, nil), + ValidateDiagFunc: validation.ToDiagFunc(validation.IntAtLeast(0)), + }, + "driver_tracing": { + Type: schema.TypeString, + Description: envNameFieldDescription(fmt.Sprintf("Specifies the logging level to be used by the driver. Valid options are: %v.", docs.PossibleValuesListed(allDriverLogLevels)), snowflakeenvs.DriverTracing), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.DriverTracing, nil), + ValidateDiagFunc: validators.NormalizeValidation(toDriverLogLevel), + }, + "tmp_directory_path": { + Type: schema.TypeString, + Description: envNameFieldDescription("Sets temporary directory used by the driver for operations like encrypting, compressing etc.", snowflakeenvs.TmpDirectoryPath), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.TmpDirectoryPath, nil), + }, + "disable_console_login": { + Type: schema.TypeString, + Description: envNameFieldDescription("Indicates whether console login should be disabled in the driver.", snowflakeenvs.DisableConsoleLogin), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.DisableConsoleLogin, resources.BooleanDefault), + ValidateDiagFunc: validators.ValidateBooleanStringWithDefault, + }, + // TODO(SNOW-1761318): handle DisableSamlURLCheck after upgrading the driver to at least 1.10.1 "profile": { Type: schema.TypeString, // TODO(SNOW-1754364): Note that a default file path is already filled on sdk side. @@ -314,9 +357,16 @@ func Provider() *schema.Provider { DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Profile, "default"), }, // Deprecated attributes + "account": { + Type: schema.TypeString, + Description: envNameFieldDescription("Use `account_name` and `organization_name` instead. Specifies your Snowflake account identifier assigned, by Snowflake. The [account locator](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-account-locator-in-a-region) format is not supported. For information about account identifiers, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html). Required unless using `profile`.", snowflakeenvs.Account), + Optional: true, + DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Account, nil), + Deprecated: "Use `account_name` and `organization_name` instead of `account`", + }, "username": { Type: schema.TypeString, - Description: envNameFieldDescription("Username for username+password authentication. Required unless using `profile`.", snowflakeenvs.Username), + Description: envNameFieldDescription("Username for user + password authentication. Required unless using `profile`.", snowflakeenvs.Username), Optional: true, DefaultFunc: schema.EnvDefaultFunc(snowflakeenvs.Username, nil), Deprecated: "Use `user` instead of `username`", @@ -581,49 +631,185 @@ func ConfigureProvider(ctx context.Context, s *schema.ResourceData) (any, diag.D } } - config := &gosnowflake.Config{ - Application: "terraform-provider-snowflake", + config, err := getDriverConfigFromTerraform(s) + if err != nil { + return nil, diag.FromErr(err) } - if v, ok := s.GetOk("account"); ok && v.(string) != "" { - config.Account = v.(string) + if v, ok := s.GetOk("profile"); ok && v.(string) != "" { + tomlConfig, err := getDriverConfigFromTOML(v.(string)) + if err != nil { + return nil, diag.FromErr(err) + } + config = sdk.MergeConfig(config, tomlConfig) } - // backwards compatibility until we can remove this - if v, ok := s.GetOk("username"); ok && v.(string) != "" { - config.User = v.(string) + client, clientErr := sdk.NewClient(config) + + // needed for tests verifying different provider setups + if os.Getenv(resource.EnvTfAcc) != "" && os.Getenv(string(testenvs.ConfigureClientOnce)) == "true" { + configuredClient = client + configureClientError = clientErr + } else { + configuredClient = nil + configureClientError = nil } - if v, ok := s.GetOk("user"); ok && v.(string) != "" { - config.User = v.(string) + if clientErr != nil { + return nil, diag.FromErr(clientErr) } - if v, ok := s.GetOk("password"); ok && v.(string) != "" { - config.Password = v.(string) + return &provider.Context{Client: client}, nil +} + +func getDriverConfigFromTOML(profile string) (*gosnowflake.Config, error) { + if profile == "default" { + return sdk.DefaultConfig(), nil + } + path, err := sdk.GetConfigFileName() + if err != nil { + return nil, err } - if v, ok := s.GetOk("warehouse"); ok && v.(string) != "" { - config.Warehouse = v.(string) + profileConfig, err := sdk.ProfileConfig(profile) + if err != nil { + return nil, fmt.Errorf(`could not retrieve "%s" profile config from file %s: %w`, profile, path, err) + } + if profileConfig == nil { + return nil, fmt.Errorf(`profile "%s" not found in file %s`, profile, path) } + return profileConfig, nil +} - if v, ok := s.GetOk("role"); ok && v.(string) != "" { - config.Role = v.(string) +func getDriverConfigFromTerraform(s *schema.ResourceData) (*gosnowflake.Config, error) { + config := &gosnowflake.Config{ + Application: "terraform-provider-snowflake", } - if v, ok := s.GetOk("region"); ok && v.(string) != "" { - config.Region = v.(string) + err := errors.Join( + // account_name and organization_name are handled below + handleStringField(s, "user", &config.User), + handleStringField(s, "password", &config.Password), + handleStringField(s, "warehouse", &config.Warehouse), + handleStringField(s, "role", &config.Role), + handleBooleanStringAttribute(s, "validate_default_parameters", &config.ValidateDefaultParameters), + // params are handled below + // client ip + func() error { + if v, ok := s.GetOk("client_ip"); ok && v.(string) != "" { + config.ClientIP = net.ParseIP(v.(string)) + } + return nil + }(), + // protocol + func() error { + if v, ok := s.GetOk("protocol"); ok && v.(string) != "" { + protocol, err := toProtocol(v.(string)) + if err != nil { + return err + } + config.Protocol = string(protocol) + } + return nil + }(), + handleStringField(s, "host", &config.Host), + handleIntAttribute(s, "port", &config.Port), + // authenticator + func() error { + if v, ok := s.GetOk("authenticator"); ok && v.(string) != "" { + authType, err := sdk.ToAuthenticatorType(v.(string)) + if err != nil { + return err + } + config.Authenticator = authType + } + return nil + }(), + handleStringField(s, "passcode", &config.Passcode), + handleBoolField(s, "passcode_in_password", &config.PasscodeInPassword), + // okta url + func() error { + if v, ok := s.GetOk("okta_url"); ok && v.(string) != "" { + oktaURL, err := url.Parse(v.(string)) + if err != nil { + return fmt.Errorf("could not parse okta_url err = %w", err) + } + config.OktaURL = oktaURL + } + return nil + }(), + handleDurationInSecondsAttribute(s, "login_timeout", &config.LoginTimeout), + handleDurationInSecondsAttribute(s, "request_timeout", &config.RequestTimeout), + handleDurationInSecondsAttribute(s, "jwt_expire_timeout", &config.JWTExpireTimeout), + handleDurationInSecondsAttribute(s, "client_timeout", &config.ClientTimeout), + handleDurationInSecondsAttribute(s, "jwt_client_timeout", &config.JWTClientTimeout), + handleDurationInSecondsAttribute(s, "external_browser_timeout", &config.ExternalBrowserTimeout), + handleBoolField(s, "insecure_mode", &config.InsecureMode), + // ocsp fail open + func() error { + if v := s.Get("ocsp_fail_open").(string); v != provider.BooleanDefault { + parsed, err := provider.BooleanStringToBool(v) + if err != nil { + return err + } + if parsed { + config.OCSPFailOpen = gosnowflake.OCSPFailOpenTrue + } else { + config.OCSPFailOpen = gosnowflake.OCSPFailOpenFalse + } + } + return nil + }(), + // token + func() error { + if v, ok := s.GetOk("token"); ok && v.(string) != "" { + config.Token = v.(string) + config.Authenticator = gosnowflake.AuthTypeOAuth + } + return nil + }(), + // token accessor is handled below + handleBoolField(s, "keep_session_alive", &config.KeepSessionAlive), + // private key and private key passphrase are handled below + handleBoolField(s, "disable_telemetry", &config.DisableTelemetry), + handleBooleanStringAttribute(s, "client_request_mfa_token", &config.ClientRequestMfaToken), + handleBooleanStringAttribute(s, "client_store_temporary_credential", &config.ClientStoreTemporaryCredential), + handleBoolField(s, "disable_query_context_cache", &config.DisableQueryContextCache), + handleBooleanStringAttribute(s, "include_retry_reason", &config.IncludeRetryReason), + handleIntAttribute(s, "max_retry_count", &config.MaxRetryCount), + // driver tracing + func() error { + if v, ok := s.GetOk("driver_tracing"); ok { + driverLogLevel, err := toDriverLogLevel(v.(string)) + if err != nil { + return err + } + config.Tracing = string(driverLogLevel) + } + return nil + }(), + handleStringField(s, "tmp_directory_path", &config.TmpDirPath), + handleBooleanStringAttribute(s, "disable_console_login", &config.DisableConsoleLogin), + // profile is handled in the calling function + // TODO(SNOW-1761318): handle DisableSamlURLCheck after upgrading the driver to at least 1.10.1 + + // deprecated + handleStringField(s, "account", &config.Account), + handleStringField(s, "username", &config.User), + handleStringField(s, "region", &config.Region), + // session params are handled below + // browser auth is handled below + // private key path is handled below + ) + if err != nil { + return nil, err } - if v := s.Get("validate_default_parameters").(string); v != provider.BooleanDefault { - parsed, err := provider.BooleanStringToBool(v) - if err != nil { - return nil, diag.FromErr(err) - } - if parsed { - config.ValidateDefaultParameters = gosnowflake.ConfigBoolTrue - } else { - config.ValidateDefaultParameters = gosnowflake.ConfigBoolFalse - } + // account_name and organization_name override legacy account field + accountName := s.Get("account_name").(string) + organizationName := s.Get("organization_name").(string) + if accountName != "" && organizationName != "" { + config.Account = strings.Join([]string{organizationName, accountName}, "-") } m := make(map[string]interface{}) @@ -643,98 +829,14 @@ func ConfigureProvider(ctx context.Context, s *schema.ResourceData) (any, diag.D } config.Params = params - if v, ok := s.GetOk("client_ip"); ok && v.(string) != "" { - config.ClientIP = net.ParseIP(v.(string)) - } - - if v, ok := s.GetOk("protocol"); ok && v.(string) != "" { - config.Protocol = v.(string) - } - - if v, ok := s.GetOk("host"); ok && v.(string) != "" { - config.Host = v.(string) - } - - if v, ok := s.GetOk("port"); ok && v.(int) > 0 { - config.Port = v.(int) - } - // backwards compatibility until we can remove this if v, ok := s.GetOk("browser_auth"); ok && v.(bool) { config.Authenticator = gosnowflake.AuthTypeExternalBrowser } - if v, ok := s.GetOk("authenticator"); ok && v.(string) != "" { - authType, err := toAuthenticatorType(v.(string)) - if err != nil { - return "", diag.FromErr(err) - } - config.Authenticator = authType - } - - if v, ok := s.GetOk("passcode"); ok && v.(string) != "" { - config.Passcode = v.(string) - } - - if v, ok := s.GetOk("passcode_in_password"); ok && v.(bool) { - config.PasscodeInPassword = v.(bool) - } - if v, ok := s.GetOk("okta_url"); ok && v.(string) != "" { - oktaURL, err := url.Parse(v.(string)) - if err != nil { - return nil, diag.FromErr(fmt.Errorf("could not parse okta_url err = %w", err)) - } - config.OktaURL = oktaURL - } - - if v, ok := s.GetOk("login_timeout"); ok && v.(int) > 0 { - config.LoginTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("request_timeout"); ok && v.(int) > 0 { - config.RequestTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("jwt_expire_timeout"); ok && v.(int) > 0 { - config.JWTExpireTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("client_timeout"); ok && v.(int) > 0 { - config.ClientTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("jwt_client_timeout"); ok && v.(int) > 0 { - config.JWTClientTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("external_browser_timeout"); ok && v.(int) > 0 { - config.ExternalBrowserTimeout = time.Second * time.Duration(int64(v.(int))) - } - - if v, ok := s.GetOk("insecure_mode"); ok && v.(bool) { - config.InsecureMode = v.(bool) - } - - if v := s.Get("ocsp_fail_open").(string); v != provider.BooleanDefault { - parsed, err := provider.BooleanStringToBool(v) - if err != nil { - return nil, diag.FromErr(err) - } - if parsed { - config.OCSPFailOpen = gosnowflake.OCSPFailOpenTrue - } else { - config.OCSPFailOpen = gosnowflake.OCSPFailOpenFalse - } - } - - if v, ok := s.GetOk("token"); ok && v.(string) != "" { - config.Token = v.(string) - config.Authenticator = gosnowflake.AuthTypeOAuth - } - if v, ok := s.GetOk("token_accessor"); ok { - if len(v.([]interface{})) > 0 { - tokenAccessor := v.([]interface{})[0].(map[string]interface{}) + if len(v.([]any)) > 0 { + tokenAccessor := v.([]any)[0].(map[string]any) tokenEndpoint := tokenAccessor["token_endpoint"].(string) refreshToken := tokenAccessor["refresh_token"].(string) clientID := tokenAccessor["client_id"].(string) @@ -742,92 +844,23 @@ func ConfigureProvider(ctx context.Context, s *schema.ResourceData) (any, diag.D redirectURI := tokenAccessor["redirect_uri"].(string) accessToken, err := GetAccessTokenWithRefreshToken(tokenEndpoint, clientID, clientSecret, refreshToken, redirectURI) if err != nil { - return nil, diag.FromErr(fmt.Errorf("could not retrieve access token from refresh token, err = %w", err)) + return nil, fmt.Errorf("could not retrieve access token from refresh token, err = %w", err) } config.Token = accessToken config.Authenticator = gosnowflake.AuthTypeOAuth } } - if v, ok := s.GetOk("keep_session_alive"); ok && v.(bool) { - config.KeepSessionAlive = v.(bool) - } - privateKeyPath := s.Get("private_key_path").(string) privateKey := s.Get("private_key").(string) privateKeyPassphrase := s.Get("private_key_passphrase").(string) v, err := getPrivateKey(privateKeyPath, privateKey, privateKeyPassphrase) if err != nil { - return nil, diag.FromErr(fmt.Errorf("could not retrieve private key: %w", err)) + return nil, fmt.Errorf("could not retrieve private key: %w", err) } if v != nil { config.PrivateKey = v } - if v, ok := s.GetOk("disable_telemetry"); ok && v.(bool) { - config.DisableTelemetry = v.(bool) - } - - if v := s.Get("client_request_mfa_token").(string); v != provider.BooleanDefault { - parsed, err := provider.BooleanStringToBool(v) - if err != nil { - return nil, diag.FromErr(err) - } - if parsed { - config.ClientRequestMfaToken = gosnowflake.ConfigBoolTrue - } else { - config.ClientRequestMfaToken = gosnowflake.ConfigBoolFalse - } - } - - if v := s.Get("client_store_temporary_credential").(string); v != provider.BooleanDefault { - parsed, err := provider.BooleanStringToBool(v) - if err != nil { - return nil, diag.FromErr(err) - } - if parsed { - config.ClientStoreTemporaryCredential = gosnowflake.ConfigBoolTrue - } else { - config.ClientStoreTemporaryCredential = gosnowflake.ConfigBoolFalse - } - } - - if v, ok := s.GetOk("disable_query_context_cache"); ok && v.(bool) { - config.DisableQueryContextCache = v.(bool) - } - - if v, ok := s.GetOk("profile"); ok && v.(string) != "" { - profile := v.(string) - if profile == "default" { - defaultConfig := sdk.DefaultConfig() - config = sdk.MergeConfig(config, defaultConfig) - } else { - profileConfig, err := sdk.ProfileConfig(profile) - if err != nil { - return "", diag.FromErr(errors.New("could not retrieve profile config: " + err.Error())) - } - if profileConfig == nil { - return "", diag.FromErr(errors.New("profile with name: " + profile + " not found in config file")) - } - // merge any credentials found in profile with config - config = sdk.MergeConfig(config, profileConfig) - } - } - - client, clientErr := sdk.NewClient(config) - - // needed for tests verifying different provider setups - if os.Getenv("TF_ACC") != "" && os.Getenv("SF_TF_ACC_TEST_CONFIGURE_CLIENT_ONCE") == "true" { - configuredClient = client - configureClientError = clientErr - } else { - configuredClient = nil - configureClientError = nil - } - - if clientErr != nil { - return nil, diag.FromErr(clientErr) - } - - return &provider.Context{Client: client}, nil + return config, nil } diff --git a/pkg/provider/provider_acceptance_test.go b/pkg/provider/provider_acceptance_test.go index 20f9b0cabe..61650a0b97 100644 --- a/pkg/provider/provider_acceptance_test.go +++ b/pkg/provider/provider_acceptance_test.go @@ -2,16 +2,26 @@ package provider_test import ( "fmt" + "net" + "net/url" "os" "regexp" + "strings" "testing" + "time" acc "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers" + "github.com/snowflakedb/gosnowflake" + "github.com/stretchr/testify/assert" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/testenvs" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/testprofiles" + internalprovider "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/snowflakeenvs" "github.com/hashicorp/terraform-plugin-testing/helper/resource" + "github.com/hashicorp/terraform-plugin-testing/terraform" "github.com/hashicorp/terraform-plugin-testing/tfversion" "github.com/stretchr/testify/require" ) @@ -154,6 +164,303 @@ func TestAcc_Provider_configureClientOnceSwitching(t *testing.T) { }) } +func TestAcc_Provider_tomlConfig(t *testing.T) { + // TODO(SNOW-1752038): unskip + t.Skip("Skip because this test needs a TOML config incompatible with older versions, causing tests with ExternalProvider to fail.") + t.Setenv(string(testenvs.ConfigureClientOnce), "") + + user := acc.DefaultConfig(t).User + pass := acc.DefaultConfig(t).Password + account := acc.DefaultConfig(t).Account + + oktaUrl, err := url.Parse("https://example.com") + require.NoError(t, err) + + resource.Test(t, resource.TestCase{ + ProtoV6ProviderFactories: acc.TestAccProtoV6ProviderFactories, + PreCheck: func() { + acc.TestAccPreCheck(t) + testenvs.AssertEnvNotSet(t, snowflakeenvs.User) + testenvs.AssertEnvNotSet(t, snowflakeenvs.Password) + }, + TerraformVersionChecks: []tfversion.TerraformVersionCheck{ + tfversion.RequireAbove(tfversion.Version1_5_0), + }, + Steps: []resource.TestStep{ + { + Config: providerConfig(testprofiles.CompleteFields), + Check: func(s *terraform.State) error { + config := acc.TestAccProvider.Meta().(*internalprovider.Context).Client.GetConfig() + assert.Equal(t, &gosnowflake.Config{ + Account: account, + User: user, + Password: pass, + Warehouse: "SNOWFLAKE", + Role: "ACCOUNTADMIN", + ValidateDefaultParameters: gosnowflake.ConfigBoolTrue, + ClientIP: net.ParseIP("1.2.3.4"), + Protocol: "https", + Host: fmt.Sprintf("%s.snowflakecomputing.com", account), + Params: map[string]*string{ + "foo": sdk.Pointer("bar"), + }, + Port: 443, + Authenticator: gosnowflake.AuthTypeSnowflake, + PasscodeInPassword: false, + OktaURL: oktaUrl, + LoginTimeout: 30 * time.Second, + RequestTimeout: 40 * time.Second, + JWTExpireTimeout: 50 * time.Second, + ClientTimeout: 10 * time.Second, + JWTClientTimeout: 20 * time.Second, + ExternalBrowserTimeout: 60 * time.Second, + MaxRetryCount: 1, + Application: "terraform-provider-snowflake", + InsecureMode: true, + OCSPFailOpen: gosnowflake.OCSPFailOpenTrue, + Token: "token", + KeepSessionAlive: true, + DisableTelemetry: true, + Tracing: "info", + TmpDirPath: ".", + ClientRequestMfaToken: gosnowflake.ConfigBoolTrue, + ClientStoreTemporaryCredential: gosnowflake.ConfigBoolTrue, + DisableQueryContextCache: true, + IncludeRetryReason: gosnowflake.ConfigBoolTrue, + DisableConsoleLogin: gosnowflake.ConfigBoolTrue, + }, config) + + return nil + }, + }, + }, + }) +} + +func TestAcc_Provider_envConfig(t *testing.T) { + // TODO(SNOW-1752038): unskip + t.Skip("Skip because this test needs a TOML config incompatible with older versions, causing tests with ExternalProvider to fail.") + t.Setenv(string(testenvs.ConfigureClientOnce), "") + + user := acc.DefaultConfig(t).User + pass := acc.DefaultConfig(t).Password + account := acc.DefaultConfig(t).Account + + accountParts := strings.SplitN(account, "-", 2) + + oktaUrlFromEnv, err := url.Parse("https://example-env.com") + require.NoError(t, err) + + resource.Test(t, resource.TestCase{ + ProtoV6ProviderFactories: acc.TestAccProtoV6ProviderFactories, + PreCheck: func() { + acc.TestAccPreCheck(t) + testenvs.AssertEnvNotSet(t, snowflakeenvs.User) + testenvs.AssertEnvNotSet(t, snowflakeenvs.Password) + }, + TerraformVersionChecks: []tfversion.TerraformVersionCheck{ + tfversion.RequireAbove(tfversion.Version1_5_0), + }, + Steps: []resource.TestStep{ + { + PreConfig: func() { + t.Setenv(snowflakeenvs.AccountName, accountParts[1]) + t.Setenv(snowflakeenvs.OrganizationName, accountParts[0]) + t.Setenv(snowflakeenvs.User, user) + t.Setenv(snowflakeenvs.Password, pass) + t.Setenv(snowflakeenvs.Warehouse, "SNOWFLAKE") + t.Setenv(snowflakeenvs.Protocol, "https") + t.Setenv(snowflakeenvs.Port, "443") + // do not set token - it should be propagated from TOML + t.Setenv(snowflakeenvs.Role, "ACCOUNTADMIN") + t.Setenv(snowflakeenvs.Authenticator, "snowflake") + t.Setenv(snowflakeenvs.ValidateDefaultParameters, "false") + t.Setenv(snowflakeenvs.ClientIp, "2.2.2.2") + t.Setenv(snowflakeenvs.Host, "") + t.Setenv(snowflakeenvs.Authenticator, "") + t.Setenv(snowflakeenvs.Passcode, "") + t.Setenv(snowflakeenvs.PasscodeInPassword, "false") + t.Setenv(snowflakeenvs.OktaUrl, "https://example-env.com") + t.Setenv(snowflakeenvs.LoginTimeout, "100") + t.Setenv(snowflakeenvs.RequestTimeout, "200") + t.Setenv(snowflakeenvs.JwtExpireTimeout, "300") + t.Setenv(snowflakeenvs.ClientTimeout, "400") + t.Setenv(snowflakeenvs.JwtClientTimeout, "500") + t.Setenv(snowflakeenvs.ExternalBrowserTimeout, "600") + t.Setenv(snowflakeenvs.InsecureMode, "false") + t.Setenv(snowflakeenvs.OcspFailOpen, "false") + t.Setenv(snowflakeenvs.KeepSessionAlive, "false") + t.Setenv(snowflakeenvs.DisableTelemetry, "false") + t.Setenv(snowflakeenvs.ClientRequestMfaToken, "false") + t.Setenv(snowflakeenvs.ClientStoreTemporaryCredential, "false") + t.Setenv(snowflakeenvs.DisableQueryContextCache, "false") + t.Setenv(snowflakeenvs.IncludeRetryReason, "false") + t.Setenv(snowflakeenvs.MaxRetryCount, "2") + t.Setenv(snowflakeenvs.DriverTracing, "debug") + t.Setenv(snowflakeenvs.TmpDirectoryPath, "../") + t.Setenv(snowflakeenvs.DisableConsoleLogin, "false") + }, + Config: providerConfig(testprofiles.CompleteFieldsInvalid), + Check: func(s *terraform.State) error { + config := acc.TestAccProvider.Meta().(*internalprovider.Context).Client.GetConfig() + assert.Equal(t, &gosnowflake.Config{ + Account: account, + User: user, + Password: pass, + Warehouse: "SNOWFLAKE", + Role: "ACCOUNTADMIN", + ValidateDefaultParameters: gosnowflake.ConfigBoolFalse, + ClientIP: net.ParseIP("2.2.2.2"), + Protocol: "https", + Params: map[string]*string{ + "foo": sdk.Pointer("bar"), + }, + Host: fmt.Sprintf("%s.snowflakecomputing.com", account), + Port: 443, + Authenticator: gosnowflake.AuthTypeSnowflake, + PasscodeInPassword: false, + OktaURL: oktaUrlFromEnv, + LoginTimeout: 100 * time.Second, + RequestTimeout: 200 * time.Second, + JWTExpireTimeout: 300 * time.Second, + ClientTimeout: 400 * time.Second, + JWTClientTimeout: 500 * time.Second, + ExternalBrowserTimeout: 600 * time.Second, + MaxRetryCount: 2, + Application: "terraform-provider-snowflake", + InsecureMode: true, + OCSPFailOpen: gosnowflake.OCSPFailOpenFalse, + Token: "token", + KeepSessionAlive: true, + DisableTelemetry: true, + Tracing: "debug", + TmpDirPath: "../", + ClientRequestMfaToken: gosnowflake.ConfigBoolFalse, + ClientStoreTemporaryCredential: gosnowflake.ConfigBoolFalse, + DisableQueryContextCache: true, + IncludeRetryReason: gosnowflake.ConfigBoolFalse, + DisableConsoleLogin: gosnowflake.ConfigBoolFalse, + }, config) + + return nil + }, + }, + }, + }) +} + +func TestAcc_Provider_tfConfig(t *testing.T) { + // TODO(SNOW-1752038): unskip + t.Skip("Skip because this test needs a TOML config incompatible with older versions, causing tests with ExternalProvider to fail.") + t.Setenv(string(testenvs.ConfigureClientOnce), "") + + user := acc.DefaultConfig(t).User + pass := acc.DefaultConfig(t).Password + account := acc.DefaultConfig(t).Account + + accountParts := strings.SplitN(account, "-", 2) + orgName, accountName := accountParts[0], accountParts[1] + + oktaUrlFromTf, err := url.Parse("https://example-tf.com") + require.NoError(t, err) + + resource.Test(t, resource.TestCase{ + ProtoV6ProviderFactories: acc.TestAccProtoV6ProviderFactories, + PreCheck: func() { + acc.TestAccPreCheck(t) + testenvs.AssertEnvNotSet(t, snowflakeenvs.User) + testenvs.AssertEnvNotSet(t, snowflakeenvs.Password) + }, + TerraformVersionChecks: []tfversion.TerraformVersionCheck{ + tfversion.RequireAbove(tfversion.Version1_5_0), + }, + Steps: []resource.TestStep{ + { + PreConfig: func() { + t.Setenv(snowflakeenvs.OrganizationName, "invalid") + t.Setenv(snowflakeenvs.AccountName, "invalid") + t.Setenv(snowflakeenvs.User, "invalid") + t.Setenv(snowflakeenvs.Password, "invalid") + t.Setenv(snowflakeenvs.Warehouse, "invalid") + t.Setenv(snowflakeenvs.Protocol, "invalid") + t.Setenv(snowflakeenvs.Port, "-1") + t.Setenv(snowflakeenvs.Token, "") + t.Setenv(snowflakeenvs.Role, "invalid") + t.Setenv(snowflakeenvs.ValidateDefaultParameters, "false") + t.Setenv(snowflakeenvs.ClientIp, "2.2.2.2") + t.Setenv(snowflakeenvs.Host, "") + t.Setenv(snowflakeenvs.Authenticator, "invalid") + t.Setenv(snowflakeenvs.Passcode, "") + t.Setenv(snowflakeenvs.PasscodeInPassword, "false") + t.Setenv(snowflakeenvs.OktaUrl, "https://example-env.com") + t.Setenv(snowflakeenvs.LoginTimeout, "100") + t.Setenv(snowflakeenvs.RequestTimeout, "200") + t.Setenv(snowflakeenvs.JwtExpireTimeout, "300") + t.Setenv(snowflakeenvs.ClientTimeout, "400") + t.Setenv(snowflakeenvs.JwtClientTimeout, "500") + t.Setenv(snowflakeenvs.ExternalBrowserTimeout, "600") + t.Setenv(snowflakeenvs.InsecureMode, "false") + t.Setenv(snowflakeenvs.OcspFailOpen, "false") + t.Setenv(snowflakeenvs.KeepSessionAlive, "false") + t.Setenv(snowflakeenvs.DisableTelemetry, "false") + t.Setenv(snowflakeenvs.ClientRequestMfaToken, "false") + t.Setenv(snowflakeenvs.ClientStoreTemporaryCredential, "false") + t.Setenv(snowflakeenvs.DisableQueryContextCache, "false") + t.Setenv(snowflakeenvs.IncludeRetryReason, "false") + t.Setenv(snowflakeenvs.MaxRetryCount, "2") + t.Setenv(snowflakeenvs.DriverTracing, "debug") + t.Setenv(snowflakeenvs.TmpDirectoryPath, "../") + t.Setenv(snowflakeenvs.DisableConsoleLogin, "false") + }, + Config: providerConfigAllFields(testprofiles.CompleteFieldsInvalid, orgName, accountName, user, pass), + Check: func(s *terraform.State) error { + config := acc.TestAccProvider.Meta().(*internalprovider.Context).Client.GetConfig() + assert.Equal(t, &gosnowflake.Config{ + Account: account, + User: user, + Password: pass, + Warehouse: "SNOWFLAKE", + Role: "ACCOUNTADMIN", + ValidateDefaultParameters: gosnowflake.ConfigBoolTrue, + ClientIP: net.ParseIP("3.3.3.3"), + Protocol: "https", + Params: map[string]*string{ + "foo": sdk.Pointer("piyo"), + }, + Host: fmt.Sprintf("%s.snowflakecomputing.com", account), + Port: 443, + Authenticator: gosnowflake.AuthTypeSnowflake, + PasscodeInPassword: false, + OktaURL: oktaUrlFromTf, + LoginTimeout: 101 * time.Second, + RequestTimeout: 201 * time.Second, + JWTExpireTimeout: 301 * time.Second, + ClientTimeout: 401 * time.Second, + JWTClientTimeout: 501 * time.Second, + ExternalBrowserTimeout: 601 * time.Second, + MaxRetryCount: 3, + Application: "terraform-provider-snowflake", + InsecureMode: true, + OCSPFailOpen: gosnowflake.OCSPFailOpenTrue, + Token: "token", + KeepSessionAlive: true, + DisableTelemetry: true, + Tracing: "info", + TmpDirPath: "../../", + ClientRequestMfaToken: gosnowflake.ConfigBoolTrue, + ClientStoreTemporaryCredential: gosnowflake.ConfigBoolTrue, + DisableQueryContextCache: true, + IncludeRetryReason: gosnowflake.ConfigBoolTrue, + DisableConsoleLogin: gosnowflake.ConfigBoolTrue, + }, config) + + return nil + }, + }, + }, + }) +} + func TestAcc_Provider_useNonExistentDefaultParams(t *testing.T) { t.Setenv(string(testenvs.ConfigureClientOnce), "") @@ -189,6 +496,19 @@ func TestAcc_Provider_useNonExistentDefaultParams(t *testing.T) { // prove we can use tri-value booleans, similarly to the ones in resources func TestAcc_Provider_triValueBoolean(t *testing.T) { t.Setenv(string(testenvs.ConfigureClientOnce), "") + account := acc.DefaultConfig(t).Account + user := acc.DefaultConfig(t).User + password := acc.DefaultConfig(t).Password + + // Prepare a temporary TOML config that is valid for v0.97.0. + // The default TOML is not valid because we set new fields, which is incorrect from v0.97.0's point of view. + c := fmt.Sprintf(` + [default] + account='%s' + user='%s' + password='%s' + `, account, user, password) + configPath := testhelpers.TestFile(t, "config", []byte(c)) resource.Test(t, resource.TestCase{ PreCheck: func() { @@ -201,15 +521,17 @@ func TestAcc_Provider_triValueBoolean(t *testing.T) { }, Steps: []resource.TestStep{ { - ExternalProviders: map[string]resource.ExternalProvider{ - "snowflake": { - VersionConstraint: "=0.97.0", - Source: "Snowflake-Labs/snowflake", - }, + PreConfig: func() { + t.Setenv(snowflakeenvs.ConfigPath, configPath) }, - Config: providerConfigWithClientStoreTemporaryCredential(testprofiles.Default, `true`), + ExternalProviders: acc.ExternalProviderWithExactVersion("0.97.0"), + Config: providerConfigWithClientStoreTemporaryCredential(testprofiles.Default, `true`), }, { + // Use the default TOML config again. + PreConfig: func() { + t.Setenv(snowflakeenvs.ConfigPath, "") + }, ProtoV6ProviderFactories: acc.TestAccProtoV6ProviderFactories, Config: providerConfigWithClientStoreTemporaryCredential(testprofiles.Default, `true`), }, @@ -234,7 +556,7 @@ func TestAcc_Provider_invalidConfigurations(t *testing.T) { }, { Config: providerConfigWithProtocol(testprofiles.Default, "invalid"), - ExpectError: regexp.MustCompile("invalid protocol: INVALID"), + ExpectError: regexp.MustCompile("invalid protocol: invalid"), }, { Config: providerConfigWithPort(testprofiles.Default, 123456789), @@ -242,7 +564,7 @@ func TestAcc_Provider_invalidConfigurations(t *testing.T) { }, { Config: providerConfigWithAuthType(testprofiles.Default, "invalid"), - ExpectError: regexp.MustCompile("invalid authenticator type: INVALID"), + ExpectError: regexp.MustCompile("invalid authenticator type: invalid"), }, { Config: providerConfigWithOktaUrl(testprofiles.Default, "invalid"), @@ -256,6 +578,15 @@ func TestAcc_Provider_invalidConfigurations(t *testing.T) { Config: providerConfigWithTokenEndpoint(testprofiles.Default, "invalid"), ExpectError: regexp.MustCompile(`expected "token_endpoint" to have a host, got invalid`), }, + { + Config: providerConfigWithLogLevel(testprofiles.Default, "invalid"), + ExpectError: regexp.MustCompile(`invalid driver log level: invalid`), + }, + { + Config: providerConfig("non-existing"), + // .* is used to match the error message regarding of the home user location + ExpectError: regexp.MustCompile(`profile "non-existing" not found in file .*.snowflake/config`), + }, }, }) } @@ -374,6 +705,15 @@ provider "snowflake" { `, profile, tokenEndpoint) + datasourceConfig() } +func providerConfigWithLogLevel(profile, logLevel string) string { + return fmt.Sprintf(` +provider "snowflake" { + profile = "%[1]s" + driver_tracing = "%[2]s" +} +`, profile, logLevel) + datasourceConfig() +} + func providerConfigWithClientIp(profile, clientIp string) string { return fmt.Sprintf(` provider "snowflake" { @@ -402,9 +742,73 @@ provider "snowflake" { `, user, pass, profile) + datasourceConfig() } +func providerConfigWithNewAccountId(profile, orgName, accountName string) string { + return fmt.Sprintf(` +provider "snowflake" { + profile = "%[1]s" + account_name = "%[2]s" + organization_name = "%[3]s" +} +`, profile, accountName, orgName) + datasourceConfig() +} + +func providerConfigComplete(profile, user, password, orgName, accountName string) string { + return fmt.Sprintf(` +provider "snowflake" { + profile = "%[1]s" + user = "%[2]s" + password = "%[3]s" + organization_name = "%[4]s" + account_name = "%[5]s" + warehouse = "SNOWFLAKE" +} +`, profile, user, password, orgName, accountName) + datasourceConfig() +} + func datasourceConfig() string { return fmt.Sprintf(` data snowflake_database "t" { name = "%s" }`, acc.TestDatabaseName) } + +func providerConfigAllFields(profile, orgName, accountName, user, password string) string { + return fmt.Sprintf(` +provider "snowflake" { + profile = "%[1]s" + organization_name = "%[2]s" + account_name = "%[3]s" + user = "%[4]s" + password = "%[5]s" + warehouse = "SNOWFLAKE" + protocol = "https" + port = "443" + role = "ACCOUNTADMIN" + validate_default_parameters = true + client_ip = "3.3.3.3" + authenticator = "snowflake" + okta_url = "https://example-tf.com" + login_timeout = 101 + request_timeout = 201 + jwt_expire_timeout = 301 + client_timeout = 401 + jwt_client_timeout = 501 + external_browser_timeout = 601 + insecure_mode = true + ocsp_fail_open = true + keep_session_alive = true + disable_telemetry = true + client_request_mfa_token = true + client_store_temporary_credential = true + disable_query_context_cache = true + include_retry_reason = true + max_retry_count = 3 + driver_tracing = "info" + tmp_directory_path = "../../" + disable_console_login = true + params = { + foo = "piyo" + } +} +`, profile, orgName, accountName, user, password) + datasourceConfig() +} diff --git a/pkg/provider/provider_helpers.go b/pkg/provider/provider_helpers.go index 0e8088cae4..1f7d9f2c3d 100644 --- a/pkg/provider/provider_helpers.go +++ b/pkg/provider/provider_helpers.go @@ -3,7 +3,6 @@ package provider import ( "crypto/rsa" "encoding/json" - "encoding/pem" "errors" "fmt" "io" @@ -12,79 +11,78 @@ import ( "os" "strconv" "strings" + "time" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" + "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" "github.com/mitchellh/go-homedir" "github.com/snowflakedb/gosnowflake" - "github.com/youmark/pkcs8" - "golang.org/x/crypto/ssh" ) -type authenticationType string +type protocol string const ( - authenticationTypeSnowflake authenticationType = "SNOWFLAKE" - authenticationTypeOauth authenticationType = "OAUTH" - authenticationTypeExternalBrowser authenticationType = "EXTERNALBROWSER" - authenticationTypeOkta authenticationType = "OKTA" - authenticationTypeJwtLegacy authenticationType = "JWT" - authenticationTypeJwt authenticationType = "SNOWFLAKE_JWT" - authenticationTypeTokenAccessor authenticationType = "TOKENACCESSOR" - authenticationTypeUsernamePasswordMfa authenticationType = "USERNAMEPASSWORDMFA" + // these values are lower case on purpose to match gosnowflake case + protocolHttp protocol = "http" + protocolHttps protocol = "https" ) -var allAuthenticationTypes = []authenticationType{ - authenticationTypeSnowflake, - authenticationTypeOauth, - authenticationTypeExternalBrowser, - authenticationTypeOkta, - authenticationTypeJwt, - authenticationTypeTokenAccessor, - authenticationTypeUsernamePasswordMfa, +var allProtocols = []protocol{ + protocolHttp, + protocolHttps, } -func toAuthenticatorType(s string) (gosnowflake.AuthType, error) { - s = strings.ToUpper(s) - switch s { - case string(authenticationTypeSnowflake): - return gosnowflake.AuthTypeSnowflake, nil - case string(authenticationTypeOauth): - return gosnowflake.AuthTypeOAuth, nil - case string(authenticationTypeExternalBrowser): - return gosnowflake.AuthTypeExternalBrowser, nil - case string(authenticationTypeOkta): - return gosnowflake.AuthTypeOkta, nil - case string(authenticationTypeJwt), string(authenticationTypeJwtLegacy): - return gosnowflake.AuthTypeJwt, nil - case string(authenticationTypeTokenAccessor): - return gosnowflake.AuthTypeTokenAccessor, nil - case string(authenticationTypeUsernamePasswordMfa): - return gosnowflake.AuthTypeUsernamePasswordMFA, nil +func toProtocol(s string) (protocol, error) { + lowerCase := strings.ToLower(s) + switch lowerCase { + case string(protocolHttp), + string(protocolHttps): + return protocol(lowerCase), nil default: - return gosnowflake.AuthType(0), fmt.Errorf("invalid authenticator type: %s", s) + return "", fmt.Errorf("invalid protocol: %s", s) } } -type protocol string +type driverLogLevel string const ( - protocolHttp protocol = "HTTP" - protocolHttps protocol = "HTTPS" + // these values are lower case on purpose to match gosnowflake case + logLevelTrace driverLogLevel = "trace" + logLevelDebug driverLogLevel = "debug" + logLevelInfo driverLogLevel = "info" + logLevelPrint driverLogLevel = "print" + logLevelWarning driverLogLevel = "warning" + logLevelError driverLogLevel = "error" + logLevelFatal driverLogLevel = "fatal" + logLevelPanic driverLogLevel = "panic" ) -var allProtocols = []protocol{ - protocolHttp, - protocolHttps, +var allDriverLogLevels = []driverLogLevel{ + logLevelTrace, + logLevelDebug, + logLevelInfo, + logLevelPrint, + logLevelWarning, + logLevelError, + logLevelFatal, + logLevelPanic, } -func toProtocol(s string) (protocol, error) { - s = strings.ToUpper(s) - switch s { - case string(protocolHttp): - return protocolHttp, nil - case string(protocolHttps): - return protocolHttps, nil +func toDriverLogLevel(s string) (driverLogLevel, error) { + lowerCase := strings.ToLower(s) + switch lowerCase { + case string(logLevelTrace), + string(logLevelDebug), + string(logLevelInfo), + string(logLevelPrint), + string(logLevelWarning), + string(logLevelError), + string(logLevelFatal), + string(logLevelPanic): + return driverLogLevel(lowerCase), nil default: - return "", fmt.Errorf("invalid protocol: %s", s) + return "", fmt.Errorf("invalid driver log level: %s", s) } } @@ -100,7 +98,7 @@ func getPrivateKey(privateKeyPath, privateKeyString, privateKeyPassphrase string return nil, fmt.Errorf("private Key file could not be read err = %w", err) } } - return parsePrivateKey(privateKeyBytes, []byte(privateKeyPassphrase)) + return sdk.ParsePrivateKey(privateKeyBytes, []byte(privateKeyPassphrase)) } func readFile(privateKeyPath string) ([]byte, error) { @@ -121,35 +119,6 @@ func readFile(privateKeyPath string) ([]byte, error) { return privateKeyBytes, nil } -func parsePrivateKey(privateKeyBytes []byte, passhrase []byte) (*rsa.PrivateKey, error) { - privateKeyBlock, _ := pem.Decode(privateKeyBytes) - if privateKeyBlock == nil { - return nil, fmt.Errorf("could not parse private key, key is not in PEM format") - } - - if privateKeyBlock.Type == "ENCRYPTED PRIVATE KEY" { - if len(passhrase) == 0 { - return nil, fmt.Errorf("private key requires a passphrase, but private_key_passphrase was not supplied") - } - privateKey, err := pkcs8.ParsePKCS8PrivateKeyRSA(privateKeyBlock.Bytes, passhrase) - if err != nil { - return nil, fmt.Errorf("could not parse encrypted private key with passphrase, only ciphers aes-128-cbc, aes-128-gcm, aes-192-cbc, aes-192-gcm, aes-256-cbc, aes-256-gcm, and des-ede3-cbc are supported err = %w", err) - } - return privateKey, nil - } - - privateKey, err := ssh.ParseRawPrivateKey(privateKeyBytes) - if err != nil { - return nil, fmt.Errorf("could not parse private key err = %w", err) - } - - rsaPrivateKey, ok := privateKey.(*rsa.PrivateKey) - if !ok { - return nil, errors.New("privateKey not of type RSA") - } - return rsaPrivateKey, nil -} - type GetRefreshTokenResponseBody struct { AccessToken string `json:"access_token"` TokenType string `json:"token_type"` @@ -201,3 +170,47 @@ func GetAccessTokenWithRefreshToken( func envNameFieldDescription(description, envName string) string { return fmt.Sprintf("%s Can also be sourced from the `%s` environment variable.", description, envName) } + +// TODO(SNOW-1787926): reuse these handlers with the ones in resources +func handleStringField(d *schema.ResourceData, key string, field *string) error { + if v, ok := d.GetOk(key); ok { + *field = v.(string) + } + return nil +} + +func handleBoolField(d *schema.ResourceData, key string, field *bool) error { + if v, ok := d.GetOk(key); ok { + *field = v.(bool) + } + return nil +} + +func handleDurationInSecondsAttribute(d *schema.ResourceData, key string, field *time.Duration) error { + if v, ok := d.GetOk(key); ok { + *field = time.Second * time.Duration(int64(v.(int))) + } + return nil +} + +func handleIntAttribute(d *schema.ResourceData, key string, field *int) error { + if v, ok := d.GetOk(key); ok { + *field = v.(int) + } + return nil +} + +func handleBooleanStringAttribute(d *schema.ResourceData, key string, field *gosnowflake.ConfigBool) error { + if v := d.Get(key).(string); v != provider.BooleanDefault { + parsed, err := provider.BooleanStringToBool(v) + if err != nil { + return err + } + if parsed { + *field = gosnowflake.ConfigBoolTrue + } else { + *field = gosnowflake.ConfigBoolFalse + } + } + return nil +} diff --git a/pkg/provider/provider_helpers_test.go b/pkg/provider/provider_helpers_test.go index eb9e2a8caf..c565894a86 100644 --- a/pkg/provider/provider_helpers_test.go +++ b/pkg/provider/provider_helpers_test.go @@ -3,29 +3,22 @@ package provider import ( "testing" - "github.com/snowflakedb/gosnowflake" "github.com/stretchr/testify/require" ) -func Test_Provider_toAuthenticationType(t *testing.T) { +func Test_Provider_toProtocol(t *testing.T) { type test struct { input string - want gosnowflake.AuthType + want protocol } valid := []test{ // Case insensitive. - {input: "snowflake", want: gosnowflake.AuthTypeSnowflake}, + {input: "http", want: protocolHttp}, // Supported Values. - {input: "SNOWFLAKE", want: gosnowflake.AuthTypeSnowflake}, - {input: "OAUTH", want: gosnowflake.AuthTypeOAuth}, - {input: "EXTERNALBROWSER", want: gosnowflake.AuthTypeExternalBrowser}, - {input: "OKTA", want: gosnowflake.AuthTypeOkta}, - {input: "JWT", want: gosnowflake.AuthTypeJwt}, - {input: "SNOWFLAKE_JWT", want: gosnowflake.AuthTypeJwt}, - {input: "TOKENACCESSOR", want: gosnowflake.AuthTypeTokenAccessor}, - {input: "USERNAMEPASSWORDMFA", want: gosnowflake.AuthTypeUsernamePasswordMFA}, + {input: "HTTP", want: protocolHttp}, + {input: "HTTPS", want: protocolHttps}, } invalid := []test{ @@ -35,7 +28,7 @@ func Test_Provider_toAuthenticationType(t *testing.T) { for _, tc := range valid { t.Run(tc.input, func(t *testing.T) { - got, err := toAuthenticatorType(tc.input) + got, err := toProtocol(tc.input) require.NoError(t, err) require.Equal(t, tc.want, got) }) @@ -43,25 +36,31 @@ func Test_Provider_toAuthenticationType(t *testing.T) { for _, tc := range invalid { t.Run(tc.input, func(t *testing.T) { - _, err := toAuthenticatorType(tc.input) + _, err := toProtocol(tc.input) require.Error(t, err) }) } } -func Test_Provider_toProtocol(t *testing.T) { +func Test_Provider_toDriverLogLevel(t *testing.T) { type test struct { input string - want protocol + want driverLogLevel } valid := []test{ // Case insensitive. - {input: "http", want: protocolHttp}, + {input: "WARNING", want: logLevelWarning}, // Supported Values. - {input: "HTTP", want: protocolHttp}, - {input: "HTTPS", want: protocolHttps}, + {input: "trace", want: logLevelTrace}, + {input: "debug", want: logLevelDebug}, + {input: "info", want: logLevelInfo}, + {input: "print", want: logLevelPrint}, + {input: "warning", want: logLevelWarning}, + {input: "error", want: logLevelError}, + {input: "fatal", want: logLevelFatal}, + {input: "panic", want: logLevelPanic}, } invalid := []test{ @@ -71,7 +70,7 @@ func Test_Provider_toProtocol(t *testing.T) { for _, tc := range valid { t.Run(tc.input, func(t *testing.T) { - got, err := toProtocol(tc.input) + got, err := toDriverLogLevel(tc.input) require.NoError(t, err) require.Equal(t, tc.want, got) }) @@ -79,7 +78,7 @@ func Test_Provider_toProtocol(t *testing.T) { for _, tc := range invalid { t.Run(tc.input, func(t *testing.T) { - _, err := toProtocol(tc.input) + _, err := toDriverLogLevel(tc.input) require.Error(t, err) }) } diff --git a/pkg/provider/testdata/config.toml b/pkg/provider/testdata/config.toml new file mode 100644 index 0000000000..1396222be6 --- /dev/null +++ b/pkg/provider/testdata/config.toml @@ -0,0 +1,42 @@ +[basic_fields] +account = 'account' + +[complete_fields] +account='account' +accountname='accountname' +organizationname='organizationname' +user='user' +username='username' +password='password' +host='host' +warehouse='warehouse' +role='role' +clientip='clientip' +protocol='protocol' +passcode='passcode' +port=1000 +passcodeinpassword=true +oktaurl='oktaurl' +clienttimeout=10 +jwtclienttimeout=20 +logintimeout=30 +requesttimeout=40 +jwtexpiretimeout=50 +externalbrowsertimeout=60 +maxretrycount=70 +authenticator='snowflake' +insecuremode=true +ocspfailopen=true +token='token' +keepsessionalive=true +privatekey='privatekey' +privatekeypassphrase='privatekeypassphrase' +disabletelemetry=true +validatedefaultparameters=true +clientrequestmfatoken=true +clientstoretemporarycredential=true +tracing='tracing' +tmpdirpath='.' +disablequerycontextcache=true +includeretryreason=true +disableconsolelogin=true diff --git a/pkg/resources/api_authentication_integration_with_authorization_code_grant.go b/pkg/resources/api_authentication_integration_with_authorization_code_grant.go index c2086e4608..c261d19305 100644 --- a/pkg/resources/api_authentication_integration_with_authorization_code_grant.go +++ b/pkg/resources/api_authentication_integration_with_authorization_code_grant.go @@ -31,7 +31,7 @@ var apiAuthAuthorizationCodeGrantSchema = func() map[string]*schema.Schema { Description: "Specifies a list of scopes to use when making a request from the OAuth by a role with USAGE on the integration during the OAuth client credentials flow.", }, } - return helpers.MergeMaps(apiAuthCommonSchema, apiAuthAuthorizationCodeGrant) + return collections.MergeMaps(apiAuthCommonSchema, apiAuthAuthorizationCodeGrant) }() func ApiAuthenticationIntegrationWithAuthorizationCodeGrant() *schema.Resource { diff --git a/pkg/resources/api_authentication_integration_with_client_credentials.go b/pkg/resources/api_authentication_integration_with_client_credentials.go index c98039dcfc..51c2209c3e 100644 --- a/pkg/resources/api_authentication_integration_with_client_credentials.go +++ b/pkg/resources/api_authentication_integration_with_client_credentials.go @@ -26,7 +26,7 @@ var apiAuthClientCredentialsSchema = func() map[string]*schema.Schema { Description: "Specifies a list of scopes to use when making a request from the OAuth by a role with USAGE on the integration during the OAuth client credentials flow.", }, } - return helpers.MergeMaps(apiAuthCommonSchema, apiAuthClientCredentials) + return collections.MergeMaps(apiAuthCommonSchema, apiAuthClientCredentials) }() func ApiAuthenticationIntegrationWithClientCredentials() *schema.Resource { diff --git a/pkg/resources/api_authentication_integration_with_jwt_bearer.go b/pkg/resources/api_authentication_integration_with_jwt_bearer.go index b3a508cb46..22703fe675 100644 --- a/pkg/resources/api_authentication_integration_with_jwt_bearer.go +++ b/pkg/resources/api_authentication_integration_with_jwt_bearer.go @@ -29,7 +29,7 @@ var apiAuthJwtBearerSchema = func() map[string]*schema.Schema { Required: true, }, } - return helpers.MergeMaps(apiAuthCommonSchema, apiAuthJwtBearer) + return collections.MergeMaps(apiAuthCommonSchema, apiAuthJwtBearer) }() func ApiAuthenticationIntegrationWithJwtBearer() *schema.Resource { diff --git a/pkg/resources/database.go b/pkg/resources/database.go index 9330dc8290..f9de40def7 100644 --- a/pkg/resources/database.go +++ b/pkg/resources/database.go @@ -8,6 +8,7 @@ import ( "strings" "time" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/util" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/schemas" @@ -95,7 +96,7 @@ func Database() *schema.Resource { DeleteContext: DeleteDatabase, Description: "Represents a standard database. If replication configuration is specified, the database is promoted to serve as a primary database for replication.", - Schema: helpers.MergeMaps(databaseSchema, databaseParametersSchema), + Schema: collections.MergeMaps(databaseSchema, databaseParametersSchema), Importer: &schema.ResourceImporter{ StateContext: ImportName[sdk.AccountObjectIdentifier], }, diff --git a/pkg/resources/oauth_integration_test.go b/pkg/resources/oauth_integration_test.go index bd9a5438c6..c8e2d9a126 100644 --- a/pkg/resources/oauth_integration_test.go +++ b/pkg/resources/oauth_integration_test.go @@ -10,7 +10,7 @@ import ( sqlmock "github.com/DATA-DOG/go-sqlmock" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/resources" - . "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers" + . "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers/mock" "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" "github.com/stretchr/testify/require" ) diff --git a/pkg/resources/saml_integration_test.go b/pkg/resources/saml_integration_test.go index 30f3d1220c..d326c12212 100644 --- a/pkg/resources/saml_integration_test.go +++ b/pkg/resources/saml_integration_test.go @@ -10,7 +10,7 @@ import ( sqlmock "github.com/DATA-DOG/go-sqlmock" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/resources" - . "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers" + . "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers/mock" "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" "github.com/stretchr/testify/require" ) diff --git a/pkg/resources/schema.go b/pkg/resources/schema.go index ff9d85efdb..27158b069d 100644 --- a/pkg/resources/schema.go +++ b/pkg/resources/schema.go @@ -103,7 +103,7 @@ func Schema() *schema.Resource { schemaParametersCustomDiff, ), - Schema: helpers.MergeMaps(schemaSchema, schemaParametersSchema), + Schema: collections.MergeMaps(schemaSchema, schemaParametersSchema), Importer: &schema.ResourceImporter{ StateContext: ImportSchema, }, diff --git a/pkg/resources/schema_parameters.go b/pkg/resources/schema_parameters.go index fe8deedbcc..1bb4d72f16 100644 --- a/pkg/resources/schema_parameters.go +++ b/pkg/resources/schema_parameters.go @@ -5,7 +5,7 @@ import ( "strconv" "strings" - "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" "github.com/hashicorp/terraform-plugin-sdk/v2/diag" @@ -52,7 +52,7 @@ func init() { Optional: true, } } - schemaParametersSchema = helpers.MergeMaps(databaseParametersSchema, additionalSchemaParameters) + schemaParametersSchema = collections.MergeMaps(databaseParametersSchema, additionalSchemaParameters) } func schemaParametersProvider(ctx context.Context, d ResourceIdProvider, meta any) ([]*sdk.Parameter, error) { diff --git a/pkg/resources/secondary_database.go b/pkg/resources/secondary_database.go index 155699d1c9..30298a4084 100644 --- a/pkg/resources/secondary_database.go +++ b/pkg/resources/secondary_database.go @@ -6,6 +6,7 @@ import ( "fmt" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/schemas" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -55,7 +56,7 @@ func SecondaryDatabase() *schema.Resource { databaseParametersCustomDiff, ComputedIfAnyAttributeChanged(secondaryDatabaseSchema, FullyQualifiedNameAttributeName, "name"), ), - Schema: helpers.MergeMaps(secondaryDatabaseSchema, databaseParametersSchema), + Schema: collections.MergeMaps(secondaryDatabaseSchema, databaseParametersSchema), Importer: &schema.ResourceImporter{ StateContext: ImportName[sdk.AccountObjectIdentifier], }, diff --git a/pkg/resources/secret_with_basic_authentication.go b/pkg/resources/secret_with_basic_authentication.go index 595fc6831d..6bec639e38 100644 --- a/pkg/resources/secret_with_basic_authentication.go +++ b/pkg/resources/secret_with_basic_authentication.go @@ -7,6 +7,7 @@ import ( "reflect" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/logging" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -30,7 +31,7 @@ var secretBasicAuthenticationSchema = func() map[string]*schema.Schema { Description: externalChangesNotDetectedFieldDescription("Specifies the password value to store in the secret."), }, } - return helpers.MergeMaps(secretCommonSchema, secretBasicAuthentication) + return collections.MergeMaps(secretCommonSchema, secretBasicAuthentication) }() func SecretWithBasicAuthentication() *schema.Resource { diff --git a/pkg/resources/secret_with_generic_string.go b/pkg/resources/secret_with_generic_string.go index ca9dfe55d6..7715abd818 100644 --- a/pkg/resources/secret_with_generic_string.go +++ b/pkg/resources/secret_with_generic_string.go @@ -7,6 +7,7 @@ import ( "reflect" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/logging" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -24,7 +25,7 @@ var secretGenericStringSchema = func() map[string]*schema.Schema { Description: externalChangesNotDetectedFieldDescription("Specifies the string to store in the secret. The string can be an API token or a string of sensitive value that can be used in the handler code of a UDF or stored procedure. For details, see [Creating and using an external access integration](https://docs.snowflake.com/en/developer-guide/external-network-access/creating-using-external-network-access). You should not use this property to store any kind of OAuth token; use one of the other secret types for your OAuth use cases."), }, } - return helpers.MergeMaps(secretCommonSchema, secretGenericString) + return collections.MergeMaps(secretCommonSchema, secretGenericString) }() func SecretWithGenericString() *schema.Resource { diff --git a/pkg/resources/secret_with_oauth_authorization_code_grant.go b/pkg/resources/secret_with_oauth_authorization_code_grant.go index 1797d51e8c..7ce5493ecf 100644 --- a/pkg/resources/secret_with_oauth_authorization_code_grant.go +++ b/pkg/resources/secret_with_oauth_authorization_code_grant.go @@ -7,6 +7,7 @@ import ( "reflect" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/logging" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -37,7 +38,7 @@ var secretAuthorizationCodeGrantSchema = func() map[string]*schema.Schema { DiffSuppressFunc: suppressIdentifierQuoting, }, } - return helpers.MergeMaps(secretCommonSchema, secretAuthorizationCodeGrant) + return collections.MergeMaps(secretCommonSchema, secretAuthorizationCodeGrant) }() func SecretWithAuthorizationCodeGrant() *schema.Resource { diff --git a/pkg/resources/secret_with_oauth_client_credentials.go b/pkg/resources/secret_with_oauth_client_credentials.go index 866083688d..1e46e40822 100644 --- a/pkg/resources/secret_with_oauth_client_credentials.go +++ b/pkg/resources/secret_with_oauth_client_credentials.go @@ -7,6 +7,7 @@ import ( "reflect" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/logging" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -31,7 +32,7 @@ var secretClientCredentialsSchema = func() map[string]*schema.Schema { Description: "Specifies a list of scopes to use when making a request from the OAuth server by a role with USAGE on the integration during the OAuth client credentials flow.", }, } - return helpers.MergeMaps(secretCommonSchema, secretClientCredentials) + return collections.MergeMaps(secretCommonSchema, secretClientCredentials) }() func SecretWithClientCredentials() *schema.Resource { diff --git a/pkg/resources/shared_database.go b/pkg/resources/shared_database.go index 8339148656..18a882d9a8 100644 --- a/pkg/resources/shared_database.go +++ b/pkg/resources/shared_database.go @@ -6,6 +6,7 @@ import ( "fmt" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/schemas" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" @@ -56,7 +57,7 @@ func SharedDatabase() *schema.Resource { ComputedIfAnyAttributeChanged(sharedDatabaseSchema, FullyQualifiedNameAttributeName, "name"), ), - Schema: helpers.MergeMaps(sharedDatabaseSchema, sharedDatabaseParametersSchema), + Schema: collections.MergeMaps(sharedDatabaseSchema, sharedDatabaseParametersSchema), Importer: &schema.ResourceImporter{ StateContext: ImportName[sdk.AccountObjectIdentifier], }, diff --git a/pkg/resources/stream_on_directory_table.go b/pkg/resources/stream_on_directory_table.go index 9277c90616..491692d7dd 100644 --- a/pkg/resources/stream_on_directory_table.go +++ b/pkg/resources/stream_on_directory_table.go @@ -6,6 +6,7 @@ import ( "fmt" "log" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" @@ -26,7 +27,7 @@ var streamOnDirectoryTableSchema = func() map[string]*schema.Schema { ValidateDiagFunc: IsValidIdentifier[sdk.SchemaObjectIdentifier](), }, } - return helpers.MergeMaps(streamCommonSchema, streamOnDirectoryTable) + return collections.MergeMaps(streamCommonSchema, streamOnDirectoryTable) }() func StreamOnDirectoryTable() *schema.Resource { diff --git a/pkg/resources/stream_on_external_table.go b/pkg/resources/stream_on_external_table.go index 73748790eb..bee62563d7 100644 --- a/pkg/resources/stream_on_external_table.go +++ b/pkg/resources/stream_on_external_table.go @@ -6,6 +6,7 @@ import ( "fmt" "log" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" @@ -37,7 +38,7 @@ var streamOnExternalTableSchema = func() map[string]*schema.Schema { AtAttributeName: atSchema, BeforeAttributeName: beforeSchema, } - return helpers.MergeMaps(streamCommonSchema, streamOnExternalTable) + return collections.MergeMaps(streamCommonSchema, streamOnExternalTable) }() func StreamOnExternalTable() *schema.Resource { diff --git a/pkg/resources/stream_on_table.go b/pkg/resources/stream_on_table.go index 7dfaced2b1..18a99c1b55 100644 --- a/pkg/resources/stream_on_table.go +++ b/pkg/resources/stream_on_table.go @@ -6,6 +6,7 @@ import ( "fmt" "log" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" @@ -44,7 +45,7 @@ var streamOnTableSchema = func() map[string]*schema.Schema { AtAttributeName: atSchema, BeforeAttributeName: beforeSchema, } - return helpers.MergeMaps(streamCommonSchema, streamOnTable) + return collections.MergeMaps(streamCommonSchema, streamOnTable) }() func StreamOnTable() *schema.Resource { diff --git a/pkg/resources/stream_on_view.go b/pkg/resources/stream_on_view.go index c9e1bcb3a6..3b6eb72941 100644 --- a/pkg/resources/stream_on_view.go +++ b/pkg/resources/stream_on_view.go @@ -6,6 +6,7 @@ import ( "fmt" "log" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" @@ -44,7 +45,7 @@ var StreamOnViewSchema = func() map[string]*schema.Schema { AtAttributeName: atSchema, BeforeAttributeName: beforeSchema, } - return helpers.MergeMaps(streamCommonSchema, streamOnView) + return collections.MergeMaps(streamCommonSchema, streamOnView) }() func StreamOnView() *schema.Resource { diff --git a/pkg/resources/user.go b/pkg/resources/user.go index cb4a0cfa6a..2d391232ee 100644 --- a/pkg/resources/user.go +++ b/pkg/resources/user.go @@ -191,7 +191,7 @@ func User() *schema.Resource { DeleteContext: DeleteUser, Description: "Resource used to manage user objects. For more information, check [user documentation](https://docs.snowflake.com/en/sql-reference/commands-user-role#user-management).", - Schema: helpers.MergeMaps(userSchema, userParametersSchema), + Schema: collections.MergeMaps(userSchema, userParametersSchema), Importer: &schema.ResourceImporter{ StateContext: GetImportUserFunc(sdk.UserTypePerson), }, @@ -224,7 +224,7 @@ func ServiceUser() *schema.Resource { DeleteContext: DeleteUser, Description: "Resource used to manage service user objects. For more information, check [user documentation](https://docs.snowflake.com/en/sql-reference/commands-user-role#user-management).", - Schema: helpers.MergeMaps(serviceUserSchema, userParametersSchema), + Schema: collections.MergeMaps(serviceUserSchema, userParametersSchema), Importer: &schema.ResourceImporter{ StateContext: GetImportUserFunc(sdk.UserTypeService), }, @@ -247,7 +247,7 @@ func LegacyServiceUser() *schema.Resource { DeleteContext: DeleteUser, Description: "Resource used to manage legacy service user objects. For more information, check [user documentation](https://docs.snowflake.com/en/sql-reference/commands-user-role#user-management).", - Schema: helpers.MergeMaps(legacyServiceUserSchema, userParametersSchema), + Schema: collections.MergeMaps(legacyServiceUserSchema, userParametersSchema), Importer: &schema.ResourceImporter{ StateContext: GetImportUserFunc(sdk.UserTypeLegacyService), }, diff --git a/pkg/schemas/security_integration.go b/pkg/schemas/security_integration.go index 051b49ea5d..cc069a0199 100644 --- a/pkg/schemas/security_integration.go +++ b/pkg/schemas/security_integration.go @@ -6,11 +6,12 @@ import ( "strings" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/helpers" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" ) var ( - SecurityIntegrationDescribeSchema = helpers.MergeMaps( + SecurityIntegrationDescribeSchema = collections.MergeMaps( DescribeApiAuthSecurityIntegrationSchema, DescribeExternalOauthSecurityIntegrationSchema, DescribeOauthIntegrationForCustomClients, diff --git a/pkg/sdk/client.go b/pkg/sdk/client.go index 94b0f9f68a..93e82afb37 100644 --- a/pkg/sdk/client.go +++ b/pkg/sdk/client.go @@ -21,7 +21,6 @@ var ( func init() { instrumentedSQL = os.Getenv(snowflakeenvs.NoInstrumentedSql) == "" - gosnowflakeLoggingLevel = os.Getenv(snowflakeenvs.GosnowflakeLogLevel) } type Client struct { @@ -146,10 +145,6 @@ func NewClient(cfg *gosnowflake.Config) (*Client, error) { driverName = "snowflake-instrumented" } - if gosnowflakeLoggingLevel != "" { - cfg.Tracing = gosnowflakeLoggingLevel - } - dsn, err := gosnowflake.DSN(cfg) if err != nil { return nil, err diff --git a/pkg/sdk/config.go b/pkg/sdk/config.go index 8c4441c330..354779b8ba 100644 --- a/pkg/sdk/config.go +++ b/pkg/sdk/config.go @@ -1,12 +1,24 @@ package sdk import ( + "crypto/rsa" + "encoding/pem" + "errors" + "fmt" "log" + "net" + "net/url" "os" "path/filepath" + "slices" + "strings" + "time" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/collections" "github.com/pelletier/go-toml/v2" "github.com/snowflakedb/gosnowflake" + "github.com/youmark/pkcs8" + "golang.org/x/crypto/ssh" ) func DefaultConfig() *gosnowflake.Config { @@ -19,20 +31,28 @@ func DefaultConfig() *gosnowflake.Config { } func ProfileConfig(profile string) (*gosnowflake.Config, error) { - configs, err := loadConfigFile() + path, err := GetConfigFileName() if err != nil { return nil, err } + configs, err := loadConfigFile(path) + if err != nil { + return nil, fmt.Errorf("could not load config file: %w", err) + } + if profile == "" { profile = "default" } var config *gosnowflake.Config if cfg, ok := configs[profile]; ok { log.Printf("[DEBUG] loading config for profile: \"%s\"", profile) - config = cfg + driverCfg, err := cfg.DriverConfig() + if err != nil { + return nil, fmt.Errorf("converting profile \"%s\" in file %s failed: %w", profile, path, err) + } + config = Pointer(driverCfg) } - if config == nil { log.Printf("[DEBUG] no config found for profile: \"%s\"", profile) return nil, nil @@ -60,6 +80,9 @@ func MergeConfig(baseConfig *gosnowflake.Config, mergeConfig *gosnowflake.Config if baseConfig.Password == "" { baseConfig.Password = mergeConfig.Password } + if baseConfig.Warehouse == "" { + baseConfig.Warehouse = mergeConfig.Warehouse + } if baseConfig.Role == "" { baseConfig.Role = mergeConfig.Role } @@ -69,11 +92,113 @@ func MergeConfig(baseConfig *gosnowflake.Config, mergeConfig *gosnowflake.Config if baseConfig.Host == "" { baseConfig.Host = mergeConfig.Host } + if !configBoolSet(baseConfig.ValidateDefaultParameters) { + baseConfig.ValidateDefaultParameters = mergeConfig.ValidateDefaultParameters + } + if mergedMap := collections.MergeMaps(mergeConfig.Params, baseConfig.Params); len(mergedMap) > 0 { + baseConfig.Params = mergedMap + } + if baseConfig.ClientIP == nil { + baseConfig.ClientIP = mergeConfig.ClientIP + } + if baseConfig.Protocol == "" { + baseConfig.Protocol = mergeConfig.Protocol + } + if baseConfig.Host == "" { + baseConfig.Host = mergeConfig.Host + } + if baseConfig.Port == 0 { + baseConfig.Port = mergeConfig.Port + } + if baseConfig.Authenticator == 0 { + baseConfig.Authenticator = mergeConfig.Authenticator + } + if baseConfig.Passcode == "" { + baseConfig.Passcode = mergeConfig.Passcode + } + if !baseConfig.PasscodeInPassword { + baseConfig.PasscodeInPassword = mergeConfig.PasscodeInPassword + } + if baseConfig.OktaURL == nil { + baseConfig.OktaURL = mergeConfig.OktaURL + } + if baseConfig.LoginTimeout == 0 { + baseConfig.LoginTimeout = mergeConfig.LoginTimeout + } + if baseConfig.RequestTimeout == 0 { + baseConfig.RequestTimeout = mergeConfig.RequestTimeout + } + if baseConfig.JWTExpireTimeout == 0 { + baseConfig.JWTExpireTimeout = mergeConfig.JWTExpireTimeout + } + if baseConfig.ClientTimeout == 0 { + baseConfig.ClientTimeout = mergeConfig.ClientTimeout + } + if baseConfig.JWTClientTimeout == 0 { + baseConfig.JWTClientTimeout = mergeConfig.JWTClientTimeout + } + if baseConfig.ExternalBrowserTimeout == 0 { + baseConfig.ExternalBrowserTimeout = mergeConfig.ExternalBrowserTimeout + } + if baseConfig.MaxRetryCount == 0 { + baseConfig.MaxRetryCount = mergeConfig.MaxRetryCount + } + if !baseConfig.InsecureMode { + baseConfig.InsecureMode = mergeConfig.InsecureMode + } + if baseConfig.OCSPFailOpen == 0 { + baseConfig.OCSPFailOpen = mergeConfig.OCSPFailOpen + } + if baseConfig.Token == "" { + baseConfig.Token = mergeConfig.Token + } + if !baseConfig.KeepSessionAlive { + baseConfig.KeepSessionAlive = mergeConfig.KeepSessionAlive + } + if baseConfig.PrivateKey == nil { + baseConfig.PrivateKey = mergeConfig.PrivateKey + } + if !baseConfig.DisableTelemetry { + baseConfig.DisableTelemetry = mergeConfig.DisableTelemetry + } + if baseConfig.Tracing == "" { + baseConfig.Tracing = mergeConfig.Tracing + } + if baseConfig.TmpDirPath == "" { + baseConfig.TmpDirPath = mergeConfig.TmpDirPath + } + if !configBoolSet(baseConfig.ClientRequestMfaToken) { + baseConfig.ClientRequestMfaToken = mergeConfig.ClientRequestMfaToken + } + if !configBoolSet(baseConfig.ClientStoreTemporaryCredential) { + baseConfig.ClientStoreTemporaryCredential = mergeConfig.ClientStoreTemporaryCredential + } + if !baseConfig.DisableQueryContextCache { + baseConfig.DisableQueryContextCache = mergeConfig.DisableQueryContextCache + } + if !configBoolSet(baseConfig.IncludeRetryReason) { + baseConfig.IncludeRetryReason = mergeConfig.IncludeRetryReason + } + if !configBoolSet(baseConfig.DisableConsoleLogin) { + baseConfig.DisableConsoleLogin = mergeConfig.DisableConsoleLogin + } return baseConfig } -func configFile() (string, error) { - // has the user overwridden the default config path? +func configBoolSet(v gosnowflake.ConfigBool) bool { + // configBoolNotSet is unexported in the driver, so we check if it's neither true nor false + return slices.Contains([]gosnowflake.ConfigBool{gosnowflake.ConfigBoolFalse, gosnowflake.ConfigBoolTrue}, v) +} + +func boolToConfigBool(v bool) gosnowflake.ConfigBool { + if v { + return gosnowflake.ConfigBoolTrue + } + return gosnowflake.ConfigBoolFalse +} + +func GetConfigFileName() (string, error) { + // has the user overridden the default config path? if configPath, ok := os.LookupEnv("SNOWFLAKE_CONFIG_PATH"); ok { if configPath != "" { return configPath, nil @@ -87,20 +212,238 @@ func configFile() (string, error) { return filepath.Join(dir, ".snowflake", "config"), nil } -func loadConfigFile() (map[string]*gosnowflake.Config, error) { - path, err := configFile() +// TODO(SNOW-1787920): improve TOML parsing +type ConfigDTO struct { + Account *string `toml:"account"` + AccountName *string `toml:"accountname"` + OrganizationName *string `toml:"organizationname"` + User *string `toml:"user"` + Username *string `toml:"username"` + Password *string `toml:"password"` + Host *string `toml:"host"` + Warehouse *string `toml:"warehouse"` + Role *string `toml:"role"` + Params *map[string]*string `toml:"params"` + ClientIp *string `toml:"clientip"` + Protocol *string `toml:"protocol"` + Passcode *string `toml:"passcode"` + Port *int `toml:"port"` + PasscodeInPassword *bool `toml:"passcodeinpassword"` + OktaUrl *string `toml:"oktaurl"` + ClientTimeout *int `toml:"clienttimeout"` + JwtClientTimeout *int `toml:"jwtclienttimeout"` + LoginTimeout *int `toml:"logintimeout"` + RequestTimeout *int `toml:"requesttimeout"` + JwtExpireTimeout *int `toml:"jwtexpiretimeout"` + ExternalBrowserTimeout *int `toml:"externalbrowsertimeout"` + MaxRetryCount *int `toml:"maxretrycount"` + Authenticator *string `toml:"authenticator"` + InsecureMode *bool `toml:"insecuremode"` + OcspFailOpen *bool `toml:"ocspfailopen"` + Token *string `toml:"token"` + KeepSessionAlive *bool `toml:"keepsessionalive"` + PrivateKey *string `toml:"privatekey,multiline"` + PrivateKeyPassphrase *string `toml:"privatekeypassphrase"` + DisableTelemetry *bool `toml:"disabletelemetry"` + ValidateDefaultParameters *bool `toml:"validatedefaultparameters"` + ClientRequestMfaToken *bool `toml:"clientrequestmfatoken"` + ClientStoreTemporaryCredential *bool `toml:"clientstoretemporarycredential"` + Tracing *string `toml:"tracing"` + TmpDirPath *string `toml:"tmpdirpath"` + DisableQueryContextCache *bool `toml:"disablequerycontextcache"` + IncludeRetryReason *bool `toml:"includeretryreason"` + DisableConsoleLogin *bool `toml:"disableconsolelogin"` +} + +func (c *ConfigDTO) DriverConfig() (gosnowflake.Config, error) { + driverCfg := gosnowflake.Config{} + pointerAttributeSet(c.Account, &driverCfg.Account) + if c.AccountName != nil && c.OrganizationName != nil { + driverCfg.Account = fmt.Sprintf("%s-%s", *c.OrganizationName, *c.AccountName) + } + pointerAttributeSet(c.User, &driverCfg.User) + pointerAttributeSet(c.Username, &driverCfg.User) + pointerAttributeSet(c.Password, &driverCfg.Password) + pointerAttributeSet(c.Host, &driverCfg.Host) + pointerAttributeSet(c.Warehouse, &driverCfg.Warehouse) + pointerAttributeSet(c.Role, &driverCfg.Role) + pointerAttributeSet(c.Params, &driverCfg.Params) + pointerIpAttributeSet(c.ClientIp, &driverCfg.ClientIP) + pointerAttributeSet(c.Protocol, &driverCfg.Protocol) + pointerAttributeSet(c.Passcode, &driverCfg.Passcode) + pointerAttributeSet(c.Port, &driverCfg.Port) + pointerAttributeSet(c.PasscodeInPassword, &driverCfg.PasscodeInPassword) + err := pointerUrlAttributeSet(c.OktaUrl, &driverCfg.OktaURL) if err != nil { - return nil, err + return gosnowflake.Config{}, err } + pointerTimeInSecondsAttributeSet(c.ClientTimeout, &driverCfg.ClientTimeout) + pointerTimeInSecondsAttributeSet(c.JwtClientTimeout, &driverCfg.JWTClientTimeout) + pointerTimeInSecondsAttributeSet(c.LoginTimeout, &driverCfg.LoginTimeout) + pointerTimeInSecondsAttributeSet(c.RequestTimeout, &driverCfg.RequestTimeout) + pointerTimeInSecondsAttributeSet(c.JwtExpireTimeout, &driverCfg.JWTExpireTimeout) + pointerTimeInSecondsAttributeSet(c.ExternalBrowserTimeout, &driverCfg.ExternalBrowserTimeout) + pointerAttributeSet(c.MaxRetryCount, &driverCfg.MaxRetryCount) + if c.Authenticator != nil { + authenticator, err := ToAuthenticatorType(*c.Authenticator) + if err != nil { + return gosnowflake.Config{}, err + } + driverCfg.Authenticator = authenticator + } + pointerAttributeSet(c.InsecureMode, &driverCfg.InsecureMode) + if c.OcspFailOpen != nil { + if *c.OcspFailOpen { + driverCfg.OCSPFailOpen = gosnowflake.OCSPFailOpenTrue + } else { + driverCfg.OCSPFailOpen = gosnowflake.OCSPFailOpenFalse + } + } + pointerAttributeSet(c.Token, &driverCfg.Token) + pointerAttributeSet(c.KeepSessionAlive, &driverCfg.KeepSessionAlive) + if c.PrivateKey != nil { + passphrase := make([]byte, 0) + if c.PrivateKeyPassphrase != nil { + passphrase = []byte(*c.PrivateKeyPassphrase) + } + privKey, err := ParsePrivateKey([]byte(*c.PrivateKey), passphrase) + if err != nil { + return gosnowflake.Config{}, err + } + driverCfg.PrivateKey = privKey + } + pointerAttributeSet(c.DisableTelemetry, &driverCfg.DisableTelemetry) + pointerConfigBoolAttributeSet(c.ValidateDefaultParameters, &driverCfg.ValidateDefaultParameters) + pointerConfigBoolAttributeSet(c.ClientRequestMfaToken, &driverCfg.ClientRequestMfaToken) + pointerConfigBoolAttributeSet(c.ClientStoreTemporaryCredential, &driverCfg.ClientStoreTemporaryCredential) + pointerAttributeSet(c.Tracing, &driverCfg.Tracing) + pointerAttributeSet(c.TmpDirPath, &driverCfg.TmpDirPath) + pointerAttributeSet(c.DisableQueryContextCache, &driverCfg.DisableQueryContextCache) + pointerConfigBoolAttributeSet(c.IncludeRetryReason, &driverCfg.IncludeRetryReason) + pointerConfigBoolAttributeSet(c.DisableConsoleLogin, &driverCfg.DisableConsoleLogin) + + return driverCfg, nil +} + +func pointerAttributeSet[T any](src, dst *T) { + if src != nil { + *dst = *src + } +} + +func pointerTimeInSecondsAttributeSet(src *int, dst *time.Duration) { + if src != nil { + *dst = time.Second * time.Duration(*src) + } +} + +func pointerConfigBoolAttributeSet(src *bool, dst *gosnowflake.ConfigBool) { + if src != nil { + *dst = boolToConfigBool(*src) + } +} + +func pointerIpAttributeSet(src *string, dst *net.IP) { + if src != nil { + *dst = net.ParseIP(*src) + } +} + +func pointerUrlAttributeSet(src *string, dst **url.URL) error { + if src != nil { + url, err := url.Parse(*src) + if err != nil { + return err + } + *dst = url + } + return nil +} + +func loadConfigFile(path string) (map[string]ConfigDTO, error) { dat, err := os.ReadFile(path) if err != nil { return nil, err } - var s map[string]*gosnowflake.Config + var s map[string]ConfigDTO err = toml.Unmarshal(dat, &s) if err != nil { - log.Printf("[DEBUG] error unmarshalling config file: %v\n", err) - return nil, nil + return nil, fmt.Errorf("unmarshalling config file %s: %w", path, err) } return s, nil } + +func ParsePrivateKey(privateKeyBytes []byte, passphrase []byte) (*rsa.PrivateKey, error) { + privateKeyBlock, _ := pem.Decode(privateKeyBytes) + if privateKeyBlock == nil { + return nil, fmt.Errorf("could not parse private key, key is not in PEM format") + } + + if privateKeyBlock.Type == "ENCRYPTED PRIVATE KEY" { + if len(passphrase) == 0 { + return nil, fmt.Errorf("private key requires a passphrase, but private_key_passphrase was not supplied") + } + privateKey, err := pkcs8.ParsePKCS8PrivateKeyRSA(privateKeyBlock.Bytes, passphrase) + if err != nil { + return nil, fmt.Errorf("could not parse encrypted private key with passphrase, only ciphers aes-128-cbc, aes-128-gcm, aes-192-cbc, aes-192-gcm, aes-256-cbc, aes-256-gcm, and des-ede3-cbc are supported err = %w", err) + } + return privateKey, nil + } + + // TODO(SNOW-1754327): check if we can simply use ssh.ParseRawPrivateKeyWithPassphrase + privateKey, err := ssh.ParseRawPrivateKey(privateKeyBytes) + if err != nil { + return nil, fmt.Errorf("could not parse private key err = %w", err) + } + + rsaPrivateKey, ok := privateKey.(*rsa.PrivateKey) + if !ok { + return nil, errors.New("privateKey not of type RSA") + } + return rsaPrivateKey, nil +} + +type AuthenticationType string + +const ( + AuthenticationTypeSnowflake AuthenticationType = "SNOWFLAKE" + AuthenticationTypeOauth AuthenticationType = "OAUTH" + AuthenticationTypeExternalBrowser AuthenticationType = "EXTERNALBROWSER" + AuthenticationTypeOkta AuthenticationType = "OKTA" + AuthenticationTypeJwtLegacy AuthenticationType = "JWT" + AuthenticationTypeJwt AuthenticationType = "SNOWFLAKE_JWT" + AuthenticationTypeTokenAccessor AuthenticationType = "TOKENACCESSOR" + AuthenticationTypeUsernamePasswordMfa AuthenticationType = "USERNAMEPASSWORDMFA" +) + +var AllAuthenticationTypes = []AuthenticationType{ + AuthenticationTypeSnowflake, + AuthenticationTypeOauth, + AuthenticationTypeExternalBrowser, + AuthenticationTypeOkta, + AuthenticationTypeJwtLegacy, + AuthenticationTypeJwt, + AuthenticationTypeTokenAccessor, + AuthenticationTypeUsernamePasswordMfa, +} + +func ToAuthenticatorType(s string) (gosnowflake.AuthType, error) { + switch strings.ToUpper(s) { + case string(AuthenticationTypeSnowflake): + return gosnowflake.AuthTypeSnowflake, nil + case string(AuthenticationTypeOauth): + return gosnowflake.AuthTypeOAuth, nil + case string(AuthenticationTypeExternalBrowser): + return gosnowflake.AuthTypeExternalBrowser, nil + case string(AuthenticationTypeOkta): + return gosnowflake.AuthTypeOkta, nil + case string(AuthenticationTypeJwt), string(AuthenticationTypeJwtLegacy): + return gosnowflake.AuthTypeJwt, nil + case string(AuthenticationTypeTokenAccessor): + return gosnowflake.AuthTypeTokenAccessor, nil + case string(AuthenticationTypeUsernamePasswordMfa): + return gosnowflake.AuthTypeUsernamePasswordMFA, nil + default: + return gosnowflake.AuthType(0), fmt.Errorf("invalid authenticator type: %s", s) + } +} diff --git a/pkg/sdk/config_test.go b/pkg/sdk/config_test.go index d34d812a88..dd5b8bd649 100644 --- a/pkg/sdk/config_test.go +++ b/pkg/sdk/config_test.go @@ -1,11 +1,17 @@ package sdk import ( - "os" - "path/filepath" + "crypto/x509" + "encoding/pem" + "fmt" + "net" + "net/url" "testing" + "time" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/helpers/random" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/snowflakeenvs" + "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/testhelpers" "github.com/snowflakedb/gosnowflake" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -25,40 +31,120 @@ func TestLoadConfigFile(t *testing.T) { password='abcd1234' role='SECURITYADMIN' ` - configPath := testFile(t, "config", []byte(c)) - t.Setenv(snowflakeenvs.ConfigPath, configPath) + configPath := testhelpers.TestFile(t, "config", []byte(c)) - m, err := loadConfigFile() + m, err := loadConfigFile(configPath) require.NoError(t, err) - assert.Equal(t, "TEST_ACCOUNT", m["default"].Account) - assert.Equal(t, "TEST_USER", m["default"].User) - assert.Equal(t, "abcd1234", m["default"].Password) - assert.Equal(t, "ACCOUNTADMIN", m["default"].Role) - assert.Equal(t, "TEST_ACCOUNT", m["securityadmin"].Account) - assert.Equal(t, "TEST_USER", m["securityadmin"].User) - assert.Equal(t, "abcd1234", m["securityadmin"].Password) - assert.Equal(t, "SECURITYADMIN", m["securityadmin"].Role) + assert.Equal(t, "TEST_ACCOUNT", *m["default"].Account) + assert.Equal(t, "TEST_USER", *m["default"].User) + assert.Equal(t, "abcd1234", *m["default"].Password) + assert.Equal(t, "ACCOUNTADMIN", *m["default"].Role) + assert.Equal(t, "TEST_ACCOUNT", *m["securityadmin"].Account) + assert.Equal(t, "TEST_USER", *m["securityadmin"].User) + assert.Equal(t, "abcd1234", *m["securityadmin"].Password) + assert.Equal(t, "SECURITYADMIN", *m["securityadmin"].Role) } func TestProfileConfig(t *testing.T) { - c := ` + unencryptedKey, encryptedKey := random.GenerateRSAPrivateKeyEncrypted(t, "password") + + c := fmt.Sprintf(` [securityadmin] - account='TEST_ACCOUNT' - user='TEST_USER' - password='abcd1234' - role='SECURITYADMIN' - ` - configPath := testFile(t, "config", []byte(c)) + account='account' + accountname='accountname' + organizationname='organizationname' + user='user' + password='password' + host='host' + warehouse='warehouse' + role='role' + clientip='1.1.1.1' + protocol='http' + passcode='passcode' + port=1 + passcodeinpassword=true + oktaurl='https://example.com' + clienttimeout=10 + jwtclienttimeout=20 + logintimeout=30 + requesttimeout=40 + jwtexpiretimeout=50 + externalbrowsertimeout=60 + maxretrycount=1 + authenticator='jwt' + insecuremode=true + ocspfailopen=true + token='token' + keepsessionalive=true + privatekey="""%s""" + privatekeypassphrase='%s' + disabletelemetry=true + validatedefaultparameters=true + clientrequestmfatoken=true + clientstoretemporarycredential=true + tracing='tracing' + tmpdirpath='.' + disablequerycontextcache=true + includeretryreason=true + disableconsolelogin=true + + [securityadmin.params] + foo = 'bar' + `, encryptedKey, "password") + configPath := testhelpers.TestFile(t, "config", []byte(c)) t.Run("with found profile", func(t *testing.T) { t.Setenv(snowflakeenvs.ConfigPath, configPath) config, err := ProfileConfig("securityadmin") require.NoError(t, err) - assert.Equal(t, "TEST_ACCOUNT", config.Account) - assert.Equal(t, "TEST_USER", config.User) - assert.Equal(t, "abcd1234", config.Password) - assert.Equal(t, "SECURITYADMIN", config.Role) + require.NotNil(t, config.PrivateKey) + + gotKey, err := x509.MarshalPKCS8PrivateKey(config.PrivateKey) + require.NoError(t, err) + gotUnencryptedKey := pem.EncodeToMemory( + &pem.Block{ + Type: "PRIVATE KEY", + Bytes: gotKey, + }, + ) + + assert.Equal(t, "organizationname-accountname", config.Account) + assert.Equal(t, "user", config.User) + assert.Equal(t, "password", config.Password) + assert.Equal(t, "warehouse", config.Warehouse) + assert.Equal(t, "role", config.Role) + assert.Equal(t, map[string]*string{"foo": Pointer("bar")}, config.Params) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.ValidateDefaultParameters) + assert.Equal(t, "1.1.1.1", config.ClientIP.String()) + assert.Equal(t, "http", config.Protocol) + assert.Equal(t, "host", config.Host) + assert.Equal(t, 1, config.Port) + assert.Equal(t, gosnowflake.AuthTypeJwt, config.Authenticator) + assert.Equal(t, "passcode", config.Passcode) + assert.Equal(t, true, config.PasscodeInPassword) + assert.Equal(t, "https://example.com", config.OktaURL.String()) + assert.Equal(t, 10*time.Second, config.ClientTimeout) + assert.Equal(t, 20*time.Second, config.JWTClientTimeout) + assert.Equal(t, 30*time.Second, config.LoginTimeout) + assert.Equal(t, 40*time.Second, config.RequestTimeout) + assert.Equal(t, 50*time.Second, config.JWTExpireTimeout) + assert.Equal(t, 60*time.Second, config.ExternalBrowserTimeout) + assert.Equal(t, 1, config.MaxRetryCount) + assert.Equal(t, true, config.InsecureMode) + assert.Equal(t, "token", config.Token) + assert.Equal(t, gosnowflake.OCSPFailOpenTrue, config.OCSPFailOpen) + assert.Equal(t, true, config.KeepSessionAlive) + assert.Equal(t, unencryptedKey, string(gotUnencryptedKey)) + assert.Equal(t, true, config.DisableTelemetry) + assert.Equal(t, "tracing", config.Tracing) + assert.Equal(t, ".", config.TmpDirPath) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.ClientRequestMfaToken) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.ClientStoreTemporaryCredential) + assert.Equal(t, true, config.DisableQueryContextCache) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.IncludeRetryReason) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.IncludeRetryReason) + assert.Equal(t, gosnowflake.ConfigBoolTrue, config.DisableConsoleLogin) }) t.Run("with not found profile", func(t *testing.T) { @@ -70,71 +156,157 @@ func TestProfileConfig(t *testing.T) { }) t.Run("with not found config", func(t *testing.T) { - dir, err := os.UserHomeDir() - require.NoError(t, err) - t.Setenv(snowflakeenvs.ConfigPath, dir) + filename := random.AlphaN(8) + t.Setenv(snowflakeenvs.ConfigPath, filename) config, err := ProfileConfig("orgadmin") - require.Error(t, err) + require.ErrorContains(t, err, fmt.Sprintf("could not load config file: open %s: no such file or directory", filename)) require.Nil(t, config) }) } func Test_MergeConfig(t *testing.T) { - createConfig := func(user string, password string, account string, region string) *gosnowflake.Config { - return &gosnowflake.Config{ - User: user, - Password: password, - Account: account, - Region: region, - } - } + oktaUrl1, err := url.Parse("https://example1.com") + require.NoError(t, err) + oktaUrl2, err := url.Parse("https://example2.com") + require.NoError(t, err) - t.Run("merge configs", func(t *testing.T) { - config1 := createConfig("user", "password", "account", "") - config2 := createConfig("user2", "", "", "region2") + config1 := &gosnowflake.Config{ + Account: "account1", + User: "user1", + Password: "password1", + Warehouse: "warehouse1", + Role: "role1", + ValidateDefaultParameters: 1, + Params: map[string]*string{ + "foo": Pointer("1"), + }, + ClientIP: net.ParseIP("1.1.1.1"), + Protocol: "protocol1", + Host: "host1", + Port: 1, + Authenticator: 1, + Passcode: "passcode1", + PasscodeInPassword: false, + OktaURL: oktaUrl1, + LoginTimeout: 1, + RequestTimeout: 1, + JWTExpireTimeout: 1, + ClientTimeout: 1, + JWTClientTimeout: 1, + ExternalBrowserTimeout: 1, + MaxRetryCount: 1, + InsecureMode: false, + OCSPFailOpen: 1, + Token: "token1", + KeepSessionAlive: false, + PrivateKey: random.GenerateRSAPrivateKey(t), + DisableTelemetry: false, + Tracing: "tracing1", + TmpDirPath: "tmpdirpath1", + ClientRequestMfaToken: gosnowflake.ConfigBoolFalse, + ClientStoreTemporaryCredential: gosnowflake.ConfigBoolFalse, + DisableQueryContextCache: false, + IncludeRetryReason: 1, + DisableConsoleLogin: gosnowflake.ConfigBoolFalse, + } - config := MergeConfig(config1, config2) + config2 := &gosnowflake.Config{ + Account: "account2", + User: "user2", + Password: "password2", + Warehouse: "warehouse2", + Role: "role2", + ValidateDefaultParameters: 1, + Params: map[string]*string{ + "foo": Pointer("2"), + }, + ClientIP: net.ParseIP("2.2.2.2"), + Protocol: "protocol2", + Host: "host2", + Port: 2, + Authenticator: 2, + Passcode: "passcode2", + PasscodeInPassword: true, + OktaURL: oktaUrl2, + LoginTimeout: 2, + RequestTimeout: 2, + JWTExpireTimeout: 2, + ClientTimeout: 2, + JWTClientTimeout: 2, + ExternalBrowserTimeout: 2, + MaxRetryCount: 2, + InsecureMode: true, + OCSPFailOpen: 2, + Token: "token2", + KeepSessionAlive: true, + PrivateKey: random.GenerateRSAPrivateKey(t), + DisableTelemetry: true, + Tracing: "tracing2", + TmpDirPath: "tmpdirpath2", + ClientRequestMfaToken: gosnowflake.ConfigBoolTrue, + ClientStoreTemporaryCredential: gosnowflake.ConfigBoolTrue, + DisableQueryContextCache: true, + IncludeRetryReason: gosnowflake.ConfigBoolTrue, + DisableConsoleLogin: gosnowflake.ConfigBoolTrue, + } - require.Equal(t, "user", config.User) - require.Equal(t, "password", config.Password) - require.Equal(t, "account", config.Account) - require.Equal(t, "region2", config.Region) - require.Equal(t, "", config.Role) + t.Run("base config empty", func(t *testing.T) { + config := MergeConfig(&gosnowflake.Config{}, config1) require.Equal(t, config1, config) - require.Equal(t, "user", config1.User) - require.Equal(t, "password", config1.Password) - require.Equal(t, "account", config1.Account) - require.Equal(t, "region2", config1.Region) - require.Equal(t, "", config1.Role) }) - t.Run("merge configs inverted", func(t *testing.T) { - config1 := createConfig("user", "password", "account", "") - config2 := createConfig("user2", "", "", "region2") - - config := MergeConfig(config2, config1) + t.Run("merge config empty", func(t *testing.T) { + config := MergeConfig(config1, &gosnowflake.Config{}) - require.Equal(t, "user2", config.User) - require.Equal(t, "password", config.Password) - require.Equal(t, "account", config.Account) - require.Equal(t, "region2", config.Region) - require.Equal(t, "", config.Role) + require.Equal(t, config1, config) + }) - require.Equal(t, config2, config) - require.Equal(t, "user2", config2.User) - require.Equal(t, "password", config2.Password) - require.Equal(t, "account", config2.Account) - require.Equal(t, "region2", config2.Region) - require.Equal(t, "", config2.Role) + t.Run("both configs filled - base config takes precedence", func(t *testing.T) { + config := MergeConfig(config1, config2) + require.Equal(t, config1, config) }) } -func testFile(t *testing.T, filename string, dat []byte) string { - t.Helper() - path := filepath.Join(t.TempDir(), filename) - err := os.WriteFile(path, dat, 0o600) - require.NoError(t, err) - return path +func Test_toAuthenticationType(t *testing.T) { + type test struct { + input string + want gosnowflake.AuthType + } + + valid := []test{ + // Case insensitive. + {input: "snowflake", want: gosnowflake.AuthTypeSnowflake}, + + // Supported Values. + {input: "SNOWFLAKE", want: gosnowflake.AuthTypeSnowflake}, + {input: "OAUTH", want: gosnowflake.AuthTypeOAuth}, + {input: "EXTERNALBROWSER", want: gosnowflake.AuthTypeExternalBrowser}, + {input: "OKTA", want: gosnowflake.AuthTypeOkta}, + {input: "JWT", want: gosnowflake.AuthTypeJwt}, + {input: "SNOWFLAKE_JWT", want: gosnowflake.AuthTypeJwt}, + {input: "TOKENACCESSOR", want: gosnowflake.AuthTypeTokenAccessor}, + {input: "USERNAMEPASSWORDMFA", want: gosnowflake.AuthTypeUsernamePasswordMFA}, + } + + invalid := []test{ + {input: ""}, + {input: "foo"}, + } + + for _, tc := range valid { + t.Run(tc.input, func(t *testing.T) { + got, err := ToAuthenticatorType(tc.input) + require.NoError(t, err) + require.Equal(t, tc.want, got) + }) + } + + for _, tc := range invalid { + t.Run(tc.input, func(t *testing.T) { + _, err := ToAuthenticatorType(tc.input) + require.Error(t, err) + }) + } } diff --git a/pkg/sdk/internal/client/client_test.go b/pkg/sdk/internal/client/client_test.go index 982e581856..a95036ff21 100644 --- a/pkg/sdk/internal/client/client_test.go +++ b/pkg/sdk/internal/client/client_test.go @@ -7,7 +7,6 @@ import ( "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/snowflakeenvs" "github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk" - "github.com/snowflakedb/gosnowflake" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) @@ -29,17 +28,3 @@ func TestNewClientWithoutInstrumentedSQL(t *testing.T) { assert.Contains(t, sql.Drivers(), "snowflake") }) } - -func TestNewClientWithDebugLoggingSetFromEnv(t *testing.T) { - t.Run("set gosnowflake driver logging to debug", func(t *testing.T) { - if os.Getenv(snowflakeenvs.GosnowflakeLogLevel) == "" { - t.Skipf("Skipping TestNewClientWithDebugLoggingSet, because %s is not set", snowflakeenvs.GosnowflakeLogLevel) - } - - config := sdk.DefaultConfig() - _, err := sdk.NewClient(config) - require.NoError(t, err) - - assert.Equal(t, "debug", gosnowflake.GetLogger().GetLogLevel()) - }) -} diff --git a/pkg/sdk/sweepers_test.go b/pkg/sdk/sweepers_test.go index 49ce33c9dc..a9269a2337 100644 --- a/pkg/sdk/sweepers_test.go +++ b/pkg/sdk/sweepers_test.go @@ -104,6 +104,9 @@ func Test_Sweeper_NukeStaleObjects(t *testing.T) { }) // TODO [SNOW-867247]: nuke stale objects (e.g. created more than 2 weeks ago) + + // TODO [SNOW-867247]: nuke external oauth integrations because of errors like + // Error: 003524 (22023): SQL execution error: An integration with the given issuer already exists for this account } // TODO [SNOW-867247]: generalize nuke methods (sweepers too) diff --git a/pkg/testhelpers/helpers.go b/pkg/testhelpers/helpers.go index 298658a63c..6bcd26ce6d 100644 --- a/pkg/testhelpers/helpers.go +++ b/pkg/testhelpers/helpers.go @@ -1,25 +1,18 @@ package testhelpers import ( - "database/sql" + "os" "testing" - sqlmock "github.com/DATA-DOG/go-sqlmock" "github.com/stretchr/testify/require" ) -func WithMockDb(t *testing.T, f func(*sql.DB, sqlmock.Sqlmock)) { +func TestFile(t *testing.T, filename string, data []byte) string { t.Helper() - r := require.New(t) - db, mock, err := sqlmock.New() - r.NoError(err) - defer db.Close() + f, err := os.CreateTemp(t.TempDir(), filename) + require.NoError(t, err) - // Because we are using TypeSet not TypeList, order is non-deterministic. - mock.MatchExpectationsInOrder(false) - - f(db, mock) - if err := mock.ExpectationsWereMet(); err != nil { - t.Errorf("there were unfulfilled expectations: %s", err) - } + err = os.WriteFile(f.Name(), data, 0o600) + require.NoError(t, err) + return f.Name() } diff --git a/pkg/testhelpers/mock/mock.go b/pkg/testhelpers/mock/mock.go new file mode 100644 index 0000000000..8d72aa8eda --- /dev/null +++ b/pkg/testhelpers/mock/mock.go @@ -0,0 +1,25 @@ +package mock + +import ( + "database/sql" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/stretchr/testify/require" +) + +func WithMockDb(t *testing.T, f func(*sql.DB, sqlmock.Sqlmock)) { + t.Helper() + r := require.New(t) + db, mock, err := sqlmock.New() + r.NoError(err) + defer db.Close() + + // Because we are using TypeSet not TypeList, order is non-deterministic. + mock.MatchExpectationsInOrder(false) + + f(db, mock) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled expectations: %s", err) + } +} diff --git a/templates/index.md.tmpl b/templates/index.md.tmpl index 0aade73a3a..11f3ba84bb 100644 --- a/templates/index.md.tmpl +++ b/templates/index.md.tmpl @@ -17,6 +17,8 @@ Coverage is focused on part of Snowflake related to access control. ## Example Provider Configuration +This is an example configuration of the provider in `main.tf` in a configuration directory. More examples are provided [below](#order-precedence). + {{tffile "examples/provider/provider.tf"}} ## Configuration Schema @@ -36,7 +38,7 @@ The Snowflake provider support multiple ways to authenticate: * Private Key * Config File -In all cases account and username are required. +In all cases `organization_name`, `account_name` and `user` are required. ### Keypair Authentication Environment Variables @@ -113,30 +115,72 @@ export SNOWFLAKE_USER='...' export SNOWFLAKE_PASSWORD='...' ``` -### Config File +## Order Precedence -If you choose to use a config file, the optional `profile` attribute specifies the profile to use from the config file. If no profile is specified, the default profile is used. The Snowflake config file lives at `~/.snowflake/config` and uses [TOML](https://toml.io/) format. You can override this location by setting the `SNOWFLAKE_CONFIG_PATH` environment variable. If no username and account are specified, the provider will fall back to reading the config file. +Currently, the provider can be configured in three ways: +1. In a Terraform file located in the Terraform module with other resources. -```shell +Example content of the Terraform file configuration: + +```terraform +provider "snowflake" { + organization_name = "..." + account_name = "..." + username = "..." + password = "..." +} +``` + +2. In environmental variables (envs). This is mainly used to provide sensitive values. + + +```bash +export SNOWFLAKE_USER="..." +export SNOWFLAKE_PRIVATE_KEY_PATH="~/.ssh/snowflake_key" +``` + +3. In a TOML file (default in ~/.snowflake/config). Notice the use of different profiles. The profile name needs to be specified in the Terraform configuration file in `profile` field. When this is not specified, `default` profile is loaded. +When a `default` profile is not present in the TOML file, it is treated as "empty", without failing. + +Example content of the Terraform file configuration: + +```terraform +provider "snowflake" { + profile = "default" +} +``` + +Example content of the TOML file configuration: + +```toml [default] -account='TESTACCOUNT' -user='TEST_USER' -password='hunter2' +organizationname='organization_name' +accountname='account_name' +user='user' +password='password' role='ACCOUNTADMIN' -[securityadmin] -account='TESTACCOUNT' -user='TEST_USER' -password='hunter2' -role='SECURITYADMIN' +[secondary_test_account] +organizationname='organization_name' +accountname='account2_name' +user='user' +password='password' +role='ACCOUNTADMIN' ``` -## Order Precedence +Not all fields must be configured in one source; users can choose which fields are configured in which source. +Provider uses an established hierarchy of sources. The current behavior is that for each field: +1. Check if it is present in the provider configuration. If yes, use this value. If not, go to step 2. +1. Check if it is present in the environment variables. If yes, use this value. If not, go to step 3. +1. Check if it is present in the TOML config file (specifically, use the profile name configured in one of the steps above). If yes, use this value. If not, the value is considered empty. + +An example TOML file contents: + +{{ codefile "toml" "examples/additional/provider_config_toml.MD" | trimspace }} + +An example terraform configuration file equivalent: -The Snowflake provider will use the following order of precedence when determining which credentials to use: -1) Provider Configuration -2) Environment Variables -3) Config File +{{ codefile "terraform" "examples/additional/provider_config_tf.MD" | trimspace }} {{ index (split (codefile "" "examples/additional/deprecated_resources.MD") "```") 1 | trimspace }}