feat: Functions sdk update #3254

sfc-gh-asawicki · 2024-12-08T15:54:00Z

Update and test functions SDK as part of ongoing functions&procedures rework:

Regenerate SDK after adjusting defs
Wrap function definitions in $$
Wrap arguments in double quotes
Extract FunctionDetails out of DESCRIBE output rows
Add MetricLevel to the SDK
Adjust alter's structure (add set/unset)
Add missing fields to SHOW output
Use extended in for SHOW
Add/Generate assertions for function, function details, and function parameters
Add missing unit tests and adjust existing ones
Add missing integration tests and remove most of the existing ones or adjust them
Adjust existing resource

Additionally:

Allow simplified setup for local testing
Add helpers to interact with user's stage
Improve SDK generation (remove part of the unnecessary newlines - some are still left)

… details struct

github-actions · 2024-12-08T17:56:37Z

Integration tests failure for e97df58c923d269261474b7c544b08e03e9fff0b

github-actions · 2024-12-08T20:44:43Z

Integration tests failure for 503e095147e2c18268c2b8e4e132455b8c4aeb18

pkg/acceptance/bettertestspoc/assert/objectparametersassert/gen/object_parameters_def.go

pkg/acceptance/helpers/function_client.go

sfc-gh-jcieslak · 2024-12-09T09:37:38Z

pkg/sdk/functions_ext.go

+	Body                       *string // present for all function types (hidden when SECURE)
+	NullHandling               *string // present for all function types but SQL
+	Volatility                 *string // present for all function types but SQL
+	ExternalAccessIntegrations *string // list present for python, java, and scala


I'm guessing that in the next prs we are going to have concrete types instead of plain string values?

yes, most probably when adding the resources (this change is already inflated :( )

sfc-gh-jcieslak · 2024-12-09T09:42:02Z

pkg/sdk/functions_ext.go

+	} else {
+		*field = *d.Value
+	}
+	return nil


maybe this and the function below should be enclosed in the description_helpers where we would hold functions that help with more complicated descriptions to parse (?)

let's tackle this one we have more objects handled this way (for now there is no consistency, not only because the history but also because of the different columns returned in rows response by Snowflake - we have so many *property types in the SDK)

sfc-gh-jcieslak · 2024-12-09T10:00:44Z

pkg/sdk/functions_gen_test.go

 		opts.Comment = String("comment")
 		opts.Imports = []FunctionImport{
 			{
 				Import: "@udf_libs/echohandler.jar",
 			},
 		}
 		opts.Handler = "Echo.echoVarchar"
-		opts.FunctionDefinition = String("return x")
-		assertOptsValidAndSQLEquals(t, opts, `CREATE OR REPLACE TEMPORARY SECURE FUNCTION %s (x VARCHAR(100) DEFAULT 'test') COPY GRANTS RETURNS VARCHAR(100) NOT NULL LANGUAGE SCALA CALLED ON NULL INPUT IMMUTABLE RUNTIME_VERSION = '2.0' COMMENT = 'comment' IMPORTS = ('@udf_libs/echohandler.jar') HANDLER = 'Echo.echoVarchar' AS 'return x'`, id.FullyQualifiedName())
+		opts.FunctionDefinition = String(wrapFunctionDefinition("return x"))


make a ticket/leave todo in sql generator to support "dollar quoting" there

I will add it

pkg/sdk/common_types.go

pkg/sdk/functions_def.go

pkg/sdk/functions_impl_gen.go

sfc-gh-jmichalak · 2024-12-09T12:27:43Z

pkg/resources/function_acceptance_test.go

@@ -142,7 +142,7 @@ func TestAcc_Function_complex(t *testing.T) {
 					resource.TestCheckResourceAttr(resourceName, "comment", "Terraform acceptance test"),
 					resource.TestCheckResourceAttr(resourceName, "statement", statement),
 					resource.TestCheckResourceAttr(resourceName, "arguments.#", "1"),
-					resource.TestCheckResourceAttr(resourceName, "arguments.0.name", "D"),
+					resource.TestCheckResourceAttr(resourceName, "arguments.0.name", "d"),


Please mention the change in casing in the migration guide.

Will do in the PR with resource rework.

pkg/sdk/functions_ext.go

pkg/acceptance/helpers/function_client.go

pkg/acceptance/helpers/function_setup_helpers.go

pkg/acceptance/testdatatypes/testdatatypes.go

pkg/sdk/testint/functions_integration_test.go

github-actions · 2024-12-09T13:49:37Z

Integration tests failure for 7a392ecb0819a0dd6cf36439c7a8d32a93d53a3f

🤖 I have created a release *beep* *boop* --- ## [0.100.0](v0.99.0...v0.100.0) (2024-12-12) ### 🎉 **What's new:** * Account v1 readiness ([#3236](#3236)) ([5df33a8](5df33a8)) * Account v1 readiness generated files ([#3242](#3242)) ([3df59dd](3df59dd)) * Account v1 readiness resource ([#3252](#3252)) ([8f5698d](8f5698d)) * Add a new account roles data source ([#3257](#3257)) ([b3d6b9e](b3d6b9e)) * Add account data source ([#3261](#3261)) ([6087fc9](6087fc9)) * Add all other functions and procedures implementations ([#3275](#3275)) ([7a6f68d](7a6f68d)) * Basic functions implementation ([#3269](#3269)) ([6d4a103](6d4a103)) * Basic procedures implementation ([#3271](#3271)) ([933335f](933335f)) * Docs, test, and missing parameter ([#3280](#3280)) ([10517f3](10517f3)) * Functions and procedures schemas and generated sources ([#3262](#3262)) ([9b70f87](9b70f87)) * Functions sdk update ([#3254](#3254)) ([fc1eace](fc1eace)) * Handle missing fields in function and procedure ([#3273](#3273)) ([53e7a0a](53e7a0a)) * Procedures schemas and generated sources ([#3263](#3263)) ([211ad46](211ad46)) * Procedures sdk update ([#3255](#3255)) ([682606a](682606a)) * Rework account parameter resource ([#3264](#3264)) ([15aa9c2](15aa9c2)) * Rework data types ([#3244](#3244)) ([05ada91](05ada91)) * support table data type ([#3274](#3274)) ([13401d5](13401d5)) * Tag association v1 readiness ([#3210](#3210)) ([04f6d54](04f6d54)) * Test imports and small fixes ([#3276](#3276)) ([a712195](a712195)) * Unsafe execute v1 readiness ([#3266](#3266)) ([c4f1e8f](c4f1e8f)) * Use new data types in sql builder for functions and procedures ([#3247](#3247)) ([69f677a](69f677a)) ### 🔧 **Misc** * Add ShowByID filtering generation ([#3227](#3227)) ([548ec42](548ec42)) * Adress small task-related todos ([#3243](#3243)) ([40de9ae](40de9ae)) * Apply masking ([#3234](#3234)) ([c209a8a](c209a8a)) * fix missing references in toOpts and changes with newlines ([#3240](#3240)) ([246547f](246547f)) * function tests ([#3279](#3279)) ([5af6efb](5af6efb)) * Improve config builders ([#3207](#3207)) ([425787c](425787c)) * Revert to proper env ([#3238](#3238)) ([5d4ed3b](5d4ed3b)) * Use service user for ci ([#3228](#3228)) ([2fb50d7](2fb50d7)) ### 🐛 **Bug fixes:** * Make blocked_roles_field optional in OAuth security integrations ([#3267](#3267)) ([7197b57](7197b57)) * Minor fixes ([#3231](#3231)) ([1863bf6](1863bf6)) * Minor fixes 2 ([#3230](#3230)) ([73b7e74](73b7e74)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snowflake-release-please[bot] <105954990+snowflake-release-please[bot]@users.noreply.github.com>

sfc-gh-asawicki added 30 commits December 8, 2024 16:42

functions SDK update starts here

b5d5673

Add TODOs for create

4e6f7ad

Modify opts impl template in generator to reduce whitespace

70f697f

Regenerate to replace struct creation

42fb768

Update TODOs

5211af8

Remove single quotes from AS and add AGGREGATE for python (WIP)

8a94f3c

Add mapping from function detail row to properties and start function…

fec412b

… details struct

Add describe details func

a53949d

Generate snowflake object assertions for functions (in current shape)

2459bc3

Extract sample function definitions in different languages

32eea23

Extract identity function

9d48ab6

Add minimal set of creation tests to implement

d00002b

Remove TODO

3992634

Fix python procedure

14e5136

Go through existing TODOs in definition with manual tests

25f8ab8

Add target

bbac373

Returned details for different types of functions

2a1f707

Update a few comments with knowledge

3a00577

Check weird names

287ffc5

Set double quotes in argument names

a80bdc4

Add mapping to describe options

f82e434

Move TODOs before regeneration

a59b407

Update alter for functions

485726d

Regenerate functions (WIP - no test/compilation)

36f01e7

Reorganize, Rename, and Pass unit tests

db226a6

Test empty and non-empty secrets

a56e644

Fix secret reference

2b3ce4b

Finish describe mapping

0b7d036

Add parameters to each create

92b0f1d

Pass first integration tests

d5487fb

sfc-gh-asawicki added 11 commits December 8, 2024 16:42

Test create SQL without args

905ecf3

Add a TODO

cb51f3e

Start testing alters

9d5ff09

Add describe assertions

e24a38d

Pass function integration tests

1bb7685

Fix runtime version for scala and add missing show outputs

3b686f4

Solve TODOs

3f37bd9

Remove target path in appropriate tests

474e42d

Clean up TODOs

b5123b1

Run pre-push

086e202

Fix cleanup

e97df58

Fix function resource

503e095

sfc-gh-asawicki marked this pull request as ready for review December 9, 2024 11:43

sfc-gh-asawicki added 2 commits December 9, 2024 12:44

Merge branch 'main' into functions-sdk-update

383d219

Fix acceptance test

7a392ec

sfc-gh-asawicki requested review from sfc-gh-jcieslak and sfc-gh-jmichalak December 9, 2024 12:01

sfc-gh-jcieslak reviewed Dec 9, 2024

View reviewed changes

sfc-gh-jmichalak requested changes Dec 9, 2024

View reviewed changes

sfc-gh-jmichalak mentioned this pull request Dec 9, 2024

feat: Procedures sdk update #3255

Merged

sfc-gh-asawicki requested review from sfc-gh-jmichalak and sfc-gh-jcieslak December 9, 2024 16:56

sfc-gh-jmichalak approved these changes Dec 9, 2024

View reviewed changes

sfc-gh-asawicki merged commit fc1eace into main Dec 9, 2024
8 of 9 checks passed

sfc-gh-asawicki deleted the functions-sdk-update branch December 9, 2024 17:11

snowflake-release-please bot mentioned this pull request Dec 9, 2024

chore(main): release 0.100.0 #3237

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Functions sdk update #3254

feat: Functions sdk update #3254

sfc-gh-asawicki commented Dec 8, 2024 •

edited

Loading

github-actions bot commented Dec 8, 2024

github-actions bot commented Dec 8, 2024

sfc-gh-jcieslak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024 •

edited

Loading

sfc-gh-jcieslak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

sfc-gh-jcieslak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

sfc-gh-jmichalak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

github-actions bot commented Dec 9, 2024

feat: Functions sdk update #3254

feat: Functions sdk update #3254

Conversation

sfc-gh-asawicki commented Dec 8, 2024 • edited Loading

github-actions bot commented Dec 8, 2024

github-actions bot commented Dec 8, 2024

sfc-gh-jcieslak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024 • edited Loading

Choose a reason for hiding this comment

sfc-gh-jcieslak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-jcieslak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-jmichalak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 9, 2024

sfc-gh-asawicki commented Dec 8, 2024 •

edited

Loading

sfc-gh-asawicki Dec 9, 2024 •

edited

Loading