Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-802269: Spark parity #10

Open
sfc-gh-mrojas opened this issue Mar 8, 2023 · 0 comments
Open

SNOW-802269: Spark parity #10

sfc-gh-mrojas opened this issue Mar 8, 2023 · 0 comments

Comments

@sfc-gh-mrojas
Copy link
Collaborator

We have found all these methods as gaps to achieve parity and lower the entry-bar
<style type="text/css"></style>

Session.execute     Executes the input SQL query text in Snowflake
Session.executeUpdate     Executes the SQL update query in Snowflake
Session.time     Executes some code block and prints to stdout the time taken to execute the block.
SessionBuilder.from_snowsql     sets the connection from the snowsql config files
SessionBuilder.from_env     utility method that sets up the connection properties from SNOW_xxx or SNOWSQL_xxx environment variables if they are set
Column.as(symbol: Symbol)     Function that gives the column an alias using a symbol.
Column.bitwiseAND(other: Any)     Function that computes bitwise AND of this expression with another expression.
Column.bitwiseOR(other: Any)     Function that computes bitwise OR of this expression with another expression.
Column.bitwiseXOR(other: Any)     Function that computes bitwise XOR of this expression with another expression.
Column.cast(to: String)     Function that casts the column to a different data type, using the canonical string representation of the type.
Column.contains(other: Any)     Function that returns a boolean column based on a pattern match.
Column.contains(expr: String)     Function that validates if the values of the column contain the value from the parameter expression.
Column.eqNullSafe(other: Any)     Function that performs quality test that is safe for null values.
Column.getField(fieldName: String)     Function that gets a value by field name or key in a object.
Column.getItem(key: Any)     Function that gets an item at position ordinal out of an array, or gets a value by key in a object.
Column.isNaN()     Function that returns True if the current expression is NaN.
Column.isNotNull()     Function that validates if the values of the column are not null.
Column.isNull()     Function that validates if the values of the column are null.
Column.isin(strings: String*)     Function that validates if the value of the column is within the list of strings from parameter.
Column.like(literal: String)     Function that returns a boolean column based on a match.
Column.notEqual(other: Any)     Function that returns the result of the comparison of two columns.
Column.not_isin(strings: String*)     Function that validates if the value of the column is not within the list of strings from parameter.
Column.regexp_replace(pattern: String, replacement: String)     Function that replaces column's values according to the regex pattern and replacement value parameters.
Column.rlike(literal: String)     Function that returns a boolean column based on a regex match.
Column.startsWith(expr: String)     Function that validates if the values of the column start with the parameter expression.
Column.substr(pos: Int, len: Int)     Function that returns the portion of the string or binary value str, starting from the character/byte specified by pos, with limited length.
Column.substr(pos: Column, len: Column)     Function that returns the portion of the string or binary value str, starting from the character/byte specified by pos, with limited length.
       
array colName: String, colNames: String* Column Wrapper for Snowflake built-in array function.
array c: Column* Column Wrapper for Snowflake built-in array function.
asc colname: String Column Function to convert column name into column and order in an ascending manner.
base64 col: Column Column Computes the BASE64 encoding of a column.
bround colName: Column Column Implementation for Spark bround.
bround colName: Column, scale: Int Column Implementation for Spark bround.
cbrt columnName: String Column Wrapper for Spark CBRT(String) function.
cbrt e: Column Column Wrapper for Spark CBRT(Column) function.
collect_list s: String Column Wrapper for Snowflake built-in collect_list function.
collect_list c: Column Column Wrapper for Snowflake built-in collect_list function.
collect_set e: String Column Aggregate function: returns a set of objects with duplicate elements eliminated.
collect_set e: Column Column Aggregate function: returns a set of objects with duplicate elements eliminated.
conv c: Column, fromBase: Int, toBase: Int Column Wrapper for Snowflake built-in conv function.
date_add start: Column, days: Column Column Returns the date that is days days after start.
date_add start: Column, days: Int Column Returns the date that is days days after start.
date_format c: Column, s: String Column Wrapper for Snowflake built-in date_format function.
date_sub start: Column, days: Int Column Implementation for Spark date_sub.
desc c: String Column Function to convert column name into column and order in a descending manner.
element_at column: Column, index: T Column Returns element of array at given index in value if column is array.
expr s: String Column Function to convert a string into an SQL expression.
first c: Column, nulls: Boolean Column Wrapper for Snowflake built-in last function.
first s: String, nulls: Boolean Column Wrapper for Snowflake built-in first function.
first s: String Column Wrapper for Snowflake built-in first function.
first c: Column Column Wrapper for Snowflake built-in first function.
format_number x: Column, d: Int Column Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places with HALF_EVEN round mode, and returns the result as a string column.
format_string format: String, arguments: Column* Column Formats the arguments in printf-style and returns the result as a string column.
from_json e: Column Column Wrapper for Spark from_json column function
from_unixtime   Column, String Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
from_unixtime   Column Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the yyyy-MM-dd HH:mm:ss format.
from_utc_timestamp   Column, Column Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone.
from_utc_timestamp   Column, String Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone.
hex   Column Wrapper for Snowflake built-in hex_encode function.
instr   Column, String Locate the position of the first occurrence of substr column in the given string.
isnull   Column Wrapper for Snowflake built-in isnull function.
json_tuple   Column, String* Wrapper for Spark json_tuple.
last   Column, Boolean Wrapper for Snowflake built-in last function.
last   String, Boolean Wrapper for Snowflake built-in last function.
last   String Wrapper for Snowflake built-in last function.
last   Column Wrapper for Snowflake built-in last function.
@github-actions github-actions bot changed the title Spark parity SNOW-802269: Spark parity Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants