Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize 2021-09-11-sql-dateadd.md #1244

Merged
merged 7 commits into from
Mar 30, 2022
Merged

Optimize 2021-09-11-sql-dateadd.md #1244

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0629ae4
Update 2021-09-11-sql-dateadd.md
johnblust Mar 17, 2022
5f890da
Merge branch 'current' into optimize-dateadd-across-data-warehouses
johnblust Mar 17, 2022
53c0c84
Fixed grammer errors
johnblust Mar 22, 2022
0206c1f
Update 2021-09-11-sql-dateadd.md
johnblust Mar 22, 2022
8fa2674
Merge branch 'current' into optimize-dateadd-across-data-warehouses
johnblust Mar 24, 2022
1be3506
Update 2021-09-11-sql-dateadd.md
johnblust Mar 30, 2022
cbe87ab
Merge branch 'current' into optimize-dateadd-across-data-warehouses
johnblust Mar 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 28 additions & 9 deletions website/blog/2021-09-11-sql-dateadd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "DATEADD Across Data Warehouses"
description: "Abstract away the need to look up the SQL dateadd function syntax every time you use it, by standardizing your syntax with dbt macros."
title: "DATEADD SQL Function Across Data Warehouses"
description: "DATEADD Function syntax varies across data warehouses. Learn how to standardize your syntax no matter the container."
slug: sql-dateadd

authors: david_krevitt
Expand All @@ -16,11 +16,20 @@ I’ve used the dateadd SQL function thousands of times.

I’ve googled the syntax of the dateadd SQL function all of those times except one, when I decided to hit the "are you feeling lucky" button and go for it.

In switching between SQL dialects (BigQuery, Postgres and Snowflake are my primaries), I can literally never remember the argument order (or exact function name) of dateadd.

This article will go over how the DATEADD function works, the nuances of using it across the major cloud warehouses, and how to standardize the syntax variances using dbt macro.

<!--truncate-->

In switching between SQL dialects (BigQuery, Postgres and Snowflake are my primaries), I can literally never remember the argument order (or exact function name) of dateadd.
## What is the DATEADD SQL Function?

The DATEADD function in SQL adds a time/date interval to a date and then returns the date. This allows you to add or subtract a certain period of time from a given start date.

Sounds simple enough, but this function lets you do some pretty useful things like calculating an estimated shipment date based on the ordered date.


## Differences in dateadd functions across platforms
## Differences in DATEADD syntax across data warehouse platforms

All of them accept the same rough parameters, in slightly different syntax and order:

Expand All @@ -30,23 +39,29 @@ All of them accept the same rough parameters, in slightly different syntax and o

The *functions themselves* are named slightly differently, which is common across SQL dialects.

### For example, in Snowflake…
### For example, the DATEADD function in Snowflake…

```sql
dateadd( {{ datepart }}, {{ interval }}, {{ from_date }} )
```

*Hour, minute and second are supported!*

### In BigQuery…
### The DATEADD Function in Databricks
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't believe dateadd in dbt_utils is compatible with databricks (looking at the databricks function docs for date_add: https://docs.databricks.com/sql/language-manual/functions/date_add.html). This is a good one to confirm w/ DX (likely Joel) though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joellabes What are your thoughts?

Some context: This post ran through how to use DATEADD in each warehouse so I decided to add to add a Databricks section to the content. We're not sure about compatibility with dbt_utils. If not, I should probably just remove the section for Databricks again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbt utils itself only supports BQ/SF/RS/Postgres, but other packages can implement overrides. That is done here for dateadd on spark (which powers Databricks, right?) https://github.com/dbt-labs/spark-utils/blob/main/macros/dbt_utils/cross_db_utils/dateadd.sql

So yes you can include Databricks in this.

Note that these will be moving to dbt core in v1.2: dbt-labs/dbt-core#4813

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, so I'll make a note of that in this section of the content. Thanks Joel!!


```sql
date_add( {{ startDate }}, {{ numDays }} )
```

### The DATEADD Function in BigQuery…

```sql
date_add( {{ from_date }}, INTERVAL {{ interval }} {{ datepart }} )
```

*Dateparts of less than a day (hour / minute / second) are not supported.*

### In Postgres...
### The DATEADD Function in Postgres...

Postgres doesn’t provide a dateadd function out of the box, so you’ve got to go it alone - but the syntax looks very similar to BigQuery’s function…

Expand All @@ -62,7 +77,7 @@ So I made this handy 2 x 2 matrix to help sort the differences out:

I am sorry - that’s just a blank 2x2 matrix. I've surrendered to just searching for the docs.

## The tiniest drag on quality of life
## Standardizing your DATEADD SQL syntax with a dbt macro

But couldn’t we be doing something better with those keystrokes, like typing out and then deleting a tweet?

Expand All @@ -84,7 +99,7 @@ Adding 1 month to today would look like...
>
> *TL;DR: dbt allows data practitioners to write code like software engineers, which in this case means not repeating yourself unnecessarily.*

## Compiling away your dateadd troubles
### Compiling away your DATEADD troubles

When we run dbt, the dateadd macro compiles your function into the SQL dialect of the warehouse adapter you’re running on—it’s running the same SQL you would’ve written yourself in your native query browser.

Expand Down Expand Up @@ -126,3 +141,7 @@ And it’s actually quite a simple 31-line macro ([source here](https://github.c

Enjoy! FYI I've used dateadd macro in dbt-utils on BigQuery, Postgres, Redshift and Snowflake, but it likely works across most other warehouses.

*Note: While `dbt_utils` doesn't support Databricks by default, you can use other packages that [implement overrides](https://docs.getdbt.com/reference/dbt-jinja-functions/dispatch#overriding-package-macros) as a workaround.*

*This [spark_utils package](https://github.com/dbt-labs/spark-utils/blob/main/macros/dbt_utils/cross_db_utils/dateadd.sql) can help you implement the override needed to add support for Databricks dateadd*