Skip to content

Commit

Permalink
Polars spreadsheet IO along with example and test case (#776)
Browse files Browse the repository at this point in the history
Adds materializer for polars and excel spreadsheets.

This means you can easily load and save from excel using the data save/loader framework
Hamilton has that wraps this polars code.

----
co-authored by: "swapdewalkar@gmail.com" <swapdewalkar@gmail.com>
  • Loading branch information
swapdewalkar authored Mar 25, 2024
1 parent 7a34277 commit 31da460
Show file tree
Hide file tree
Showing 6 changed files with 639 additions and 315 deletions.
2 changes: 1 addition & 1 deletion examples/dask/hello_world/business_logic.py
10 changes: 10 additions & 0 deletions examples/polars/materialization/my_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@
file="./df.avro",
combine=df_builder,
),
# materialize the dataframe to a spreadsheet file
to.spreadsheet(
dependencies=output_columns,
id="df_to_spreadsheet",
workbook="./df.xlsx",
worksheet="Sheet1",
combine=df_builder,
),
# materialize the dataframe to a database
to.database(
dependencies=output_columns,
Expand All @@ -93,6 +101,7 @@
"df_to_feather_build_result",
"df_to_json_build_result",
"df_to_avro_build_result",
"df_to_spreadsheet_build_result",
"df_to_database_build_result",
], # because combine is used, we can get that result here.
inputs=initial_columns,
Expand All @@ -102,4 +111,5 @@
print(additional_outputs["df_to_feather_build_result"])
print(additional_outputs["df_to_json_build_result"])
print(additional_outputs["df_to_avro_build_result"])
print(additional_outputs["df_to_spreadsheet_build_result"])
print(additional_outputs["df_to_database_build_result"])
Loading

0 comments on commit 31da460

Please sign in to comment.