Skip to content

Commit

Permalink
Snowflake compare annotation fix, add docs (#344)
Browse files Browse the repository at this point in the history
* changes

* doc fix

* version bump
  • Loading branch information
rhaffar authored Oct 30, 2024
1 parent 1ea649a commit 223b61c
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 3 deletions.
2 changes: 1 addition & 1 deletion datacompy/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
Then extended to carry that functionality over to Spark Dataframes.
"""

__version__ = "0.14.1"
__version__ = "0.14.2"

import platform
from warnings import warn
Expand Down
2 changes: 1 addition & 1 deletion datacompy/snowflake.py
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ def _validate_dataframe(self, df_name: str, index: str) -> None:
The "index" of the dataframe - df1 or df2.
"""
df = getattr(self, index)
if not isinstance(df, "sp.DataFrame"):
if not isinstance(df, sp.DataFrame):
raise TypeError(f"{df_name} must be a valid sp.Dataframe")

# force all columns to be non-case-sensitive
Expand Down
20 changes: 19 additions & 1 deletion docs/source/developer_instructions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,24 @@ Run ``python -m pytest`` to run all unittests defined in the subfolder
`pytest-runner <https://pypi.python.org/pypi/pytest-runner>`_.


Snowflake testing
-----------------
Testing the Snowflake compare requires the use of a Snowflake cluster, as Snowflake does not support local running.
This means that Snowflake tests do not get run in CICD, and changes to the Snowflake Compare must be validated by
the process of running these tests locally.

Note that you must have the following environment variables set in order to instantiate a Snowflake Connection (for testing purposes):

- "SF_ACCOUNT": with your SF account
- "SF_UID": with your SF username
- "SF_PWD": with your SF password
- "SF_WAREHOUSE": with your desired SF warehouse
- "SF_DATABASE": with a valid database with which you have access
- "SF_SCHEMA": with a valid schema belonging to the provided database

Once these are set, you are free to run the suite of Snowflake tests.


Management of Requirements
--------------------------

Expand Down Expand Up @@ -130,4 +148,4 @@ Finally upload to PyPi::
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

# real pypi
twine upload dist/*
twine upload dist/*

0 comments on commit 223b61c

Please sign in to comment.