Merge pull request #93 from dbt-msft/update-testing-docs

update contribution docs and add Python 3.10

.github/workflows/integration-tests-azure.yml

@@ -17,7 +17,7 @@ jobs:
       fail-fast: false
       max-parallel: 1
       matrix:
-        python_version: ["3.7", "3.8", "3.9"]
+        python_version: ["3.7", "3.8", "3.9", "3.10"]
     runs-on: ubuntu-latest
     container:
       image: ghcr.io/dbt-msft/dbt-sqlserver:${{ matrix.python_version }}

CONTRIBUTING.md

@@ -1,13 +1,19 @@
 # Development of the adapter
 
-Python 3.9 is used for developing the adapter. To get started, setup your environment as follows:
+The Synapse adapter uses the [dbt-sqlserver](https://github.com/dbt-msft/dbt-sqlserver) adapter underneath.
+This repository mostly contains a set of macros that override the behavior of dbt-sqlserver so that it works with Synapse.
 
-Create a virtual environment, pyenv is used in the example:
+Python 3.10 is used for developing the adapter. To get started, bootstrap your environment as follows:
+
+Create a virtual environment, [pyenv](https://github.com/pyenv/pyenv) is used in the example:
 
 ```shell
+pyenv install 3.7.13
+pyenv install 3.8.13
 pyenv install 3.9.12
-pyenv virtualenv 3.9.12 dbt-synapse
-pyenv activate dbt-synapse
+pyenv install 3.10.4
+pyenv virtualenv 3.10.4 dbt-synapse
+pyenv local dbt-synapse 3.9.12 3.8.13 3.7.13
 ```
 
 Install the development dependencies and pre-commit and get information about possible make commands:
@@ -17,31 +23,43 @@ make dev
 make help
 ```
 
+[Pre-commit](https://pre-commit.com/) helps us to maintain a consistent style and code quality across the entire project.
+After running `make dev`, pre-commit will automatically validate your commits and fix any formatting issues whenever possible.
+
 ## Testing
 
-### pytest-dbt-adapter
+The functional tests require a running Synapse Dedicated SQL Pool instance.
+You can configure the connection to this instance with the file `test.env` in the root of the project.
+You can use the provided `test.env.sample` as a base.
+
+```shell
+cp test.env.sample test.env
+```
 
-The package [pytest-dbt-adapter](https://github.com/dbt-labs/dbt-adapter-tests) is used for running tests against the adapter.
-However, this is no longer the recommended way to test adapters and we should into replacing this with [the recommended way to test new adapter](https://docs.getdbt.com/docs/contributing/testing-a-new-adapter)
+We use tox to isolate the environment for testing and for making it possible to test in multiple environments.
+You can use the following commands to run the unit and the functional tests respectively:
 
-### Tox
+```shell
+make functional
+```
 
-Running the unit tests:
+This will start tox and test in all environments. If you only want to run the tests in a specific environment, you can use the following commands:
 
 ```shell
-make test
+tox -e py310 -- -v tests/functional # will only functional run in Python 3.10
 ```
 
 ## CI/CD
 
-We use the Docker image from our [dbt-sqlserver](https://github.com/dbt-msft/dbt-sqlserver/blob/master/CONTRIBUTING.md) repo.
+We use Docker images that have all the things we need to test the adapter in the CI/CD workflows.
+The Dockerfile and image are part of the [dbt-sqlserver](https://github.com/dbt-msft/dbt-sqlserver) repository.
+
+All CI/CD pipelines are using GitHub Actions. The following pipelines are available:
 
-There is a Circle CI workflow with jobs that run the following tasks:
+* `integration-tests-azure`: runs the integration tests for Azure SQL Server.
+* `release-version`: publishes the adapter to PyPI.
 
-* Run the unit tests
-* Use the adapter to connect to a SQL Server Docker container
-* Run the pytest-dbt-adapter specs against a SQL Server Docker container
-* Use the adapter to connect to an Azure SQL Database with various options
+There is an additional [Pre-commit](https://pre-commit.ci/) pipeline that validates the code style.
 
 ### Azure integration tests
 
@@ -60,4 +78,8 @@ The following environment variables are available:
 ## Releasing a new version
 
 Make sure the version number is bumped in `__version__.py`. Then, create a git tag named `v<version>` and push it to GitHub.
-A CircleCI workflow will be triggered to build the package and push it to PyPI. 
+A GitHub Actions workflow will be triggered to build the package and push it to PyPI.
+
+Make sure that the dependency to dbt-sqlserver is bumped to a compatible version in `setup.py`.
+
+If you're releasing support for a new version of `dbt-core`, also bump the `dbt_version` in `setup.py`.

setup.py

@@ -6,13 +6,16 @@
 from setuptools import find_namespace_packages, setup
 from setuptools.command.install import install
 
+package_name = "dbt-synapse"
+authors_list = ["Nandan Hegde", "Chaerin Lee", "Alieu Sanneh", "Anders Swanson", "Sam Debruyn"]
+dbt_version = "1.1"
+dbt_sqlserver_requirement = "dbt-sqlserver>=1.1.0rc1,<1.2.0"
+description = """An Azure Synapse adapter plugin for dbt (data build tool)"""
+
 this_directory = os.path.abspath(os.path.dirname(__file__))
 with open(os.path.join(this_directory, "README.md")) as f:
     long_description = f.read()
 
-package_name = "dbt-synapse"
-authors_list = ["Nandan Hegde", "Chaerin Lee", "Alieu Sanneh", "Anders Swanson", "Sam Debruyn"]
-
 
 # get this from a separate file
 def _dbt_synapse_version():
@@ -26,9 +29,7 @@ def _dbt_synapse_version():
 
 
 package_version = _dbt_synapse_version()
-description = """An Azure Synapse adapter plugin for dbt (data build tool)"""
 
-dbt_version = "1.1"
 # the package version should be the dbt version, with maybe some things on the
 # ends of it. (0.18.1 vs 0.18.1a1, 0.18.1.1, ...)
 if not package_version.startswith(dbt_version):
@@ -65,7 +66,7 @@ def run(self):
     url="https://github.com/dbt-msft/dbt-synapse",
     packages=find_namespace_packages(include=["dbt", "dbt.*"]),
     include_package_data=True,
-    install_requires=["dbt-sqlserver>=1.1.0rc1,<1.2.0"],
+    install_requires=[dbt_sqlserver_requirement],
     cmdclass={
         "verify": VerifyVersionCommand,
     },

tox.ini

@@ -1,6 +1,6 @@
 [tox]
 skipsdist = True
-envlist = py{37,38,39}
+envlist = py{37,38,39,310}
 
 [testenv]
 deps =
@@ -12,6 +12,6 @@ allowlist_externals =
     /bin/sh
     az
 
-[testenv:py{37,38,39}-azure]
+[testenv:py{37,38,39,310}-azure]
 commands_pre =
     az login --service-principal --username="{env:AZURE_CLIENT_ID:}" --password="{env:AZURE_CLIENT_SECRET:}" --tenant="{env:AZURE_TENANT_ID:}"