Skip to content

Commit

Permalink
updated changelog, cleaned up postgres-test-doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@ha2179
ha2179 committed Jun 28, 2023
1 parent 79e227f commit df7ba42
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 194 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
* [Doc] Document --persist-replace in API section (#539)
* [Fix] Fixed CI issue by updating `invalid_connection_string_duckdb` in `test_magic.py` (#631)
* [Fix] Refactored `ResultSet` to lazy loading (#470)
* [Doc] Added guide on running PostgreSQL integration tests ([#643](https://github.com/ploomber/jupysql/issues/643))

## 0.7.9 (2023-06-19)

Expand Down
202 changes: 8 additions & 194 deletions doc/integrations/postgres-test-doc.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
"id": "e5a346bc",
"metadata": {},
"source": [
"### Install PostgreSQL client\n",
"### Install Dependencies\n",
"\n",
"To connect to a PostgreSQL database from Python, you need a client library. We recommend using `psycopg2`, but there are others like `pg8000`, and `asyncpg`. JupySQL supports the [following connectors.](https://docs.sqlalchemy.org/en/14/dialects/postgresql.html#dialect-postgresql)"
]
Expand All @@ -80,7 +80,7 @@
"id": "9fb18c1e",
"metadata": {},
"source": [
"#### Installing `psycopg2`\n",
"#### Installing PostgreSQL client `psycopg2`\n",
"If you have `conda` installed, it is more reliable to use it:"
]
},
Expand Down Expand Up @@ -123,13 +123,13 @@
"id": "b3867555",
"metadata": {},
"source": [
"#### Install dockerctx\n",
"#### Installing dockerctx\n",
"\n",
"dockerctx is a context manager for managing the lifetime of a docker container. We'll be using this library as we're spinning up docker container to launch the database instance and running integration tests over it.\n",
"\n",
"\n",
"```{tip}\n",
"Make sure the earlier dependencies - psycopg2, pgspecial are installed before installing dockerctx, otherwise it might throw an error\n",
"Make sure the earlier dependencies - psycopg2, pgspecial are installed before installing dockerctx, otherwise as it dependes upon those libraries it might throw an error\n",
"```\n"
]
},
Expand All @@ -152,15 +152,6 @@
"If you have trouble getting it to work, [message us on Slack.](https://ploomber.io/community)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "441eb80b",
"metadata": {},
"source": [
"Now as we have the required dependencies, we will spin up a PostgreSQL docker instance to test our integration with the database."
]
},
{
"attachments": {},
"cell_type": "markdown",
Expand Down Expand Up @@ -190,15 +181,6 @@
"%pip install jupysql pandas pyarrow --quiet"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d4f83052",
"metadata": {},
"source": [
"You also need a PostgreSQL connector. Here's a list of [supported connectors.](https://docs.sqlalchemy.org/en/14/dialects/postgresql.html#dialect-postgresql) We recommend using `psycopg2`. The easiest way to install it is via:"
]
},
{
"attachments": {},
"cell_type": "markdown",
Expand All @@ -216,44 +198,8 @@
"id": "5586988f",
"metadata": {},
"source": [
"You also need Docker installed and running to start the PostgreSQL instance."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6c6092c1",
"metadata": {},
"source": [
"## Start PostgreSQL instance\n",
"\n",
"We fetch the official image, create a new database, and user (this will take 1-2 minutes). We are using credentials as defined in test config file at ```\"src/sql/_testing.py\"```"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "919dd542",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%bash\n",
"docker run --name postgres -e POSTGRES_DB=db \\\n",
" -e POSTGRES_USER=ploomber_app \\\n",
" -e POSTGRES_PASSWORD=ploomber_app_password \\\n",
" -p 5432:5432 -d postgres"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "dd336465",
"metadata": {},
"source": [
"```{important}\n",
"Check if docker container is up and running before proceeding forward\n",
"```{Important}\n",
"You need to make sure that [Docker Desktop](https://docs.docker.com/desktop/) is installed and running before moving forward as tests are configured to spin up new PostgreSQL container, run defined tests on it, and clean up.\n",
"```"
]
},
Expand All @@ -265,7 +211,7 @@
"source": [
"### Running Integration Tests\n",
"\n",
"Now, as we have PostgreSQL docker instance set up we can go ahead and run the integration tests. Integrations tests are located at ```\"src/tests/integration/\"``` we will run them using following commands."
"Now, as we have all the dependencies installed we are all set to run the integration tests. Integrations tests are located at ```\"src/tests/integration/\"``` we will run them using following commands."
]
},
{
Expand Down Expand Up @@ -333,7 +279,7 @@
"metadata": {},
"source": [
"```{Important}\n",
"The following error might show up if docker is not installed and/or the container is not running. Hence make sure to check if PostgrSQL docker instance is up and running before executing the pytest commands\n",
"The following error might show up if docker is not installed and/or the container is not properly running. Hence keep an eye that you see the newly spun DB container while running the tests.\n",
"```"
]
},
Expand All @@ -346,138 +292,6 @@
"```sh\n",
"docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "90e71436",
"metadata": {},
"source": [
"## Clean up\n",
"\n",
"To stop and remove the container:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "561a5f90",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n",
"fe9f1cf0d371 postgres \"docker-entrypoint.s…\" 6 seconds ago Up 5 seconds 0.0.0.0:5432->5432/tcp postgres\n"
]
}
],
"source": [
"! docker container ls"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fbc3261a",
"metadata": {},
"outputs": [],
"source": [
"%%capture out\n",
"! docker container ls --filter ancestor=postgres --quiet"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ee146b23",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Container id: \n"
]
}
],
"source": [
"container_id = out.stdout.strip()\n",
"print(f\"Container id: {container_id}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bd32d5f3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\"docker container stop\" requires at least 1 argument.\n",
"See 'docker container stop --help'.\n",
"\n",
"Usage: docker container stop [OPTIONS] CONTAINER [CONTAINER...]\n",
"\n",
"Stop one or more running containers\n"
]
}
],
"source": [
"! docker container stop {container_id}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6604151d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\"docker container rm\" requires at least 1 argument.\n",
"See 'docker container rm --help'.\n",
"\n",
"Usage: docker container rm [OPTIONS] CONTAINER [CONTAINER...]\n",
"\n",
"Remove one or more containers\n"
]
}
],
"source": [
"! docker container rm {container_id}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7fa95fc4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n"
]
}
],
"source": [
"! docker container ls"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da12a798",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit df7ba42

Please sign in to comment.