Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secrets best practices #5

Merged
merged 2 commits into from
Feb 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
NASA_API_KEY = <YOUR_NASA_API_KEY>
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,7 @@ target/
.mypy_cache/

# Jupyter Book
_build/
_build/

# python-dotenv
.env
27 changes: 18 additions & 9 deletions docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,26 @@
title:
author: World Bank Development Data Group
logo: docs/images/logo.png
only_build_toc_files: true

repository:
url: https://github.com/worldbank/template
branch: main
url: https://github.com/worldbank/template
branch: main

#######################################################################################
# HTML-specific settings
html:
home_page_in_navbar: false
extra_navbar: ""
use_edit_page_button: true
use_repository_button: true
use_issues_button: true
baseurl: https://worldbank.github.io/template
home_page_in_navbar: false
extra_navbar: ""
use_edit_page_button: true
use_repository_button: true
use_issues_button: true
baseurl: https://worldbank.github.io/template

only_build_toc_files: true
#######################################################################################
# Execution settings
execute:
execute_notebooks: force
allow_errors: true
exclude_patterns:
- notebooks/nasa-apod.ipynb
7 changes: 4 additions & 3 deletions docs/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ parts:
- caption: Examples
numbered: True
chapters:
- file: notebooks/world-bank-api.ipynb
- file: notebooks/world-bank-package.ipynb
- file: notebooks/world-bank-api.ipynb
- file: notebooks/world-bank-package.ipynb
- file: notebooks/nasa-apod.ipynb
- caption: Additional Resources
chapters:
- url: https://datapartnership.org
Expand All @@ -16,4 +17,4 @@ parts:
- url: https://www.worldbank.org/en/about/unit/unit-dec
title: World Bank DEC
- url: https://www.worldbank.org/en/research/dime
title: World Bank DIME
title: World Bank DIME
283 changes: 283 additions & 0 deletions notebooks/nasa-apod.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,283 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "90700fdc-fcc7-4e54-8c9e-449879d8c66d",
"metadata": {
"tags": []
},
"source": [
"# Securely Using API Keys\n",
"\n",
"> The following are (opinionated) best practices to store and use API keys in your source code. If you disagree, please consider [contributing](https://github.com/worldbank/template/issues/new/choose). "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ac0ed1c0",
"metadata": {},
"source": [
"## Environment Variables\n",
"\n",
"An [environment variable](https://en.wikipedia.org/wiki/Environment_variable) is a dynamic-named value that can be used to store information on a computer. For instance, an environment variable can be used to store settings and/or privileged information (e.g. API keys) on your local computer or server.\n",
"\n",
"To set a environment variable to a new value, in **Unix-like** systems, you must pass a `name` and a `value` pair as shown below in the terminal.\n",
"\n",
"```shell\n",
"export SECRET_API_KEY = <MY-SECRET_API_KEY>\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "080bd097-f128-4759-946d-793368230804",
"metadata": {
"tags": []
},
"source": [
"The `value` is acessible by the `name` without being exposed throughout the sytem. In particular, in [Python](https://python.org), the value can be retrieve as follows."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8d023b4e-496b-440c-91a7-199bceb44d7d",
"metadata": {
"tags": []
},
"source": [
"```python\n",
"secret_api_key = os.getenv(\"SECRET_API_KEY\")\n",
"```"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "54a99582-d509-4ab8-be42-ddb4921c0f45",
"metadata": {
"tags": []
},
"source": [
"Alternatively, it is customary to use a `.env` file to organize and load environments variables as needed. Packages such as [dotenv](https://www.npmjs.com/package/dotenv) and [python-dotenv](https://pypi.org/project/python-dotenv/) will automatically load environments variables for you from the `.env` file.\n",
"\n",
"```shell\n",
"source .env\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "3c0cc26a-2a99-49b0-a406-d57f31fff8ee",
"metadata": {},
"source": [
"With [Python](https://python.org),"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "960398ce-eadb-45e3-b160-53e6c9250dd0",
"metadata": {
"tags": [
"remove_output"
]
},
"outputs": [],
"source": [
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv()"
]
},
{
"cell_type": "markdown",
"id": "bda573d0-c877-42e6-8ee5-3000b780b4b7",
"metadata": {
"tags": []
},
"source": [
"With [Jupyter](https://jupyter.org),"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "2e700464-b50d-4b06-b0aa-afaafa17e68e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%load_ext dotenv\n",
"%dotenv"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "660db869",
"metadata": {},
"source": [
"The <span style=\"color:#3EACAD\">template</span> includes `.env.example` as an example; to use, simply rename it to `.env` and add your settings and secrets to it. Please note that `.env` **must** never be commited/versioned (for example, to GitHub) and **should** ignored on `.gitignore`. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "74484f7e",
"metadata": {},
"source": [
"```{tip}\n",
"While environments variables are a convenient way to minimize the security risk, it is important to emphasize secrets are still stored in plaintext in your computer. It is strongly recommended to use instead a secret manager, such as [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) or [1Password](https://developer.1password.com/docs/cli/secret-references).\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "14e89727",
"metadata": {},
"source": [
"## Astronomy Picture of the Day"
]
},
{
"cell_type": "markdown",
"id": "b4c0f3e8-7756-41bb-aa21-cc2eee5ff67f",
"metadata": {},
"source": [
"One of the most popular APIs is NASA's [Astronomy Picture of the Day](https://apod.nasa.gov/apod/astropix.html). Let's see in the following example how to use the NASA API with a secret API key."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "d797ef77-6ca4-4f9d-a1f8-abbfd9884b07",
"metadata": {
"tags": [
"hide-cell"
]
},
"outputs": [],
"source": [
"import os\n",
"\n",
"import httpx\n",
"from IPython.core.display import HTML\n",
"from IPython.display import Image"
]
},
{
"cell_type": "markdown",
"id": "ece37244",
"metadata": {},
"source": [
"First, you will have to [generate your API key](https://api.nasa.gov) and set up the environment variable `NASA_API_KEY` with its value. Now you are ready to use it in your code. For instance, in this example, we assign it to `api_key`. Please note that the value is never exposed and the notebook can be securely shared with anyone. "
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "7b914e66-7ae8-4d8b-9621-d6dc5ec49631",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"api_key = os.getenv(\"NASA_API_KEY\")"
]
},
{
"cell_type": "markdown",
"id": "10b5b12a",
"metadata": {},
"source": [
"Now, we are ready to make the request to the NASA API. According to the [documentation](https://github.com/nasa/apod-api#docs), the `api_key` is passed a parameter to the GET request. "
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "1990c3b9-f145-4c1f-bbb5-82f50801a011",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"async with httpx.AsyncClient() as client:\n",
" r = await client.get(\n",
" \"https://api.nasa.gov/planetary/apod\", params={\"api_key\": api_key}\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "e952e343",
"metadata": {},
"source": [
"Voilà!"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "bd1cb597-0144-43e8-bed8-12145a831a0c",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"<img src=\"https://apod.nasa.gov/apod/image/2302/jonesemberson1.jpg\"/>"
],
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Image(url=r.json()[\"hdurl\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8c7cb67e-c7ba-4ed3-bee0-36d303c1517d",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
},
"vscode": {
"interpreter": {
"hash": "ce6d896885f4e28373aa2ff7c44f136ed5a497e2abd203a79a632f5859ed7bb5"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}