Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: moving tutorials in the same directory #950

Merged
merged 1 commit into from
Jul 30, 2024
+897 −827
Merged

docs: moving tutorials in the same directory #950

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
763 changes: 0 additions & 763 deletions docs/application-tutorial/sha256.ipynb
View file
Open in desktop

This file was deleted.

12 changes: 6 additions & 6 deletions docs/tutorials/see-all-tutorials.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@

#### Code examples on GitHub

* [Key value database](../application-tutorial/key_value_database.ipynb)
* [SHA-256 ](../application-tutorial/sha256.ipynb)
* [Game of Life](../../frontends/concrete-python/examples/game_of_life/game_of_life.md)
* [XOR distance](../../frontends/concrete-python/examples/xor_distance/xor_distance.md)
* [SHA1 with Modules](../../frontends/concrete-python/examples/sha1/sha1.md)
* [Levenshtein distance with Modules](../../frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.md)
* [Key value database](../../frontends/concrete-python/examples/key_value_database/key_value_database.ipynb)
* [SHA-256 ](../../frontends/concrete-python/examples/sha256/sha256.ipynb)
umut-sahin marked this conversation as resolved.
Show resolved Hide resolved
* [Game of Life](../../frontends/concrete-python/examples/game_of_life/README.md)
* [XOR distance](../../frontends/concrete-python/examples/xor_distance/README.md)
* [SHA1 with Modules](../../frontends/concrete-python/examples/sha1/README.md)
* [Levenshtein distance with Modules](../../frontends/concrete-python/examples/levenshtein_distance/README.md)

#### Blog tutorials

Expand Down
0 ...hon/examples/game_of_life/game_of_life.md → ...te-python/examples/game_of_life/README.md
View file
Open in desktop
File renamed without changes.
0 ...amples/key-value-database/dynamic-size.py → ...amples/key_value_database/dynamic-size.py
View file
Open in desktop
File renamed without changes.
105 changes: 47 additions & 58 deletions ...ication-tutorial/key_value_database.ipynb → ...y_value_database/key_value_database.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"\n",
"This is an interactive tutorial of an Encrypted Key Value Database. The database allows for three operations, **Insert, Replace, and Query**. All the operations are implemented as fully-homomorphic encrypted circuits.\n",
"\n",
"In `examples/key-value-database/`, you will find the following files:\n",
"In `frontends/concrete-python/examples/key_value_database/`, you will find the following files:\n",
"\n",
"- `static-size.py`: This file contains a static size database implementation, meaning that the number of entries is given as a parameter at the beginning.\n",
"- `dynamic-size.py`: This file contains a dynamic size database implementation, meaning that the database starts as a zero entry database, and is grown as needed.\n",
Expand Down Expand Up @@ -62,7 +62,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# The number of entries in the database\n",
"NUMBER_OF_ENTRIES = 5\n",
"# The number of bits in each chunk\n",
Expand Down Expand Up @@ -94,7 +93,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Key and Value size must be a multiple of chunk size\n",
"assert KEY_SIZE % CHUNK_SIZE == 0\n",
"assert VALUE_SIZE % CHUNK_SIZE == 0\n",
Expand Down Expand Up @@ -123,7 +121,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Indexers for each part of the state\n",
"FLAG = 0\n",
"KEY = slice(1, 1 + NUMBER_OF_KEY_CHUNKS)\n",
Expand Down Expand Up @@ -166,13 +163,15 @@
"source": [
"def encode(number: int, width: int) -> np.array:\n",
" binary_repr = np.binary_repr(number, width=width)\n",
" blocks = [binary_repr[i:i+CHUNK_SIZE] for i in range(0, len(binary_repr), CHUNK_SIZE)]\n",
" blocks = [binary_repr[i : i + CHUNK_SIZE] for i in range(0, len(binary_repr), CHUNK_SIZE)]\n",
" return np.array([int(block, 2) for block in blocks])\n",
"\n",
"\n",
"# Encode a number with the key size\n",
"def encode_key(number: int) -> np.array:\n",
" return encode(number, width=KEY_SIZE)\n",
"\n",
"\n",
"# Encode a number with the value size\n",
"def encode_value(number: int) -> np.array:\n",
" return encode(number, width=VALUE_SIZE)"
Expand Down Expand Up @@ -202,7 +201,7 @@
"def decode(encoded_number: np.array) -> int:\n",
" result = 0\n",
" for i in range(len(encoded_number)):\n",
" result += 2**(CHUNK_SIZE*i) * encoded_number[(len(encoded_number) - i) - 1]\n",
" result += 2 ** (CHUNK_SIZE * i) * encoded_number[(len(encoded_number) - i) - 1]\n",
" return result"
]
},
Expand All @@ -224,10 +223,10 @@
"outputs": [],
"source": [
"def keep_selected(value, selected):\n",
" if selected:\n",
" return value\n",
" else:\n",
" return 0"
" if selected:\n",
" return value\n",
" else:\n",
" return 0"
]
},
{
Expand Down Expand Up @@ -269,8 +268,8 @@
"outputs": [],
"source": [
"def keep_selected_using_lut(value, selected):\n",
" packed = (2 ** CHUNK_SIZE) * selected + value\n",
" return keep_selected_lut[packed]"
" packed = (2**CHUNK_SIZE) * selected + value\n",
" return keep_selected_lut[packed]"
]
},
{
Expand Down Expand Up @@ -308,7 +307,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Insert a key value pair into the database\n",
"# - state: The state of the database\n",
"# - key: The key to insert\n",
Expand All @@ -334,11 +332,11 @@
" # | 1 | 1 | -> Used, skip\n",
" packed_flag_and_found = (found * 2) + flags[i]\n",
" # Use the packed flag and found bit to determine if the entry is unused\n",
" is_selected = (packed_flag_and_found == 0)\n",
" is_selected = packed_flag_and_found == 0\n",
"\n",
" # Update the selection array\n",
" selection[i] = is_selected\n",
" # Update the found bit, so all entries will be \n",
" # Update the found bit, so all entries will be\n",
" # skipped after the first unused entry is found\n",
" found += is_selected\n",
"\n",
Expand All @@ -352,12 +350,12 @@
"\n",
" # Create a packed selection and key array\n",
" # This array is used to update the key of the selected entry\n",
" packed_selection_and_key = (selection * (2 ** CHUNK_SIZE)) + key\n",
" packed_selection_and_key = (selection * (2**CHUNK_SIZE)) + key\n",
" key_update = keep_selected_lut[packed_selection_and_key]\n",
"\n",
" # Create a packed selection and value array\n",
" # This array is used to update the value of the selected entry\n",
" packed_selection_and_value = selection * (2 ** CHUNK_SIZE) + value\n",
" packed_selection_and_value = selection * (2**CHUNK_SIZE) + value\n",
" value_update = keep_selected_lut[packed_selection_and_value]\n",
"\n",
" # Update the state update array with the key and value update arrays\n",
Expand Down Expand Up @@ -391,7 +389,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Replace the value of a key in the database\n",
"# If the key is not in the database, nothing happens\n",
"# If the key is in the database, the value is replaced\n",
Expand All @@ -405,8 +402,6 @@
" keys = state[:, KEY]\n",
" values = state[:, VALUE]\n",
"\n",
" \n",
"\n",
" # Create an equal_rows array\n",
" # This array is used to select all entries with the given key\n",
" # The equal_rows array is created by comparing the keys in the state\n",
Expand All @@ -415,7 +410,7 @@
" # keys = [[1, 0, 1, 0], [0, 1, 0, 1, 1]]\n",
" # key = [1, 0, 1, 0]\n",
" # equal_rows = [1, 0]\n",
" equal_rows = (np.sum((keys - key) == 0, axis=1) == NUMBER_OF_KEY_CHUNKS)\n",
" equal_rows = np.sum((keys - key) == 0, axis=1) == NUMBER_OF_KEY_CHUNKS\n",
bcm-at-zama marked this conversation as resolved.
Show resolved Hide resolved
"\n",
" # Create a selection array\n",
" # This array is used to select the entry to change the value of\n",
Expand All @@ -424,10 +419,10 @@
" # The reason for combining the equal_rows array with the flags array\n",
" # is to make sure that only used entries are selected\n",
" selection = (flags * 2 + equal_rows == 3).reshape((-1, 1))\n",
" \n",
"\n",
" # Create a packed selection and value array\n",
" # This array is used to update the value of the selected entry\n",
" packed_selection_and_value = selection * (2 ** CHUNK_SIZE) + value\n",
" packed_selection_and_value = selection * (2**CHUNK_SIZE) + value\n",
" set_value = keep_selected_lut[packed_selection_and_value]\n",
"\n",
" # Create an inverse selection array\n",
Expand All @@ -439,7 +434,7 @@
"\n",
" # Create a packed inverse selection and value array\n",
" # This array is used to keep the value of the entries that are not selected\n",
" packed_inverse_selection_and_values = inverse_selection * (2 ** CHUNK_SIZE) + values\n",
" packed_inverse_selection_and_values = inverse_selection * (2**CHUNK_SIZE) + values\n",
" kept_values = keep_selected_lut[packed_inverse_selection_and_values]\n",
"\n",
" # Update the values of the state with the new values\n",
Expand Down Expand Up @@ -470,7 +465,6 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Query the database for a key and return the value\n",
"# - state: The state of the database\n",
"# - key: The key to query\n",
Expand Down Expand Up @@ -500,7 +494,7 @@
"\n",
" # Create a packed selection and value array\n",
" # This array is used to get the value of the selected entry\n",
" packed_selection_and_values = selection * (2 ** CHUNK_SIZE) + values\n",
" packed_selection_and_values = selection * (2**CHUNK_SIZE) + values\n",
" value_selection = keep_selected_lut[packed_selection_and_values]\n",
"\n",
" # Sum the value selection array to get the value\n",
Expand Down Expand Up @@ -530,7 +524,7 @@
" A key-value database that uses fully homomorphic encryption circuits to store the data.\n",
" \"\"\"\n",
"\n",
" # The state of the database, it holds all the \n",
" # The state of the database, it holds all the\n",
" # keys and values as a table of entries\n",
" _state: np.ndarray\n",
"\n",
Expand All @@ -541,15 +535,15 @@
"\n",
" # Below is the initialization of the database.\n",
"\n",
" # First, we initialize the state, and provide the necessary input sets. \n",
" # In versions later than concrete-numpy.0.9.0, we can use the `direct circuit` \n",
" # functionality to define the bit-widths of encrypted values rather than using \n",
" # `input sets`. Input sets are used to determine the required bit-width of the \n",
" # encrypted values. Hence, we add the largest possible value in the database \n",
" # First, we initialize the state, and provide the necessary input sets.\n",
" # In versions later than concrete-numpy.0.9.0, we can use the `direct circuit`\n",
" # functionality to define the bit-widths of encrypted values rather than using\n",
" # `input sets`. Input sets are used to determine the required bit-width of the\n",
" # encrypted values. Hence, we add the largest possible value in the database\n",
" # to the input sets.\n",
"\n",
" # Within the initialization phase, we create the required configuration, \n",
" # compilers, circuits, and keys. Circuit and key generation phase is \n",
" # Within the initialization phase, we create the required configuration,\n",
" # compilers, circuits, and keys. Circuit and key generation phase is\n",
" # timed and printed in the output.\n",
"\n",
" def __init__(self):\n",
Expand All @@ -562,16 +556,16 @@
" # The input set for the query circuit\n",
" inputset_binary = [\n",
" (\n",
" np.zeros(STATE_SHAPE, dtype=np.int64), # state\n",
" np.ones(NUMBER_OF_KEY_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # key\n",
" np.zeros(STATE_SHAPE, dtype=np.int64), # state\n",
" np.ones(NUMBER_OF_KEY_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # key\n",
" )\n",
" ]\n",
" # The input set for the insert and replace circuits\n",
" inputset_ternary = [\n",
" (\n",
" np.zeros(STATE_SHAPE, dtype=np.int64), # state\n",
" np.ones(NUMBER_OF_KEY_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # key\n",
" np.ones(NUMBER_OF_VALUE_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # value\n",
" np.zeros(STATE_SHAPE, dtype=np.int64), # state\n",
" np.ones(NUMBER_OF_KEY_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # key\n",
" np.ones(NUMBER_OF_VALUE_CHUNKS, dtype=np.int64) * (2**CHUNK_SIZE - 1), # value\n",
" )\n",
" ]\n",
"\n",
Expand All @@ -591,18 +585,12 @@
" # - \"encrypted\": The input is encrypted\n",
" # - \"plain\": The input is not encrypted\n",
" insert_compiler = fhe.Compiler(\n",
" _insert_impl,\n",
" {\"state\": \"encrypted\", \"key\": \"encrypted\", \"value\": \"encrypted\"}\n",
" _insert_impl, {\"state\": \"encrypted\", \"key\": \"encrypted\", \"value\": \"encrypted\"}\n",
" )\n",
" replace_compiler = fhe.Compiler(\n",
" _replace_impl,\n",
" {\"state\": \"encrypted\", \"key\": \"encrypted\", \"value\": \"encrypted\"}\n",
" _replace_impl, {\"state\": \"encrypted\", \"key\": \"encrypted\", \"value\": \"encrypted\"}\n",
" )\n",
" query_compiler = fhe.Compiler(\n",
" _query_impl,\n",
" {\"state\": \"encrypted\", \"key\": \"encrypted\"}\n",
" )\n",
"\n",
" query_compiler = fhe.Compiler(_query_impl, {\"state\": \"encrypted\", \"key\": \"encrypted\"})\n",
"\n",
" ## Compile the circuits\n",
" # The circuits are compiled with the input set and the configuration\n",
Expand Down Expand Up @@ -659,10 +647,10 @@
" print(f\"(took {end - start:.3f} seconds)\")\n",
"\n",
" ### The Interface Functions\n",
" \n",
" # The following methods are used to interact with the database. \n",
" # They are used to insert, replace and query the database. \n",
" # The methods are implemented by encrypting the inputs, \n",
"\n",
" # The following methods are used to interact with the database.\n",
" # They are used to insert, replace and query the database.\n",
" # The methods are implemented by encrypting the inputs,\n",
" # running the circuit and decrypting the output.\n",
"\n",
" # Insert a key-value pair into the database\n",
Expand Down Expand Up @@ -703,16 +691,14 @@
" print()\n",
" print(f\"Querying...\")\n",
" start = time.time()\n",
" result = self._query_circuit.encrypt_run_decrypt(\n",
" self._state, encode_key(key)\n",
" )\n",
" result = self._query_circuit.encrypt_run_decrypt(self._state, encode_key(key))\n",
" end = time.time()\n",
" print(f\"(took {end - start:.3f} seconds)\")\n",
"\n",
" if result[0] == 0:\n",
" return None\n",
"\n",
" return decode(result[1:])\n"
" return decode(result[1:])"
]
},
{
Expand Down Expand Up @@ -992,10 +978,10 @@
"source": [
"# Define lower/upper bounds for the key\n",
"minimum_key = 1\n",
"maximum_key = 2 ** KEY_SIZE - 1\n",
"maximum_key = 2**KEY_SIZE - 1\n",
"# Define lower/upper bounds for the value\n",
"minimum_value = 1\n",
"maximum_value = 2 ** VALUE_SIZE - 1"
"maximum_value = 2**VALUE_SIZE - 1"
]
},
{
Expand Down Expand Up @@ -1076,6 +1062,9 @@
"metadata": {
"execution": {
"timeout": 10800
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
Expand Down
0 ...xamples/key-value-database/static-size.py → ...xamples/key_value_database/static-size.py
View file
Open in desktop
File renamed without changes.
0 ...enshtein_distance/levenshtein_distance.md → ...n/examples/levenshtein_distance/README.md
View file
Open in desktop
File renamed without changes.
0 ...nds/concrete-python/examples/sha1/sha1.md → ...s/concrete-python/examples/sha1/README.md
View file
Open in desktop
File renamed without changes.
Loading
Loading