Skip to content

Commit

Permalink
[DOCS]: Tidy up docs (#502)
Browse files Browse the repository at this point in the history 
## Describe the changes

This PR tidies up docs and updates golang build instructions
  • Loading branch information
@jeremyfelder
jeremyfelder authored May 6, 2024
1 parent 34f0212 commit 6134cfe
Show file tree
Hide file tree
Showing 26 changed files with 490 additions and 398 deletions.
89 changes: 52 additions & 37 deletions docs/docs/icicle/core.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,54 @@

ICICLE Core is a library written in C++/CUDA. All the ICICLE primitives are implemented within ICICLE Core.

The Core is split into logical modules that can be compiled into static libraries using different [strategies](#compilation-strategies). You can then [link](#linking) these libraries with your C++ project or write your own [bindings](#writing-new-bindings-for-icicle) for other programming languages. If you want to use ICICLE with existing bindings please refer to [Rust](/icicle/rust-bindings) / [Golang](/icicle/golang-bindings).
The Core is split into logical modules that can be compiled into static libraries using different [strategies](#compilation-strategies). You can then [link](#linking) these libraries with your C++ project or write your own [bindings](#writing-new-bindings-for-icicle) for other programming languages. If you want to use ICICLE with existing bindings please refer to the [Rust](/icicle/rust-bindings) or [Golang](/icicle/golang-bindings) bindings documentation.

## Supported curves, fields and operations

### Supported curves and operations

| Operation\Curve | [bn254](https://neuromancer.sk/std/bn/bn254) | [bls12-377](https://neuromancer.sk/std/bls/BLS12-377) | [bls12-381](https://neuromancer.sk/std/bls/BLS12-381) | [bw6-761](https://eprint.iacr.org/2020/351) | grumpkin |
| --- | :---: | :---: | :---: | :---: | :---: |
| [MSM][MSM_DOCS] | ✅ | ✅ | ✅ | ✅ | ✅ |
| G2 | ✅ | ✅ | ✅ | ✅ | ❌ |
| [NTT][NTT_DOCS] | ✅ | ✅ | ✅ | ✅ | ❌ |
| ECNTT | ✅ | ✅ | ✅ | ✅ | ❌ |
| [VecOps][VECOPS_CODE] | ✅ | ✅ | ✅ | ✅ | ✅ |
| [Polynomials][POLY_DOCS] | ✅ | ✅ | ✅ | ✅ | ❌ |
| [Poseidon](primitives/poseidon) | ✅ | ✅ | ✅ | ✅ | ✅ |
| [Merkle Tree](primitives/poseidon#the-tree-builder) | ✅ | ✅ | ✅ | ✅ | ✅ |

### Supported fields and operations

| Operation\Field | [babybear](https://eprint.iacr.org/2023/824.pdf) | [Stark252](https://docs.starknet.io/documentation/architecture_and_concepts/Cryptography/p-value/) |
| --- | :---: | :---: |
| [VecOps][VECOPS_CODE] | ✅ | ✅ |
| [Polynomials][POLY_DOCS] | ✅ | ✅ |
| [NTT][NTT_DOCS] | ✅ | ✅ |
| Extension Field | ✅ | ❌ |

### Supported hashes

| Hash | Sizes |
| --- | :---: |
| Keccak | 256, 512 |

## Compilation strategies

Most of the codebase is curve/field agnostic, which means it can be compiled for different curves and fields. When you build ICICLE Core you choose a single curve or field. If you need multiple curves or fields - you just compile ICICLE into multiple static libraries. It's that simple. Currently, the following choices are supported:
Most of the codebase is curve/field agnostic, which means it can be compiled for different curves and fields. When you build ICICLE Core you choose a single curve or field. If you need multiple curves or fields, you compile ICICLE once per curve or field that is needed. It's that simple. Currently, the following choices are supported:

- [Field mode](#compiling-for-a-field) - used for STARK fields like BabyBear / Mersenne / Goldilocks. Includes field arithmetic, NTT, Poseidon, Extension fields and other primitives.
- [Curve mode](#compiling-for-a-curve) - used for SNARK curves like BN254/ BLS curves / Grumpkin / etc. Curve mode is built upon field mode, so it includes everything that field does. It also includes curve operations / MSM / ECNTT / G2 and other curve-related primitives.
- [Field mode][COMPILE_FIELD_MODE] - used for STARK fields like BabyBear / Mersenne / Goldilocks. Includes field arithmetic, NTT, Poseidon, Extension fields and other primitives.
- [Curve mode][COMPILE_CURVE_MODE] - used for SNARK curves like BN254 / BLS curves / Grumpkin / etc. Curve mode is built upon field mode, so it includes everything that field does It also includes curve operations / MSM / ECNTT / G2 and other curve-related primitives.

:::info

If you only want to use curve's scalar/base field, you still need to go with a curve mode. You can disable MSM with [options](#compilation-options)
If you only want to use a curve's scalar or base field, you still need to use curve mode. You can disable MSM with [options](#compilation-options)

:::

### Compiling for a field

ICICLE supports the following STARK fields:
- [BabyBear](https://eprint.iacr.org/2023/824.pdf)

Field mode includes:
- [Field arithmetic](https://github.com/ingonyama-zk/icicle/blob/main/icicle/include/fields/field.cuh) - field multiplication, addition, subtraction
- [NTT](icicle/primitives/ntt) - FFT / iFFT
- [Poseidon Hash](icicle/primitives/poseidon)
- [Vector operations](https://github.com/ingonyama-zk/icicle/blob/main/icicle/include/vec_ops/vec_ops.cuh)
- [Polynomial](#) - structs and methods to work with polynomials

You can compile ICICLE for a STARK field using this command:
You can compile ICICLE for a field using this command:

```sh
cd icicle
Expand All @@ -38,24 +58,10 @@ cmake -DFIELD=<FIELD> -S . -B build
cmake --build build -j
```

Icicle Supports the following `<FIELD>` FIELDS:
- `babybear`

This command will output `libingo_field_<FIELD>.a` into `build/lib`.

### Compiling for a curve

ICICLE supports the following SNARK curves:
- [BN254](https://neuromancer.sk/std/bn/bn254)
- [BLS12-377](https://neuromancer.sk/std/bls/BLS12-377)
- [BLS12-381](https://neuromancer.sk/std/bls/BLS12-381)
- [BW6-761](https://eprint.iacr.org/2020/351)
- Grumpkin

Curve mode includes everything you can find in field mode with addition of:
- [MSM](icicle/primitives/msm) - MSM / Batched MSM
- [ECNTT](#)

:::note

Field related primitives will be compiled for the scalar field of the curve
Expand All @@ -81,31 +87,31 @@ There exist multiple options that allow you to customize your build or enable ad

#### EXT_FIELD

Used only in a [field mode](#compiling-for-a-field) to add Extension field into a build. Adds NTT for the extension field.
Used only in [field mode][COMPILE_FIELD_MODE] to add an Extension field. Adds all supported field operations for the extension field.

Default: `OFF`

Usage: `-DEXT_FIELD=ON`

#### G2

Used only in a [curve mode](#compiling-for-a-curve) to add G2 definitions into a build. Also adds G2 MSM.
Used only in [curve mode][COMPILE_CURVE_MODE] to add G2 definitions. Also adds G2 MSM.

Default: `OFF`

Usage: `-DG2=ON`

#### ECNTT

Used only in a [curve mode](#compiling-for-a-curve) to add ECNTT function into a build.
Used only in [curve mode][COMPILE_CURVE_MODE] to add ECNTT function.

Default: `OFF`

Usage: `-DECNTT=ON`

#### MSM

Used only in a [curve mode](#compiling-for-a-curve) to add MSM function into a build. As MSM takes a lot of time to build, you can disable it with this option to reduce compilation time.
Used only in [curve mode][COMPILE_CURVE_MODE] to add MSM function. As MSM takes a lot of time to build, you can disable it with this option to reduce compilation time.

Default: `ON`

Expand Down Expand Up @@ -149,14 +155,13 @@ To link ICICLE with your project you first need to compile ICICLE with options o

Refer to our [c++ examples](https://github.com/ingonyama-zk/icicle/tree/main/examples/c%2B%2B) for more info. Take a look at this [CMakeLists.txt](https://github.com/ingonyama-zk/icicle/blob/main/examples/c%2B%2B/msm/CMakeLists.txt#L22)


## Writing new bindings for ICICLE

Since ICICLE Core is written in CUDA / C++ its really simple to generate static libraries. These static libraries can be installed on any system and called by higher level languages such as Golang.

Static libraries can be loaded into memory once and used by multiple programs, reducing memory usage and potentially improving performance. They also allow you to separate functionality into distinct modules so your static library may need to compile only specific features that you want to use.

Let's review the [Golang bindings](golang-bindings.md) since its a pretty verbose example (compared to rust which hides it pretty well) of using static libraries. Golang has a library named `CGO` which can be used to link static libraries. Here's a basic example on how you can use cgo to link these libraries:
Let's review the [Golang bindings][GOLANG_BINDINGS] since its a pretty verbose example (compared to rust which hides it pretty well) of using static libraries. Golang has a library named `CGO` which can be used to link static libraries. Here's a basic example on how you can use cgo to link these libraries:

```go
/*
Expand All @@ -178,4 +183,14 @@ func main() {

The comments on the first line tell `CGO` which libraries to import as well as which header files to include. You can then call methods which are part of the static library and defined in the header file, `C.projective_from_affine_bn254` is an example.

If you wish to create your own bindings for a language of your choice we suggest you start by investigating how you can call static libraries.
If you wish to create your own bindings for a language of your choice we suggest you start by investigating how you can call static libraries.

<!-- Begin Links -->
[GOLANG_BINDINGS]: golang-bindings.md
[COMPILE_CURVE_MODE]: #compiling-for-a-curve
[COMPILE_FIELD_MODE]: #compiling-for-a-field
[NTT_DOCS]: primitives/ntt
[MSM_DOCS]: primitives/msm
[POLY_DOCS]: polynomials/overview
[VECOPS_CODE]: https://github.com/ingonyama-zk/icicle/blob/main/icicle/include/vec_ops/vec_ops.cuh
<!-- End Links -->
58 changes: 43 additions & 15 deletions docs/docs/icicle/golang-bindings.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Golang bindings

Golang bindings allow you to use ICICLE as a golang library.
The source code for all Golang libraries can be found [here](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang).
The source code for all Golang packages can be found [here](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang).

The Golang bindings are comprised of multiple packages.

[`core`](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/core) which defines all shared methods and structures, such as configuration structures, or memory slices.

[`cuda-runtime`](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/cuda_runtime) which defines abstractions for CUDA methods for allocating memory, initializing and managing streams, and `DeviceContext` which enables users to define and keep track of devices.

Each curve has its own package which you can find [here](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/curves). If your project uses BN254 you only need to install that single package named [`bn254`](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/curves/bn254).
Each supported curve, field, and hash has its own package which you can find in the respective directories [here](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang). If your project uses BN254 you only need to import that single package named [`bn254`](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/curves/bn254).

## Using ICICLE Golang bindings in your project

Expand All @@ -31,22 +31,30 @@ For a specific commit
go get github.com/ingonyama-zk/icicle@<commit_id>
```

To build the shared libraries you can run this script:
To build the shared libraries you can run [this](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/golang/build.sh) script:

```bash
./build.sh [-curve=<curve> | -field=<field>] [-cuda_version=<version>] [-g2] [-ecntt] [-devmode]
```sh
./build.sh [-curve=<curve>] [-field=<field>] [-hash=<hash>] [-cuda_version=<version>] [-g2] [-ecntt] [-devmode]

curve - The name of the curve to build or "all" to build all supported curves
field - The name of the field to build or "all" to build all supported fields
hash - The name of the hash to build or "all" to build all supported hashes
-g2 - Optional - build with G2 enabled
-ecntt - Optional - build with ECNTT enabled
-devmode - Optional - build in devmode
-help - Optional - Displays usage information
```
- **`curve`** - The name of the curve to build or "all" to build all curves
- **`field`** - The name of the field to build or "all" to build all fields
- **`g2`** - Optional - build with G2 enabled
- **`ecntt`** - Optional - build with ECNTT enabled
- **`devmode`** - Optional - build in devmode
- Usage can be displayed with the flag `-help`

:::note

If more than one curve or more than one field or more than one hash is supplied, the last one supplied will be built

:::

To build ICICLE libraries for all supported curves with G2 and ECNTT enabled.

```bash
./build.sh all -g2 -ecntt
./build.sh -curve=all -g2 -ecntt
```

If you wish to build for a specific curve, for example bn254, without G2 or ECNTT enabled.
Expand All @@ -73,11 +81,9 @@ import (
To run all tests, for all curves:

```bash
go test --tags=g2 ./... -count=1
go test ./... -count=1
```

If you dont want to include g2 tests then drop `--tags=g2`.

If you wish to run test for a specific curve:

```bash
Expand Down Expand Up @@ -106,3 +112,25 @@ func main() {
```

Replace `/path/to/shared/libs` with the actual path where the shared libraries are located on your system.

## Supported curves, fields and operations

### Supported curves and operations

| Operation\Curve | bn254 | bls12_377 | bls12_381 | bw6-761 | grumpkin |
| --- | :---: | :---: | :---: | :---: | :---: |
| MSM | ✅ | ✅ | ✅ | ✅ | ✅ |
| G2 | ✅ | ✅ | ✅ | ✅ | ❌ |
| NTT | ✅ | ✅ | ✅ | ✅ | ❌ |
| ECNTT | ✅ | ✅ | ✅ | ✅ | ❌ |
| VecOps | ✅ | ✅ | ✅ | ✅ | ✅ |
| Polynomials | ✅ | ✅ | ✅ | ✅ | ❌ |

### Supported fields and operations

| Operation\Field | babybear |
| --- | :---: |
| VecOps | ✅ |
| Polynomials | ✅ |
| NTT | ✅ |
| Extension Field | ✅ |
9 changes: 2 additions & 7 deletions docs/docs/icicle/golang-bindings/ecntt.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# ECNTT

### Supported curves

`bls12-377`, `bls12-381`, `bn254`

## ECNTT Method

The `ECNtt[T any]()` function performs the Elliptic Curve Number Theoretic Transform (EC-NTT) on the input points slice, using the provided dir (direction), cfg (configuration), and stores the results in the results slice.
Expand All @@ -12,14 +8,13 @@ The `ECNtt[T any]()` function performs the Elliptic Curve Number Theoretic Trans
func ECNtt[T any](points core.HostOrDeviceSlice, dir core.NTTDir, cfg *core.NTTConfig[T], results core.HostOrDeviceSlice) core.IcicleError
```

### Parameters:
### Parameters

- **`points`**: A slice of elliptic curve points (in projective coordinates) that will be transformed. The slice can be stored on the host or the device, as indicated by the `core.HostOrDeviceSlice` type.
- **`dir`**: The direction of the EC-NTT transform, either `core.KForward` or `core.KInverse`.
- **`cfg`**: A pointer to an `NTTConfig` object, containing configuration options for the NTT operation.
- **`results`**: A slice that will store the transformed elliptic curve points (in projective coordinates). The slice can be stored on the host or the device, as indicated by the `core.HostOrDeviceSlice` type.


### Return Value

- **`CudaError`**: A `core.IcicleError` value, which will be `core.IcicleErrorCode(0)` if the EC-NTT operation was successful, or an error if something went wrong.
Expand Down Expand Up @@ -94,4 +89,4 @@ func Main() {
panic("ECNTT operation failed")
}
}
```
```
8 changes: 2 additions & 6 deletions docs/docs/icicle/golang-bindings/msm-pre-computation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,11 @@

To understand the theory behind MSM pre computation technique refer to Niall Emmart's [talk](https://youtu.be/KAWlySN7Hm8?feature=shared&t=1734).

### Supported curves

`bls12-377`, `bls12-381`, `bn254`, `bw6-761`, `grumpkin`

## Core package

## MSM `PrecomputeBases`
### MSM PrecomputeBases

`PrecomputeBases` and `G2PrecomputeBases` exists for all supported curves.
`PrecomputeBases` and `G2PrecomputeBases` exists for all supported curves.

#### Description

Expand Down
Loading

0 comments on commit 6134cfe

Please sign in to comment.