Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Extending scope of the RFC to include proving system and curve #1072

Merged
merged 5 commits into from
Jul 2, 2021
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 107 additions & 18 deletions docs/rfc/003-imports-stabilization.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,73 +18,162 @@ DRAFT
# Summary

This proposal aims to improve the import management system in Leo programs to
make program environment more reproducible and predictable. To achieve that
we suggest few changes to Leo CLI and Manifest:
make program environment more reproducible, predictable and compatible. To achieve
that we suggest few changes to Leo CLI and Manifest:

- add a "dependencies" section to Leo Manifest and add a command to pull those dependencies;
- allow custom names for imports to manually resolve name conflicts;
- store imports as they are called in Leo Manifest;
- add "curve" and "proving system" sections to the Manifest;
- add "inlcude" and "exclude" parameters for "proving system" and "curve";

Later this solution can be improved by adding a lock-file which would lock
imported packages based on both their contents and version.

# Motivation

What problems does it solve? What is the background?
The current design of imports does not provide any guarantees on what's stored
in program imports and published with the program to Aleo Package Manager.
When dependency is "added", it is stored inside imports folder, and it is possible
damirka marked this conversation as resolved.
Show resolved Hide resolved
to manually edit and/or add packages in this folder.

Current state:
- imports are published with a program to Aleo PM;
- we treat programs as files with no verification of imports (they can be changed locally and published in that state);
- name collisions cannot be resolved; a new import overwrites existing;
Also, imports are stored under package name which makes it impossible to import
damirka marked this conversation as resolved.
Show resolved Hide resolved
two different packages with the same name.

TBD
Another important detail in the scope of this proposal is that in future Leo
programs will have the ability to be run with different proving systems
and curves, possibly creating incompatibility between programs written
for different proving systems or curves. To make a foundation for these features
damirka marked this conversation as resolved.
Show resolved Hide resolved
imports need to be managed with include/exclude lists for allowed (compatible)
proving systems and curves.

# Design

## Leo Manifest
## Leo Manifest - target section

To lay the foundation for future of the Leo ecosystem and start integrating
damirka marked this conversation as resolved.
Show resolved Hide resolved
information about programs compatibility we suggest adding two new fields to
the new `[target]` section of the Leo Manifest: `proving_system` and `curve`.

Currently, Leo compiler only supports `Groth16` for proving system and `Bls12_377`
damirka marked this conversation as resolved.
Show resolved Hide resolved
for curve, they are meant to be default values in Leo Manifest.

```toml
[project]
name = "first"
version = "0.1.0"
description = "The first package"
license = "MIT"

[target]
curve = "Bls12_377"
proving_system = "Groth16"
```

These fields are meant to be used to determine whether imported program is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whether an imported program is

compatible to the original when support for different curves and proving systems
is added.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are added


## Leo Manifest - dependencies

Dependencies section:

```toml
[dependencies]
name = { author = "author", package = "package", version = "version" }

# alternative way of adding dependency record
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an alternative

[dependencies.name]
author = "author"
package = "package"
version = "1.0"
```

TBD
### Parameters description

`name` field sets the name of the dependency in Leo code. That way we allow
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That way, we allow developers

developer to resolve collisions in import names manually. So, for example,
if a developer is adding `howard/silly-sudoku` package to his program, he
might define its in-code name as `sudoku` and import it with that name:

```ts
import sudoku;
```

`package`, `author` and `version` are package name, package author and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the... in the leo add command

version respectively. They are already used as arguments in `leo add`
command, so these fields are already understood by the Leo developers.

## Leo CLI

To support updated Manifest new command should be added to Leo CLI.

```bash
# pull imports
leo pull
leo install
```

Alternatively it can be called `pull`.
```
leo pull
```

## Imports Restructurization

One of the goals of proposed changes is to allow importing packages with the
same name but different authors. To resolve name conflict we suggest storing
imports as they are named in Leo Manifest file (Leo.toml).
same name but different authors. This has to be solved not only on the
language level but also on the level of storing program imports.

We suggest using set of all 3 possible program identifiers for import
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using the set

folder name: `author-package@version`. Later it can be extended to
include hash for version, but having inital set already solves name
damirka marked this conversation as resolved.
Show resolved Hide resolved
collisions.

So, updated imports would look like:

```
leo-program
├── Leo.toml
├── README.md
├── imports
│ ├── author1-program@0.1.0
│ │ └── ...
│ ├── author2-program2@1.0.4
│ └── ...
├── inputs
│ └── ...
└── src
└── main.leo
```

This change would also affect the way imports are being processed on ASG
damirka marked this conversation as resolved.
Show resolved Hide resolved
level, and we'd need to add imports map as an argument to the Leo compiler.
damirka marked this conversation as resolved.
Show resolved Hide resolved
The Leo Manifest's dependencies sections needs to be parsed and passed as
a hashmap to the compiler:

```
first-program => author1-program@0.1.0
second-program => author2-program2@1.0.4
```

## Recursive Dependencies

<!-- The suggested change is soft. It changes only the way imports are organized
with minimal changes to other parts of the language.
This improvement introduces recursive dependencies. To solve this case preemptively
Leo CLI needs to check dependency tree and throw an error when recursive dependency
damirka marked this conversation as resolved.
Show resolved Hide resolved
is met. We suggest implementing simple dependency tree checking while fetching
imports - if imported dependency is met on higher level - abort the execution.
damirka marked this conversation as resolved.
Show resolved Hide resolved

We can consider implementing imports/username-package storage, but imports
will have to be resolved on a different level in compiler. -->
Later this solution can be improved by building a lock file containing all the
information on program dependencies, and the file itself will have enough data
to track and prevent recursion.

# Drawbacks

This change might require the update of already published programs on Aleo PM due to
Leo Manifest change. However it is possible to implement it in a backward-compatible
way.

It also introduces the danger of having recursive dependencies, this problem is addressed in the Design section above.

# Effect on Ecosystem

Proposed improvement provides safety inside Leo programs and should not affect
Expand Down