From 34777842e5b192f8374ed151c43a6cfac95cb458 Mon Sep 17 00:00:00 2001 From: damirka Date: Mon, 28 Jun 2021 21:20:45 +0300 Subject: [PATCH 1/5] draft updates for ps and curves --- docs/rfc/003-imports-stabilization.md | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/docs/rfc/003-imports-stabilization.md b/docs/rfc/003-imports-stabilization.md index cf3ce7c006..909b41271e 100644 --- a/docs/rfc/003-imports-stabilization.md +++ b/docs/rfc/003-imports-stabilization.md @@ -18,26 +18,33 @@ DRAFT # Summary This proposal aims to improve the import management system in Leo programs to -make program environment more reproducible and predictable. To achieve that +make program environment more reproducible, predictable and compatible. To achieve that we suggest few changes to Leo CLI and Manifest: - add a "dependencies" section to Leo Manifest and add a command to pull those dependencies; - allow custom names for imports to manually resolve name conflicts; -- store imports as they are called in Leo Manifest; +- add "curve" and "proving system" sections to the Manifest; +- add "inlcude" and "exclude" parameters for "proving system" and "curve"; Later this solution can be improved by adding a lock-file which would lock imported packages based on both their contents and version. # Motivation -What problems does it solve? What is the background? +The current design of imports does not provide any guarantees on what's stored +in program imports and published with the program to Aleo Package Manager. +When dependency is "added", it is stored inside imports folder, and it is possible +to manually edit and/or add packages in this folder. -Current state: -- imports are published with a program to Aleo PM; -- we treat programs as files with no verification of imports (they can be changed locally and published in that state); -- name collisions cannot be resolved; a new import overwrites existing; +Also, imports are stored under package name which makes it impossible to import +two different packages with the same name. -TBD +Another important detail in the scope of this proposal is that in future Leo +programs will have the ability to be run with different proving systems +and curves, possibly creating incompatibility between programs written +for different proving systems or curves. To make a foundation for these features +imports need to be managed with include/exclude lists for allowed (compatible) +proving systems and curves. # Design @@ -55,8 +62,6 @@ package = "package" version = "1.0" ``` -TBD - ## Leo CLI To support updated Manifest new command should be added to Leo CLI. From b371fecaea5b359a9dc054681f8e7b3314235616 Mon Sep 17 00:00:00 2001 From: damirka Date: Wed, 30 Jun 2021 22:00:48 +0300 Subject: [PATCH 2/5] wrap up --- docs/rfc/003-imports-stabilization.md | 103 +++++++++++++++++++++++--- 1 file changed, 93 insertions(+), 10 deletions(-) diff --git a/docs/rfc/003-imports-stabilization.md b/docs/rfc/003-imports-stabilization.md index 909b41271e..573118e396 100644 --- a/docs/rfc/003-imports-stabilization.md +++ b/docs/rfc/003-imports-stabilization.md @@ -18,8 +18,8 @@ DRAFT # Summary This proposal aims to improve the import management system in Leo programs to -make program environment more reproducible, predictable and compatible. To achieve that -we suggest few changes to Leo CLI and Manifest: +make program environment more reproducible, predictable and compatible. To achieve +that we suggest few changes to Leo CLI and Manifest: - add a "dependencies" section to Leo Manifest and add a command to pull those dependencies; - allow custom names for imports to manually resolve name conflicts; @@ -48,7 +48,32 @@ proving systems and curves. # Design -## Leo Manifest +## Leo Manifest - target section + +To lay the foundation for future of the Leo ecosystem and start integrating +information about programs compatibility we suggest adding two new fields to +the new `[target]` section of the Leo Manifest: `proving_system` and `curve`. + +Currently, Leo compiler only supports `Groth16` for proving system and `Bls12_377` +for curve, they are meant to be default values in Leo Manifest. + +```toml +[project] +name = "first" +version = "0.1.0" +description = "The first package" +license = "MIT" + +[target] +curve = "Bls12_377" +proving_system = "Groth16" +``` + +These fields are meant to be used to determine whether imported program is +compatible to the original when support for different curves and proving systems +is added. + +## Leo Manifest - dependencies Dependencies section: @@ -56,33 +81,89 @@ Dependencies section: [dependencies] name = { author = "author", package = "package", version = "version" } +# alternative way of adding dependency record [dependencies.name] author = "author" package = "package" version = "1.0" ``` +### Parameters description + +`name` field sets the name of the dependency in Leo code. That way we allow +developer to resolve collisions in import names manually. So, for example, +if a developer is adding `howard/silly-sudoku` package to his program, he +might define it's in-code name as `sudoku` and import it with that name: + +```ts +import sudoku; +``` + +`package`, `author` and `version` are package name, package author and +version respectively. They are already used as arguments in `leo add` +command, so these fields are already understood by the Leo developers. + ## Leo CLI To support updated Manifest new command should be added to Leo CLI. ```bash # pull imports -leo pull +leo install +``` + +Alternatively it can be called `pull`. +``` +leo pull ``` ## Imports Restructurization One of the goals of proposed changes is to allow importing packages with the -same name but different authors. To resolve name conflict we suggest storing -imports as they are named in Leo Manifest file (Leo.toml). +same name but different authors. This has to be solved not only on the +language level but also on the level of storing program imports. + +We suggest using set of all 3 possible program identifiers for import +folder name: `author-package@version`. Later it can be extended to +include hash for version, but having inital set already solves name +collisions. + +So, updated imports would look like: + +``` +leo-program +├── Leo.toml +├── README.md +├── imports +│ ├── author1-program@0.1.0 +│ │ └── ... +│ ├── author2-program2@1.0.4 +│ └── ... +├── inputs +│ └── ... +└── src + └── main.leo +``` + +This change would also affect the way imports are being processed on ASG +level, and we'd need to add imports map as an argument to the Leo compiler. +The Leo Manifest needs to be parsed and passed as a hashmap to the compiler: + +``` +first-program => author1-program@0.1.0 +second-program => author2-program2@1.0.4 +``` +## Recursive Dependencies - +Later this solution can be improved by building a lock file containing all the +information on program dependencies, and the file itself will have enough data +to track and prevent recursion. # Drawbacks @@ -90,6 +171,8 @@ This change might require the update of already published programs on Aleo PM du Leo Manifest change. However it is possible to implement it in a backward-compatible way. +It also introduces the danger of having recursive dependencies, this problem is addressed in the Design section above. + # Effect on Ecosystem Proposed improvement provides safety inside Leo programs and should not affect From 91078a44143bf1b4bd6a283299a7d90e3a40f6e5 Mon Sep 17 00:00:00 2001 From: damirka Date: Wed, 30 Jun 2021 22:14:36 +0300 Subject: [PATCH 3/5] small edits --- docs/rfc/003-imports-stabilization.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/rfc/003-imports-stabilization.md b/docs/rfc/003-imports-stabilization.md index 573118e396..08832a9cb5 100644 --- a/docs/rfc/003-imports-stabilization.md +++ b/docs/rfc/003-imports-stabilization.md @@ -93,7 +93,7 @@ version = "1.0" `name` field sets the name of the dependency in Leo code. That way we allow developer to resolve collisions in import names manually. So, for example, if a developer is adding `howard/silly-sudoku` package to his program, he -might define it's in-code name as `sudoku` and import it with that name: +might define its in-code name as `sudoku` and import it with that name: ```ts import sudoku; @@ -147,7 +147,8 @@ leo-program This change would also affect the way imports are being processed on ASG level, and we'd need to add imports map as an argument to the Leo compiler. -The Leo Manifest needs to be parsed and passed as a hashmap to the compiler: +The Leo Manifest's dependencies sections needs to be parsed and passed as +a hashmap to the compiler: ``` first-program => author1-program@0.1.0 From 98b59566f1732d703d498f22450c790977e3ac67 Mon Sep 17 00:00:00 2001 From: Alessandro Coglio Date: Wed, 30 Jun 2021 14:47:34 -0700 Subject: [PATCH 4/5] Fix typo. --- docs/rfc/003-imports-stabilization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfc/003-imports-stabilization.md b/docs/rfc/003-imports-stabilization.md index 08832a9cb5..ada0d37e16 100644 --- a/docs/rfc/003-imports-stabilization.md +++ b/docs/rfc/003-imports-stabilization.md @@ -24,7 +24,7 @@ that we suggest few changes to Leo CLI and Manifest: - add a "dependencies" section to Leo Manifest and add a command to pull those dependencies; - allow custom names for imports to manually resolve name conflicts; - add "curve" and "proving system" sections to the Manifest; -- add "inlcude" and "exclude" parameters for "proving system" and "curve"; +- add "include" and "exclude" parameters for "proving system" and "curve"; Later this solution can be improved by adding a lock-file which would lock imported packages based on both their contents and version. From 35660c851b07a1ce83c798fa45cfe3925cf13f8c Mon Sep 17 00:00:00 2001 From: damirka Date: Thu, 1 Jul 2021 19:14:06 +0300 Subject: [PATCH 5/5] grammar fixes --- docs/rfc/003-imports-stabilization.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/rfc/003-imports-stabilization.md b/docs/rfc/003-imports-stabilization.md index ada0d37e16..5fe231a643 100644 --- a/docs/rfc/003-imports-stabilization.md +++ b/docs/rfc/003-imports-stabilization.md @@ -33,16 +33,16 @@ imported packages based on both their contents and version. The current design of imports does not provide any guarantees on what's stored in program imports and published with the program to Aleo Package Manager. -When dependency is "added", it is stored inside imports folder, and it is possible +When a dependency is "added," it is stored inside imports folder, and it is possible to manually edit and/or add packages in this folder. -Also, imports are stored under package name which makes it impossible to import +Also, imports are stored under the package name which makes it impossible to import two different packages with the same name. Another important detail in the scope of this proposal is that in future Leo programs will have the ability to be run with different proving systems and curves, possibly creating incompatibility between programs written -for different proving systems or curves. To make a foundation for these features +for different proving systems or curves. To make a foundation for these features, imports need to be managed with include/exclude lists for allowed (compatible) proving systems and curves. @@ -50,12 +50,12 @@ proving systems and curves. ## Leo Manifest - target section -To lay the foundation for future of the Leo ecosystem and start integrating +To lay the foundation for the future of the Leo ecosystem and start integrating information about programs compatibility we suggest adding two new fields to the new `[target]` section of the Leo Manifest: `proving_system` and `curve`. -Currently, Leo compiler only supports `Groth16` for proving system and `Bls12_377` -for curve, they are meant to be default values in Leo Manifest. +Currently, the Leo compiler only supports `Groth16` for the proving system and `Bls12_377` +for the curve, they are meant to be default values in Leo Manifest. ```toml [project] @@ -125,7 +125,7 @@ language level but also on the level of storing program imports. We suggest using set of all 3 possible program identifiers for import folder name: `author-package@version`. Later it can be extended to -include hash for version, but having inital set already solves name +include hash for version, but having the inital set already solves name collisions. So, updated imports would look like: @@ -145,8 +145,8 @@ leo-program └── main.leo ``` -This change would also affect the way imports are being processed on ASG -level, and we'd need to add imports map as an argument to the Leo compiler. +This change would also affect the way imports are being processed on the ASG +level, and we'd need to add an imports map as an argument to the Leo compiler. The Leo Manifest's dependencies sections needs to be parsed and passed as a hashmap to the compiler: @@ -158,9 +158,9 @@ second-program => author2-program2@1.0.4 ## Recursive Dependencies This improvement introduces recursive dependencies. To solve this case preemptively -Leo CLI needs to check dependency tree and throw an error when recursive dependency +Leo CLI needs to check the dependency tree and throw an error when a recursive dependency is met. We suggest implementing simple dependency tree checking while fetching -imports - if imported dependency is met on higher level - abort the execution. +imports - if imported dependency is met on a higher level - abort the execution. Later this solution can be improved by building a lock file containing all the information on program dependencies, and the file itself will have enough data