-
Notifications
You must be signed in to change notification settings - Fork 375
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(transpiler): transpile gno standard libraries (#1695)
Merge order: 1. #1700 2. #1702 3. #1695 (this one!) -- review earlier ones first, if they're still open! This PR modifies the Gno transpiler (fka precompiler) to use Gno's standard libraries rather than Go's when performing transpilation. This creates the necessity to transpile Gno standard libraries, and as such support their native bindings. And it removes the necessity for a package like `stdshim`, and a mechanism like `stdlibWhitelist`. - Fixes #668. Fixes #1865. - Resolves #892. - Part of #814. - Makes #1475 / #1576 possible without using hacks like `stdshim`. cc/ @leohhhn @tbruyelle, as this relates to your work ## Why? - This PR enables us to perform Go type-checking across the board, and not use Go's standard libraries in transpiled code. This enables us to _properly support our own standard libraries_, such as `std` but any others we might want or need. - It also paves the way further to go full circle, and have Gno code be transpiled to Go, and then have "compilable" gno code ## Summary of changes - The transpiler has been thoroughly refactored. - The biggest change is described above: instead of maintaing the import paths like `"strconv"` and `"math"` the same (so using Gno's stdlibs in Gno, and Go's in Go), the import paths for standard libraries is now also updated to point to the Gno standard libraries. - Native functions are handled by removing their definitions when transpiling, and changing their call expressions where appropriate. This links the transpiled code directly to their native counterparts. - This removes the necessity for `stdlibWhitelist`. - As a consequence, `stdshim` is no longer needed and has been removed. - Test files are still not "strictly checked": they may reference stdlibs with no matching source, and will not be tested when running with `--gobuild`. This is because packages like `fmt` have no representation in Gno code; they only exist as injections in `tests/imports.go`. I'll fix this eventually :) - The CLI (`gno transpile`) has been changed to reflect the above changes. - Flag `--skip-fmt` has been removed (the result of transpile is always formatted, anyway), and `--gofmt-binary` too, obviously. `gno transpile` does not perform validation, but will gladly provide helpful validation with the `--gobuild` flag. - There is another PR that adds type checking in `gno lint`, without needing to run through the transpilation step first: #1730 - It now works by default by looking at "packages" rather than individual files. This is necessary so that when performing `transpile` on the `examples` directory, we can skip those where the gno.mod marks the module as draft. These modules make use of packages like "fmt", which because they don't have an underlying gno/go source, cannot be transpiled. - Running with `-gobuild` now handles more errors correctly; ie., all errors not previously captured by the `errorRe` which only matches those pertaining to a specific file/line. - `gnoFilesFromArgs` was unused and as such deleted - `gnomod`'s behaviour was slightly changed. - I am of the opinion that `gno mod download` should not precompile what it downloads; _especially_ to gather the dependencies it has. I've changed it so that it does a `OnlyImports` parse of the file it downloads to fetch additional dependencies Misc: - `Makefile` now contains a recipe to calculate the coverage for `gnovm/cmd/gno`, and also view it via the HTML interface. This is needed as it has a few extra steps (which @gfanton already previously added in the CI). - Realms r/demo/art/gnoface and r/x/manfred_outfmt have been marked as draft, as they depend on packages which are not actually present in the Gno standard libraries. - The transpiler now ignores draft packages by default. - `ReadMemPackage` now also considers Go files. This is meant to have on-chain the code for standard libraries like `std` which have native bindings. We still exclude Go code if it's not in a standard library. - `//go:build` constraints have been removed from standard libraries, as go files can only have one and we already add our own when transpiling ## Further improvements after this PR - Scope understanding in `transpiler` (so call expressions are not incorrectly rewritten) - Correctly transpile gno.mod --------- Co-authored-by: Antonio Navarro Perez <antnavper@gmail.com> Co-authored-by: Miloš Živković <milos.zivkovic@tendermint.com>
- Loading branch information
1 parent
feb3051
commit c4664ed
Showing
49 changed files
with
1,327 additions
and
971 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
// Draft | ||
|
||
module gno.land/r/demo/art/gnoface | ||
|
||
require ( | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
// Draft | ||
|
||
module gno.land/r/x/manfred_outfmt | ||
|
||
require ( | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
36 changes: 0 additions & 36 deletions
36
gnovm/cmd/gno/testdata/gno_transpile/05_skip_fmt_flag.txtar
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
File renamed without changes.
File renamed without changes.
6 changes: 3 additions & 3 deletions
6
...nspile/09_gno_files_whitelist_error.txtar → ...stdata/gno_transpile/invalid_import.txtar
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
31 changes: 31 additions & 0 deletions
31
gnovm/cmd/gno/testdata/gno_transpile/valid_gobuild_file.txtar
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Run gno transpile with -gobuild flag on an individual file | ||
|
||
gno transpile -gobuild -v main.gno | ||
|
||
! stdout .+ | ||
cmp stderr stderr.golden | ||
|
||
cmp main.gno.gen.go main.gno.gen.go.golden | ||
|
||
-- stderr.golden -- | ||
main.gno | ||
main.gno [build] | ||
-- main.gno -- | ||
package main | ||
|
||
func main() { | ||
var x = 1 | ||
_=x | ||
} | ||
-- main.gno.gen.go.golden -- | ||
// Code generated by github.com/gnolang/gno. DO NOT EDIT. | ||
|
||
//go:build gno | ||
|
||
//line main.gno:1:1 | ||
package main | ||
|
||
func main() { | ||
var x = 1 | ||
_ = x | ||
} |
72 changes: 72 additions & 0 deletions
72
gnovm/cmd/gno/testdata/gno_transpile/valid_gobuild_flag.txtar
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# Run gno transpile with -gobuild flag | ||
|
||
gno transpile -gobuild -v . | ||
|
||
! stdout .+ | ||
cmp stderr stderr.golden | ||
|
||
# The test file will be excluded from transpilation unless we pass it explicitly. | ||
cmp main.gno.gen.go main.gno.gen.go.golden | ||
! exists .main_test.gno.gen_test.go | ||
cmp sub/sub.gno.gen.go sub/sub.gno.gen.go.golden | ||
rm mai.gno.gen.gosub/sub.gno.gen.go | ||
|
||
# Re-try, but use an absolute path. | ||
gno transpile -gobuild -v $WORK | ||
|
||
! stdout .+ | ||
cmpenv stderr stderr2.golden | ||
|
||
cmp main.gno.gen.go main.gno.gen.go.golden | ||
! exists .main_test.gno.gen_test.go | ||
cmp sub/sub.gno.gen.go sub/sub.gno.gen.go.golden | ||
|
||
-- stderr.golden -- | ||
. | ||
sub | ||
. [build] | ||
sub [build] | ||
-- stderr2.golden -- | ||
$WORK | ||
$WORK/sub | ||
$WORK [build] | ||
$WORK/sub [build] | ||
-- main.gno -- | ||
package main | ||
|
||
func main() { | ||
var x = 1 | ||
_=x | ||
} | ||
-- main.gno.gen.go.golden -- | ||
// Code generated by github.com/gnolang/gno. DO NOT EDIT. | ||
|
||
//go:build gno | ||
|
||
//line main.gno:1:1 | ||
package main | ||
|
||
func main() { | ||
var x = 1 | ||
_ = x | ||
} | ||
-- main_test.gno -- | ||
package main | ||
|
||
import ( | ||
"testing" | ||
"badimport" | ||
) | ||
|
||
func TestMain(t *testing.T) { | ||
badimport.DoesNotExist() | ||
} | ||
-- sub/sub.gno -- | ||
package sub | ||
-- sub/sub.gno.gen.go.golden -- | ||
// Code generated by github.com/gnolang/gno. DO NOT EDIT. | ||
|
||
//go:build gno | ||
|
||
//line sub.gno:1:1 | ||
package sub |
41 changes: 41 additions & 0 deletions
41
gnovm/cmd/gno/testdata/gno_transpile/valid_output_flag.txtar
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# Run gno transpile with valid gno files, using the -output flag. | ||
|
||
gno transpile -v -output directory/hello/ . | ||
! stdout .+ | ||
cmp stderr stderr1.golden | ||
|
||
exists directory/hello/main.gno.gen.go | ||
! exists main.gno.gen.go | ||
rm directory | ||
|
||
# Try running using the absolute path to the directory. | ||
gno transpile -v -output directory/hello $WORK | ||
! stdout .+ | ||
cmpenv stderr stderr2.golden | ||
|
||
exists directory/hello$WORK/main.gno.gen.go | ||
! exists directory/hello/main.gno.gen.go | ||
rm directory | ||
|
||
# Try running in subdirectory, using a "relative non-local path." (ie. has "../") | ||
mkdir subdir | ||
cd subdir | ||
gno transpile -v -output hello .. | ||
! stdout .+ | ||
cmpenv stderr ../stderr3.golden | ||
|
||
exists hello$WORK/main.gno.gen.go | ||
! exists main.gno.gen.go | ||
|
||
-- stderr1.golden -- | ||
. | ||
-- stderr2.golden -- | ||
$WORK | ||
-- stderr3.golden -- | ||
.. | ||
-- main.gno -- | ||
package main | ||
|
||
func main() { | ||
println("hello") | ||
} |
44 changes: 44 additions & 0 deletions
44
gnovm/cmd/gno/testdata/gno_transpile/valid_output_gobuild.txtar
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# Run gno transpile with valid gno files, using the -output and -gobuild flags together. | ||
|
||
gno transpile -v -output directory/hello/ -gobuild . | ||
! stdout .+ | ||
cmp stderr stderr1.golden | ||
|
||
exists directory/hello/main.gno.gen.go | ||
! exists main.gno.gen.go | ||
rm directory | ||
|
||
# Try running using the absolute path to the directory. | ||
gno transpile -v -output directory/hello -gobuild $WORK | ||
! stdout .+ | ||
cmpenv stderr stderr2.golden | ||
|
||
exists directory/hello$WORK/main.gno.gen.go | ||
! exists directory/hello/main.gno.gen.go | ||
rm directory | ||
|
||
# Try running in subdirectory, using a "relative non-local path." (ie. has "../") | ||
mkdir subdir | ||
cd subdir | ||
gno transpile -v -output hello -gobuild .. | ||
! stdout .+ | ||
cmpenv stderr ../stderr3.golden | ||
|
||
exists hello$WORK/main.gno.gen.go | ||
! exists main.gno.gen.go | ||
|
||
-- stderr1.golden -- | ||
. | ||
directory/hello [build] | ||
-- stderr2.golden -- | ||
$WORK | ||
directory/hello$WORK [build] | ||
-- stderr3.golden -- | ||
.. | ||
hello$WORK [build] | ||
-- main.gno -- | ||
package main | ||
|
||
func main() { | ||
println("hello") | ||
} |
Oops, something went wrong.