Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gnovm): Executable Markdown #2357

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
1e598f6
fenced code block parser
notJoon Jun 14, 2024
168259d
file handler
notJoon Jun 14, 2024
edb648e
pseudo code for execute a parsed code
notJoon Jun 14, 2024
f60cb1c
generate hash key based on the parsed code
notJoon Jun 14, 2024
a577afc
use tree-sitter for parsing markdown content
notJoon Jun 14, 2024
b29530c
capturing output from buffer
notJoon Jun 15, 2024
6430862
refactor
notJoon Jun 15, 2024
b7ef8ad
save
notJoon Jun 16, 2024
b649727
change to use MemDB and add execute options
notJoon Jun 16, 2024
e53482b
save: need to handle import expr later
notJoon Jun 17, 2024
f0df3b3
basic CLI
notJoon Jun 17, 2024
599bd6f
execute imported function
notJoon Jun 18, 2024
da567e3
save
notJoon Jun 18, 2024
2d363bc
fix: doctest CLI
notJoon Jun 18, 2024
83d7ca0
Merge branch 'master' into testable-markdown
notJoon Jun 18, 2024
f31a102
lint
notJoon Jun 18, 2024
4f05a8e
Merge branch 'testable-markdown' of https://github.com/notJoon/gno-co…
notJoon Jun 18, 2024
23052b9
static analysis for auto fix missing package and import statement
notJoon Jun 25, 2024
fc94da3
Merge branch 'master' into testable-markdown
notJoon Jun 25, 2024
3844365
remove tree-sitter to avoid cgo linter problem
notJoon Jun 25, 2024
c8b0aef
reorganize
notJoon Jun 25, 2024
5a8a2d8
add test
notJoon Jun 26, 2024
86bdb99
wip: pkgloader
notJoon Jun 26, 2024
fbe7ddf
fix difference behavior with local and test
notJoon Jul 8, 2024
e001a33
Merge branch 'master' into testable-markdown
notJoon Jul 8, 2024
e648334
fix lint
notJoon Jul 8, 2024
82e02f0
expect result
notJoon Jul 10, 2024
273abb5
auto test name generator
notJoon Jul 10, 2024
1ee5486
use pattern when run code block instead index
notJoon Jul 11, 2024
4775fd3
generate better name
notJoon Jul 12, 2024
855bbc5
add execution option and support regex match for expected output
notJoon Jul 12, 2024
cbc060f
Merge branch 'master' into testable-markdown
notJoon Jul 12, 2024
ad329eb
ignore non-go language, add timeout
notJoon Jul 15, 2024
bf480c7
Merge branch 'master' into testable-markdown
notJoon Aug 2, 2024
1f7f238
Update gnovm/pkg/doctest/analyzer.go
notJoon Aug 2, 2024
1bb104a
Apply suggestions from code review
notJoon Aug 2, 2024
6bac3ad
resolve suggestions (doctest, exec, parser)
notJoon Aug 2, 2024
4978329
fix: partial exec, parser
notJoon Aug 2, 2024
1577034
update: compareResults, ExecuteBlock
notJoon Aug 5, 2024
2d64669
remove unnecessary lock
notJoon Aug 5, 2024
317fd41
fix: handle auto import for conflict names
notJoon Aug 5, 2024
ae25209
readme
notJoon Aug 5, 2024
45265f2
remove auto import
notJoon Aug 6, 2024
b7a3d9d
Merge branch 'master' into testable-markdown
notJoon Oct 3, 2024
1776969
Merge branch 'master' into testable-markdown
notJoon Oct 15, 2024
3142978
fix execution errors
notJoon Oct 15, 2024
d15d3f1
fix
notJoon Oct 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions gnovm/cmd/gno/doctest.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package main

import (
"context"
"flag"
"fmt"
"os"
"strings"
"time"

dt "github.com/gnolang/gno/gnovm/pkg/doctest"
"github.com/gnolang/gno/tm2/pkg/commands"
)

type doctestCfg struct {
markdownPath string
runPattern string
timeout time.Duration
}

func newDoctestCmd(io commands.IO) *commands.Command {
cfg := &doctestCfg{}

return commands.NewCommand(
commands.Metadata{
Name: "doctest",
ShortUsage: "doctest -path <markdown_file_path> [-run <pattern>] [-timeout <duration>]",
ShortHelp: "executes a specific code block from a markdown file",
},
cfg,
func(_ context.Context, args []string) error {
return execDoctest(cfg, args, io)
},
)
}

func (c *doctestCfg) RegisterFlags(fs *flag.FlagSet) {
fs.StringVar(
&c.markdownPath,
"path",
"",
"path to the markdown file",
)
fs.StringVar(
&c.runPattern,
"run",
"",
"pattern to match code block names",
)
fs.DurationVar(
&c.timeout,
"timeout",
time.Second*30,
"timeout for code execution (e.g., 30s, 1m)",
)
}

func execDoctest(cfg *doctestCfg, _ []string, io commands.IO) error {
if cfg.markdownPath == "" {
return fmt.Errorf("markdown file path is required")
}

content, err := fetchMarkdown(cfg.markdownPath)
if err != nil {
return fmt.Errorf("failed to read markdown file: %w", err)
}

ctx, cancel := context.WithTimeout(context.Background(), cfg.timeout)
defer cancel()

resultChan := make(chan []string)
errChan := make(chan error)

go func() {
results, err := dt.ExecuteMatchingCodeBlock(ctx, content, cfg.runPattern)
if err != nil {
errChan <- err
} else {
resultChan <- results
}
}()

select {
case results := <-resultChan:
if len(results) == 0 {
io.Println("No code blocks matched the pattern")
return nil
}
io.Println("Execution Result:")
io.Println(strings.Join(results, "\n\n"))
case err := <-errChan:
return fmt.Errorf("failed to execute code block: %w", err)
case <-ctx.Done():
return fmt.Errorf("execution timed out after %v", cfg.timeout)
}

return nil
}

// fetchMarkdown reads a markdown file and returns its content
func fetchMarkdown(path string) (string, error) {
content, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("failed to read file: %w", err)
}
return string(content), nil
}
97 changes: 97 additions & 0 deletions gnovm/cmd/gno/doctest_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
package main

import (
"os"
"testing"
)

func TestDoctest(t *testing.T) {
tempDir, err := os.MkdirTemp("", "doctest-test")
if err != nil {
t.Fatalf("failed to create temp directory: %v", err)
}
defer os.RemoveAll(tempDir)

markdownContent := `# Go Code Examples

This document contains two simple examples written in Go.

## Example 1: Fibonacci Sequence

The first example prints the first 10 numbers of the Fibonacci sequence.

` + "```go" + `
// @test: Fibonacci
package main

func main() {
a, b := 0, 1
for i := 0; i < 10; i++ {
println(a)
a, b = b, a+b
}
}
` + "```" + `

## Example 2: String Reversal

The second example reverses a given string and prints it.

` + "```go" + `
// @test: StringReversal
package main

func main() {
str := "Hello, Go!"
runes := []rune(str)
for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
runes[i], runes[j] = runes[j], runes[i]
}
println(string(runes))
}
` + "```" + `

These two examples demonstrate basic Go functionality without using concurrency, generics, or reflect.

` + "## std Package" + `
` + "```go" + `
// @test: StdPackage
package main

import (
"std"
)

func main() {
addr := std.GetOrigCaller()
println(addr)
}
` + "```" + `
`

mdFile, err := os.CreateTemp(tempDir, "sample-*.md")
if err != nil {
t.Fatalf("failed to create temp file: %v", err)
}
defer mdFile.Close()

_, err = mdFile.WriteString(markdownContent)
if err != nil {
t.Fatalf("failed to write to temp file: %v", err)
}

mdFilePath := mdFile.Name()

tc := []testMainCase{
{
args: []string{"doctest", "-path", mdFilePath, "-run", "StringReversal"},
stdoutShouldContain: "=== StringReversal ===\n\n!oG ,olleH",
},
{
args: []string{"doctest", "-path", mdFilePath, "-run", "StdPackage"},
stdoutShouldContain: "=== StdPackage ===\n\ng14ch5q26mhx3jk5cxl88t278nper264ces4m8nt",
},
}

testMainCaseRun(t, tc)
}
1 change: 1 addition & 0 deletions gnovm/cmd/gno/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func newGnocliCmd(io commands.IO) *commands.Command {
newDocCmd(io),
newEnvCmd(io),
newBugCmd(io),
newDoctestCmd(io),
newFmtCmd(io),
// graph
// vendor -- download deps from the chain in vendor/
Expand Down
131 changes: 131 additions & 0 deletions gnovm/pkg/doctest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Gno Doctest: Easy Code Execution and Testing

Gno Doctest is a tool that allows you to easily execute and test code blocks written in the Gno language. This tool offers a range of features, from simple code execution to complex package imports.

## Basic Usage

To use Gno Doctest, run the following command:

gno doctest -path <markdown_file_path> -run <code_block_name | "">

- `<markdown_file_path>`: Path to the markdown file containing Gno code blocks
- `<code_block_name>`: Name of the code block to run (optional)

For example, to run the code block named "print hello world" in the file "foo.md", use the following command:

gno doctest -path foo.md -run "print hello world"

## Features

### 1. Basic Code Execution

Gno Doctest can execute simple code blocks:

```go
package main

func main() {
println("Hello, World!")
}

// Output:
// Hello, World!
```

Doctest also recognizes that a block of code is a gno. The code below outputs the same result as the example above.

```go
Copy link
Contributor

@piux2 piux2 Aug 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This this a great idea, I have a few questions:

Do we intent to run both go code and gno code in the doctest?
How does Doctest detect this is a gno code?
What is the reason behind using "```go" instead of "````gno" for gno code block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really good question. Currrently, when rendering markdown, gno doesn't have universal syntax highlighting applied yet (except the custom syntax rule). So, even in official documents like "Effective gno" are using go instead of gno to specify the language.

Considering the purpose of markdown is more for documentation rather than testing, this was an inevitable purpose. So I made it recognize both languages.

However, it doesn't clearly distinguish whether it's completely go or gno. For now, I assume that it's mostly compatible with go and execute it at gno VM level. In other words, even if the language is atually written in go, everything runs as gno. This part may require deeper context analysis in the future.

To summarize in the order of your questions:

  1. Even if the code block is specified as Go, it's actually recognized and executed as gno.
  2. There's no feature to clearly distinguish between the two languages, and it relies on the language specified in the fenced code block.
  3. Go is also set to be recognized due to syntax highlighting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main difference between Gno and Go is that Gno has a few native methods unique to blockchain usage. For example:

std.AssertOriginCall
std.GetHeight

Basically, a program can either be Go code or Gno code. We cannot mix them in one source file.

There are a few options I can think of

  • Users need to explicitly mark it as either Go code or Gno code, similar to indicating the source code type with a suffix in the filename: xxx.go vs xxx.gno.
  • Only support Gno.
  • By default, the MD is treated as Gno code. Use gnostd.XXX to indicate that it is Go code. (Not sure if this has much practical purpose; it is more complicated to implement and is only for the sake of supporting Go code with chain functionality in MD.)

// @test: print hello world
package main

func main() {
println("Hello, World!")
Copy link
Contributor

@piux2 piux2 Aug 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the println does not print correct results compared with file tests.


  package main

  type ints []int

  func main() {
        a := ints{1,2,3}
      println(a)
  }

doctest

// Output:
// (slice[(1 int),(2 int),(3 int)] gno.land/r/g14ch5q26mhx3jk5cxl88t278nper264ces4m8nt/run.ints)

file test result is correct. ints is a type defined in main package.
// Output:
// (slice[(1 int),(2 int),(3 int)] main.ints)

Copy link
Member Author

@notJoon notJoon Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this is happening because the code is executed through the VMKeeper.

func ExecuteCodeBlock(c codeBlock, stdlibDir string) (string, error) {
    // ...
    addr := crypto.AddressFromPreimage([]byte("addr1"))
    acc := acck.NewAccountWithAddress(ctx, addr)
    acck.SetAccount(ctx, acc)

    msg2 := vm.NewMsgRun(addr, std.Coins{}, files)

    res, err := vmk.Run(ctx, msg2)
    // ...
}

I used this method to handle stdlibs imports. But honestly, I don't know how to maintain this functionality while also making it output main.

Copy link
Contributor

@piux2 piux2 Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about replacing it with the proper type when displaying the output in the Markdown?

}

// Output:
// Hello, World!
```

Running this code will output "Hello, World!".

## 2. Using Standard Library Packages

Doctest supports automatic import and usage of standard library packages.

If run this code, doctest will automatically import the "std" package and execute the code.

```go
// @test: omit-package-declaration
func main() {
addr := std.GetOrigCaller()
println(addr)
}
```

The code above outputs the same result as the code below.

```go
// @test: auto-import-package
package main

import "std"

func main() {
addr := std.GetOrigCaller()
println(addr)
}
```

## 3. Automatic Package Import

One of the most powerful features of Gno Doctest is its ability to handle package declarations and imports automatically.

```go
func main() {
println(math.Pi)
println(strings.ToUpper("Hello, World"))
}
```

In this code, the math and strings packages are not explicitly imported, but Doctest automatically recognizes and imports the necessary packages.

## 4. Omitting Package Declaration

Doctest can even handle cases where the `package` declaration is omitted.
piux2 marked this conversation as resolved.
Show resolved Hide resolved

```go
// @test: omit-top-level-package-declaration
func main() {
s := strings.ToUpper("Hello, World")
println(s)
}

// Output:
// HELLO, WORLD
```

This code runs normally without package declaration or import statements.
Using Gno Doctest makes code execution and testing much more convenient.

You can quickly run various Gno code snippets and check the results without complex setups.

### 7. Execution Options

Doctest supports special execution options:
Ignore Option
Use the ignore tag to skip execution of a code block:

**Ignore Option**

Use the ignore tag to skip execution of a code block:

```go,ignore
// @ignore
func main() {
println("This won't be executed")
}
```

## Conclusion

Gno Doctest simplifies the process of executing and testing Gno code snippets.
Loading
Loading