Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gnovm): Executable Markdown #2357

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
1e598f6
fenced code block parser
notJoon Jun 14, 2024
168259d
file handler
notJoon Jun 14, 2024
edb648e
pseudo code for execute a parsed code
notJoon Jun 14, 2024
f60cb1c
generate hash key based on the parsed code
notJoon Jun 14, 2024
a577afc
use tree-sitter for parsing markdown content
notJoon Jun 14, 2024
b29530c
capturing output from buffer
notJoon Jun 15, 2024
6430862
refactor
notJoon Jun 15, 2024
b7ef8ad
save
notJoon Jun 16, 2024
b649727
change to use MemDB and add execute options
notJoon Jun 16, 2024
e53482b
save: need to handle import expr later
notJoon Jun 17, 2024
f0df3b3
basic CLI
notJoon Jun 17, 2024
599bd6f
execute imported function
notJoon Jun 18, 2024
da567e3
save
notJoon Jun 18, 2024
2d363bc
fix: doctest CLI
notJoon Jun 18, 2024
83d7ca0
Merge branch 'master' into testable-markdown
notJoon Jun 18, 2024
f31a102
lint
notJoon Jun 18, 2024
4f05a8e
Merge branch 'testable-markdown' of https://github.com/notJoon/gno-co…
notJoon Jun 18, 2024
23052b9
static analysis for auto fix missing package and import statement
notJoon Jun 25, 2024
fc94da3
Merge branch 'master' into testable-markdown
notJoon Jun 25, 2024
3844365
remove tree-sitter to avoid cgo linter problem
notJoon Jun 25, 2024
c8b0aef
reorganize
notJoon Jun 25, 2024
5a8a2d8
add test
notJoon Jun 26, 2024
86bdb99
wip: pkgloader
notJoon Jun 26, 2024
fbe7ddf
fix difference behavior with local and test
notJoon Jul 8, 2024
e001a33
Merge branch 'master' into testable-markdown
notJoon Jul 8, 2024
e648334
fix lint
notJoon Jul 8, 2024
82e02f0
expect result
notJoon Jul 10, 2024
273abb5
auto test name generator
notJoon Jul 10, 2024
1ee5486
use pattern when run code block instead index
notJoon Jul 11, 2024
4775fd3
generate better name
notJoon Jul 12, 2024
855bbc5
add execution option and support regex match for expected output
notJoon Jul 12, 2024
cbc060f
Merge branch 'master' into testable-markdown
notJoon Jul 12, 2024
ad329eb
ignore non-go language, add timeout
notJoon Jul 15, 2024
bf480c7
Merge branch 'master' into testable-markdown
notJoon Aug 2, 2024
1f7f238
Update gnovm/pkg/doctest/analyzer.go
notJoon Aug 2, 2024
1bb104a
Apply suggestions from code review
notJoon Aug 2, 2024
6bac3ad
resolve suggestions (doctest, exec, parser)
notJoon Aug 2, 2024
4978329
fix: partial exec, parser
notJoon Aug 2, 2024
1577034
update: compareResults, ExecuteBlock
notJoon Aug 5, 2024
2d64669
remove unnecessary lock
notJoon Aug 5, 2024
317fd41
fix: handle auto import for conflict names
notJoon Aug 5, 2024
ae25209
readme
notJoon Aug 5, 2024
45265f2
remove auto import
notJoon Aug 6, 2024
b7a3d9d
Merge branch 'master' into testable-markdown
notJoon Oct 3, 2024
1776969
Merge branch 'master' into testable-markdown
notJoon Oct 15, 2024
3142978
fix execution errors
notJoon Oct 15, 2024
d15d3f1
fix
notJoon Oct 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions gnovm/cmd/gno/doctest.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
package main

import (
"context"
"flag"
"fmt"
"os"
"strings"

dt "github.com/gnolang/gno/gnovm/pkg/doctest"
"github.com/gnolang/gno/tm2/pkg/commands"
)

type doctestCfg struct {
markdownPath string
// codeIndex int
runPattern string
}

func newDoctestCmd(io commands.IO) *commands.Command {
cfg := &doctestCfg{}

return commands.NewCommand(
commands.Metadata{
Name: "doctest",
ShortUsage: "doctest -path <markdown_file_path> [-run <pattern>]",
ShortHelp: "executes a specific code block from a markdown file",
},
cfg,
func(_ context.Context, args []string) error {
return execDoctest(cfg, args, io)
},
)
}

func (c *doctestCfg) RegisterFlags(fs *flag.FlagSet) {
fs.StringVar(
&c.markdownPath,
"path",
"",
"path to the markdown file",
)
fs.StringVar(
&c.runPattern,
"run",
"",
"pattern to match code block names",
)
}

func execDoctest(cfg *doctestCfg, _ []string, io commands.IO) error {
if cfg.markdownPath == "" {
return fmt.Errorf("markdown file path is required")
}

content, err := fetchMarkdown(cfg.markdownPath)
if err != nil {
return fmt.Errorf("failed to read markdown file: %w", err)
}

results, err := dt.ExecuteMatchingCodeBlock(content, cfg.runPattern)
if err != nil {
return fmt.Errorf("failed to execute code block: %w", err)
}

if len(results) == 0 {
io.Println("No code blocks matched the pattern")
return nil
}

io.Println("Execution Result:")
io.Println(strings.Join(results, "\n\n"))

return nil
}

// fetchMarkdown reads a markdown file and returns its content
func fetchMarkdown(path string) (string, error) {
content, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("failed to read file: %w", err)
}
return string(content), nil
}
105 changes: 105 additions & 0 deletions gnovm/cmd/gno/doctest_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
package main

// import (
// "os"
// "testing"
// )

// func TestDoctest(t *testing.T) {
// tempDir, err := os.MkdirTemp("", "doctest-test")
// if err != nil {
// t.Fatalf("failed to create temp directory: %v", err)
// }
// defer os.RemoveAll(tempDir)

// markdownContent := `# Go Code Examples

// This document contains two simple examples written in Go.

// ## Example 1: Fibonacci Sequence

// The first example prints the first 10 numbers of the Fibonacci sequence.

// ` + "```go" + `
// // @test: Fibonacci
// package main

// func main() {
// a, b := 0, 1
// for i := 0; i < 10; i++ {
// println(a)
// a, b = b, a+b
// }
// }
// ` + "```" + `

// ## Example 2: String Reversal

// The second example reverses a given string and prints it.

// ` + "```go" + `
// // @test: StringReversal
// package main

// func main() {
// str := "Hello, Go!"
// runes := []rune(str)
// for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
// runes[i], runes[j] = runes[j], runes[i]
// }
// println(string(runes))
// }
// ` + "```" + `

// These two examples demonstrate basic Go functionality without using concurrency, generics, or reflect.

// ` + "## std Package" + `
// ` + "```go" + `
// // @test: StdPackage
// package main

// import (
// "std"
// )

// func main() {
// addr := std.GetOrigCaller()
// println(addr)
// }
// ` + "```" + `
// `

// mdFile, err := os.CreateTemp(tempDir, "sample-*.md")
// if err != nil {
// t.Fatalf("failed to create temp file: %v", err)
// }
// defer mdFile.Close()

// _, err = mdFile.WriteString(markdownContent)
// if err != nil {
// t.Fatalf("failed to write to temp file: %v", err)
// }

// mdFilePath := mdFile.Name()

// tc := []testMainCase{
// {
// args: []string{"doctest", "-h"},
// errShouldBe: "flag: help requested",
// },
// {
// args: []string{"doctest", "-path", mdFilePath, "-run", "Fibonacci"},
// stdoutShouldContain: "--- Fibonacci ---\n0\n1\n1\n2\n3\n5\n8\n13\n21\n34\n",
// },
// {
// args: []string{"doctest", "-path", mdFilePath, "-run", "StringReversal"},
// stdoutShouldContain: "--- StringReversal ---\n!oG ,olleH\n",
// },
// {
// args: []string{"doctest", "-path", mdFilePath, "-run", "StdPackage"},
// stdoutShouldContain: "--- StdPackage ---\ng14ch5q26mhx3jk5cxl88t278nper264ces4m8nt\n",
// },
// }

// testMainCaseRun(t, tc)
// }
1 change: 1 addition & 0 deletions gnovm/cmd/gno/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func newGnocliCmd(io commands.IO) *commands.Command {
newDocCmd(io),
newEnvCmd(io),
newBugCmd(io),
newDoctestCmd(io),
newFmtCmd(io),
// graph
// vendor -- download deps from the chain in vendor/
Expand Down
87 changes: 87 additions & 0 deletions gnovm/cmd/gno/testdata/doctest/1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Gno Doctest: Easy Code Execution and Testing

Gno Doctest is a tool that allows you to easily execute and test code blocks written in the Gno language. This tool offers a range of features, from simple code execution to complex package imports.

## 1. Basic Code Execution

Even the simplest form of code block can be easily executed.

```go
package main

func main() {
println("Hello, World!")
}
```

Doctest also recognizes that a block of code is a gno. The code below outputs the same result as the example above.

```go
package main

func main() {
println("Hello, World!")
}
```

Running this code will output "Hello, World!".

## 2. Using Standard Library Packages

Gno Doctest automatically recognizes and imports standard library packages.

```go
package main

import "std"

func main() {
addr := std.GetOrigCaller()
println(addr)
}
```

## 3. Utilizing Various Standard Libraries

You can use multiple standard libraries simultaneously.

```gno
package main

import "strings"

func main() {
println(strings.ToUpper("Hello, World"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't get where you're putting the output?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The execution results are not separately saved (although they are stored in the cache) and are only shown through the CLI. At first, I thought this would be sufficient, but it seems that saving the values would be more useful.

}
```

This example uses the ToUpper() function from the strings package to convert a string to uppercase.

## 4. Automatic Package Import

One of the most powerful features of Gno Doctest is its ability to handle package declarations and imports automatically.

```go
func main() {
println(math.Pi)
println(strings.ToUpper("Hello, World"))
}
```

In this code, the math and strings packages are not explicitly imported, but Doctest automatically recognizes and imports the necessary packages.

## 5. Omitting Package Declaration

Doctest can even handle cases where the package main declaration is omitted.

```go
func main() {
s := strings.ToUpper("Hello, World")
println(s)
}
```

This code runs normally without package declaration or import statements.
Using Gno Doctest makes code execution and testing much more convenient.

You can quickly run various Gno code snippets and check the results without complex setups.
Loading
Loading