Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to generate auto fixes using LLM (AI) #1177

Merged
merged 37 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
b300f90
config run
tran-the-lam Jul 30, 2024
e74d1ef
add: propose solution by gemini
tran-the-lam Jul 31, 2024
7019848
clean code
tran-the-lam Jul 31, 2024
8029f7d
remove print version,git tag,build date
tran-the-lam Aug 1, 2024
f8bb652
add flag ai-api-key, ai-api-provider
tran-the-lam Aug 1, 2024
a5648fe
add godocs for public function
tran-the-lam Aug 1, 2024
088687b
add newline in Makefile
tran-the-lam Aug 1, 2024
6a9fcc5
add ai testcase
tran-the-lam Aug 1, 2024
242d957
lint ai.go
tran-the-lam Aug 1, 2024
53f4fe4
wrapp error
tran-the-lam Aug 3, 2024
f2c7d75
rename: proposal solution -> auto fix
tran-the-lam Aug 3, 2024
d393cf0
fix: typo
tran-the-lam Aug 3, 2024
d56896f
add cache ai generate autofix
tran-the-lam Aug 3, 2024
c20d7fb
add constant
tran-the-lam Aug 3, 2024
84e3ebd
add unit test
tran-the-lam Aug 3, 2024
4ab1b80
update readme
tran-the-lam Aug 3, 2024
d585a04
update ai prompt
tran-the-lam Aug 4, 2024
48fda16
update readme
tran-the-lam Aug 6, 2024
e66f3ae
wrap generate content error
tran-the-lam Aug 6, 2024
76ccda4
Update autofix/ai.go
tran-the-lam Aug 6, 2024
7bf235f
update wrap error
tran-the-lam Aug 6, 2024
c713f1a
reduce scope block of code
tran-the-lam Aug 6, 2024
2da9a84
Update cmd/gosec/main.go
tran-the-lam Aug 6, 2024
6e42318
Update cmd/gosec/main.go
tran-the-lam Aug 6, 2024
5001251
Update cmd/gosec/main.go
tran-the-lam Aug 6, 2024
dbfc301
lint code
tran-the-lam Aug 6, 2024
bb07619
Update autofix/ai.go
tran-the-lam Aug 6, 2024
deaec0a
Update autofix/ai.go
tran-the-lam Aug 6, 2024
f2684f6
Update autofix/ai.go
tran-the-lam Aug 6, 2024
a69c578
fix test
tran-the-lam Aug 7, 2024
815a880
update comment
tran-the-lam Aug 7, 2024
4722af5
Update go mod and tidy up
ccojocar Aug 12, 2024
159f0d0
Update README and the default helper messages of the cli arguments
ccojocar Aug 12, 2024
d1c92e7
Improve the prompt
ccojocar Aug 12, 2024
3160181
Add the autofix in the SARIF report
ccojocar Aug 12, 2024
5f1bd21
Improve the comments
ccojocar Aug 12, 2024
f97e82f
Fix the tests
ccojocar Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,12 @@ gosec can ignore generated go files with default generated code comment.
gosec -exclude-generated ./...
```

### Autofix vulnerabilities
gosec can suggest fixes based on AI. To use this feature, please provide the following two parameters `ai-api-provider` and `ai-api-key`. In the current version, gosec only supports gemini. [How to create gemini key.](https://ai.google.dev/gemini-api/docs/api-key)

```bash
gosec -ai-api-provider="your_provider" -ai-api-key="your_key" ./...
```

### Annotating code

Expand Down
128 changes: 128 additions & 0 deletions autofix/ai.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
package autofix

import (
"context"
"errors"
"fmt"
"time"

"github.com/google/generative-ai-go/genai"
"google.golang.org/api/option"

"github.com/securego/gosec/v2/issue"
)

const (
GeminiModel = "gemini-1.5-flash"
AIPrompt = "In Go, what is the solution to fix the error %q. Answer limited to 200 words"
GeminiProvider = "gemini"

timeout = 30 * time.Second
)

// GenAIClient defines the interface for the GenAI client
type GenAIClient interface {
Close() error
GenerativeModel(name string) GenAIGenerativeModel
}

// GenAIGenerativeModel defines the interface for the Generative Model
type GenAIGenerativeModel interface {
GenerateContent(ctx context.Context, prompt string) (string, error)
}

// genAIClientWrapper wraps the genai.Client to implement GenAIClient
type genAIClientWrapper struct {
client *genai.Client
}

func (w *genAIClientWrapper) Close() error {
return w.client.Close()
}

func (w *genAIClientWrapper) GenerativeModel(name string) GenAIGenerativeModel {
return &genAIGenerativeModelWrapper{model: w.client.GenerativeModel(name)}
}

// genAIGenerativeModelWrapper wraps the genai.GenerativeModel to implement GenAIGenerativeModel
type genAIGenerativeModelWrapper struct {
// model is the underlying generative model
model *genai.GenerativeModel
}

tran-the-lam marked this conversation as resolved.
Show resolved Hide resolved
func (w *genAIGenerativeModelWrapper) GenerateContent(ctx context.Context, prompt string) (string, error) {
resp, err := w.model.GenerateContent(ctx, genai.Text(prompt))
if err != nil {
return "", fmt.Errorf("generate content error: %w", err)
}
if len(resp.Candidates) == 0 {
return "", errors.New("no candidates found")
}
// Return the first candidate
return fmt.Sprintf("%+v", resp.Candidates[0].Content.Parts[0]), nil
ccojocar marked this conversation as resolved.
Show resolved Hide resolved
}

tran-the-lam marked this conversation as resolved.
Show resolved Hide resolved
func NewGenAIClient(ctx context.Context, aiApiKey, endpoint string) (GenAIClient, error) {
clientOptions := []option.ClientOption{option.WithAPIKey(aiApiKey)}
if endpoint != "" {
clientOptions = append(clientOptions, option.WithEndpoint(endpoint))
}

client, err := genai.NewClient(ctx, clientOptions...)
if err != nil {
return nil, fmt.Errorf("calling gemini API: %w", err)
}

return &genAIClientWrapper{client: client}, nil
}

func generateSolutionByGemini(client GenAIClient, issues []*issue.Issue) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()

model := client.GenerativeModel(GeminiModel)
cachedAutofix := make(map[string]string)
for _, issue := range issues {
if val, ok := cachedAutofix[issue.What]; ok {
issue.Autofix = val
continue
}

prompt := fmt.Sprintf(AIPrompt, issue.What)
resp, err := model.GenerateContent(ctx, prompt)
if err != nil {
return fmt.Errorf("gemini generating content: %w", err)
}

if resp == "" {
return errors.New("gemini no candidates found")
}

issue.Autofix = resp
cachedAutofix[issue.What] = issue.Autofix
}
return nil
}

// GenerateSolution generates a solution for the given issues using the specified AI provider
func GenerateSolution(aiApiProvider, aiApiKey, endpoint string, issues []*issue.Issue) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()

var client GenAIClient

switch aiApiProvider {
case GeminiProvider:
var err error
client, err = NewGenAIClient(ctx, aiApiKey, endpoint)
if err != nil {
return fmt.Errorf("generate solution error: %w", err)
}
default:
return errors.New("ai provider not supported")
}

defer client.Close()

return generateSolutionByGemini(client, issues)
}
114 changes: 114 additions & 0 deletions autofix/ai_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
package autofix

import (
"context"
"errors"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"

"github.com/securego/gosec/v2/issue"
)

// MockGenAIClient is a mock of the GenAIClient interface
type MockGenAIClient struct {
mock.Mock
}

func (m *MockGenAIClient) Close() error {
args := m.Called()
return args.Error(0)
}

func (m *MockGenAIClient) GenerativeModel(name string) GenAIGenerativeModel {
args := m.Called(name)
return args.Get(0).(GenAIGenerativeModel)
}

// MockGenAIGenerativeModel is a mock of the GenAIGenerativeModel interface
type MockGenAIGenerativeModel struct {
mock.Mock
}

func (m *MockGenAIGenerativeModel) GenerateContent(ctx context.Context, prompt string) (string, error) {
args := m.Called(ctx, prompt)
return args.String(0), args.Error(1)
}

func TestGenerateSolutionByGemini_Success(t *testing.T) {
// Arrange
issues := []*issue.Issue{
{What: "Example issue 1"},
}

mockClient := new(MockGenAIClient)
mockModel := new(MockGenAIGenerativeModel)
mockClient.On("GenerativeModel", GeminiModel).Return(mockModel)
mockModel.On("GenerateContent", mock.Anything, mock.Anything).Return("Autofix for issue 1", nil)

// Act
err := generateSolutionByGemini(mockClient, issues)

// Assert
assert.NoError(t, err)
assert.Equal(t, "Autofix for issue 1", issues[0].Autofix)
mockClient.AssertExpectations(t)
mockModel.AssertExpectations(t)
}

func TestGenerateSolutionByGemini_NoCandidates(t *testing.T) {
// Arrange
issues := []*issue.Issue{
{What: "Example issue 2"},
}

mockClient := new(MockGenAIClient)
mockModel := new(MockGenAIGenerativeModel)
mockClient.On("GenerativeModel", GeminiModel).Return(mockModel)
mockModel.On("GenerateContent", mock.Anything, mock.Anything).Return("", nil)

// Act
err := generateSolutionByGemini(mockClient, issues)

// Assert
assert.Error(t, err)
assert.Equal(t, "gemini no candidates found", err.Error())
mockClient.AssertExpectations(t)
mockModel.AssertExpectations(t)
}

func TestGenerateSolutionByGemini_APIError(t *testing.T) {
// Arrange
issues := []*issue.Issue{
{What: "Example issue 3"},
}

mockClient := new(MockGenAIClient)
mockModel := new(MockGenAIGenerativeModel)
mockClient.On("GenerativeModel", GeminiModel).Return(mockModel)
mockModel.On("GenerateContent", mock.Anything, mock.Anything).Return("", errors.New("API error"))

// Act
err := generateSolutionByGemini(mockClient, issues)

// Assert
assert.Error(t, err)
assert.Equal(t, "gemini generating content: API error", err.Error())
mockClient.AssertExpectations(t)
mockModel.AssertExpectations(t)
}

func TestGenerateSolution_UnsupportedProvider(t *testing.T) {
// Arrange
issues := []*issue.Issue{
{What: "Example issue 4"},
}

// Act
err := GenerateSolution("unsupported-provider", "test-api-key", "", issues)

// Assert
assert.Error(t, err)
assert.Equal(t, "ai provider not supported", err.Error())
}
18 changes: 18 additions & 0 deletions cmd/gosec/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"strings"

"github.com/securego/gosec/v2"
"github.com/securego/gosec/v2/autofix"
"github.com/securego/gosec/v2/cmd/vflag"
"github.com/securego/gosec/v2/issue"
"github.com/securego/gosec/v2/report"
Expand Down Expand Up @@ -149,6 +150,15 @@ var (
// flagTerse shows only the summary of scan discarding all the logs
flagTerse = flag.Bool("terse", false, "Shows only the results and summary")

// AI platform provider to generate solutions to issues
flagAiApiProvider = flag.String("ai-api-provider", "", "AI platform provider to generate solutions to issues")

// key to implementing AI provider services
flagAiApiKey = flag.String("ai-api-key", "", "key to implementing AI provider services")

// endpoint to the AI provider
flagAiEndpoint = flag.String("ai-endpoint", "", "endpoint to the AI provider")

// exclude the folders from scan
flagDirsExclude arrayFlags

Expand Down Expand Up @@ -457,6 +467,14 @@ func main() {

reportInfo := gosec.NewReportInfo(issues, metrics, errors).WithVersion(Version)

// Call AI request to solve the issues
if *flagAiApiProvider != "" && *flagAiApiKey != "" {
err := autofix.GenerateSolution(*flagAiApiProvider, *flagAiApiKey, *flagAiEndpoint, issues)
if err != nil {
logger.Print(err)
tran-the-lam marked this conversation as resolved.
Show resolved Hide resolved
}
}

if *flagOutput == "" || *flagStdOut {
fileFormat := getPrintedFormat(*flagFormat, *flagVerbose)
if err := printReport(fileFormat, *flagColor, rootPaths, reportInfo); err != nil {
Expand Down
35 changes: 34 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,63 @@ module github.com/securego/gosec/v2

require (
github.com/ccojocar/zxcvbn-go v1.0.2
github.com/google/generative-ai-go v0.17.0
github.com/google/uuid v1.6.0
github.com/gookit/color v1.5.4
github.com/lib/pq v1.10.9
github.com/mozilla/tls-observatory v0.0.0-20210609171429-7bc42856d2e5
github.com/onsi/ginkgo/v2 v2.20.0
github.com/onsi/gomega v1.34.1
github.com/stretchr/testify v1.9.0
golang.org/x/crypto v0.26.0
golang.org/x/lint v0.0.0-20210508222113-6edffad5e616
golang.org/x/text v0.17.0
golang.org/x/tools v0.24.0
google.golang.org/api v0.186.0
gopkg.in/yaml.v3 v3.0.1
)

require (
cloud.google.com/go v0.115.0 // indirect
cloud.google.com/go/ai v0.8.0 // indirect
cloud.google.com/go/auth v0.6.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect
cloud.google.com/go/compute/metadata v0.3.0 // indirect
cloud.google.com/go/longrunning v0.5.7 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.5 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0 // indirect
go.opentelemetry.io/otel v1.26.0 // indirect
go.opentelemetry.io/otel/metric v1.26.0 // indirect
go.opentelemetry.io/otel/trace v1.26.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.20.0 // indirect
golang.org/x/net v0.28.0 // indirect
golang.org/x/oauth2 v0.21.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.23.0 // indirect
golang.org/x/time v0.5.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240617180043-68d350f18fd4 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240617180043-68d350f18fd4 // indirect
google.golang.org/grpc v1.64.1 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)

go 1.20
go 1.21

toolchain go1.22.5
Loading
Loading