Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adds ways to detect more pipelines in mconvert apkbuild #1235

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

adds ways to detect more pipelines in mconvert apkbuild #1235

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
70 changes: 43 additions & 27 deletions pkg/convert/apkbuild/apkbuild.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package apkbuild

import (
"bytes"
"context"
"crypto/sha256"
"crypto/sha512"
Expand All @@ -10,13 +11,15 @@
"net/url"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"

rlhttp "chainguard.dev/melange/pkg/http"
"chainguard.dev/melange/pkg/manifest"
"github.com/chainguard-dev/clog"
"mvdan.cc/sh/v3/syntax"

apkotypes "chainguard.dev/apko/pkg/build/types"
"chainguard.dev/melange/pkg/config"
Expand Down Expand Up @@ -328,6 +331,7 @@
}

// maps APKBUILD values to mconvert
// Question: Should not we return an error here for better error handling?
func (c ApkConvertor) mapconvert() {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return error here for better error handling now that we detect pipelines?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I would 👍

c.GeneratedMelangeConfig.Package.Name = c.Apkbuild.Pkgname
c.GeneratedMelangeConfig.Package.Description = c.Apkbuild.Pkgdesc
Expand All @@ -345,33 +349,45 @@
c.GeneratedMelangeConfig.Package.Scriptlets.Trigger.Script = "FIXME"
}

// if c.Apkbuild.Funcs["build"] != nil {
// // todo lets check the command and add the correct cmake | make | meson mconvert pipelines
// //build := c.Apkbuild.Funcs["build"]
//}

// switch c.Apkbuild.BuilderType {
//
// case BuilderTypeCMake:
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/configure"})
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/build"})
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/install"})
//
// case BuilderTypeMeson:
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/configure"})
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/compile"})
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/install"})
//
// case BuilderTypeMake:
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/configure"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/make"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/make-install"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "strip"})

//default:
// c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "# FIXME"})
//
//}
if c.Apkbuild.Funcs["build"] != nil {
var output bytes.Buffer
printer := syntax.NewPrinter()
err := printer.Print(&output, c.Apkbuild.Funcs["build"])
if err != nil {
fmt.Errorf("failed to print build function: %w", err)

Check failure on line 357 in pkg/convert/apkbuild/apkbuild.go

View workflow job for this annotation

GitHub Actions / lint 

unusedresult: result of fmt.Errorf call not used (govet)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fmt.Errorf("failed to print build function: %w", err)
return fmt.Errorf("failed to print build function: %w", err)

}

outputStr := output.String()

// Define the regular expression to find options in the form of -<option>=<value> or --<option>=<value>
re := regexp.MustCompile(`--?\w[-\w]*=[^\\\s]+`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a few test cases here to make sure this does what we expect.


// Find all matches
matches := re.FindAllString(outputStr, -1)

// Join matches into a single string separated by space and backslash
opts := strings.Join(matches, " \\\n")
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen opts being added for the existing pipelines but I still attempted to add here but it uses regex, are we intentionally avoiding it or should we add it? If yes, I can refactor it to be used by other pipelines too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question -- I wonder if some of these opts would overlap with the existing pipeline inputs? It might make sense to try to map some of these to inputs if we can.


switch {
case strings.Contains(outputStr, "cmake"):
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/configure"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/build"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "cmake/install"})
case strings.Contains(outputStr, "meson"):
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/configure", With: map[string]string{
"opts": opts,
}})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/compile"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "meson/install"})
case strings.Contains(outputStr, "autoconf"):
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/configure"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/make"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "autoconf/make-install"})
c.GeneratedMelangeConfig.Pipeline = append(c.GeneratedMelangeConfig.Pipeline, config.Pipeline{Uses: "strip"})
// default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to keep the default as "make" as before, or at least scaffold out the metadata with no pipeline?

// return "unknown", nil
}
}

for _, subPackage := range c.Apkbuild.Subpackages {
subpackage := config.Subpackage{
Expand Down