Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build windows msi for arm and amd #1796

Merged
merged 21 commits into from
Aug 7, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions cmd/launcher/svc_config_windows.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ package main
import (
"context"
"log/slog"
"time"

"github.com/kolide/launcher/pkg/launcher"

Expand Down Expand Up @@ -64,6 +65,8 @@ func checkServiceConfiguration(logger *slog.Logger, opts *launcher.Options) {
checkDependOnService(launcherServiceKey, logger)

checkRestartActions(logger)

setRecoveryActions(logger)
}

// checkDelayedAutostart checks the current value of `DelayedAutostart` (whether to wait ~2 minutes
Expand Down Expand Up @@ -184,3 +187,56 @@ func checkRestartActions(logger *slog.Logger) {

logger.Log(context.TODO(), slog.LevelInfo, "successfully set RecoveryActionsOnNonCrashFailures flag")
}

// setRecoveryActions sets the recovery actions for the launcher service.
// previously defined via wix ServicConfig Element (Util Extension) https://wixtoolset.org/docs/v3/xsd/util/serviceconfig/
func setRecoveryActions(logger *slog.Logger) {
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
sman, err := mgr.Connect()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is what we're doing everywhere here, I wonder if we should consolidate and only open it once, instead of the rapid opening and closing. (If we think so, it's fine as a followup)

if err != nil {
logger.Log(context.TODO(), slog.LevelError,
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
"connecting to service control manager",
"err", err,
)

return
}

defer sman.Disconnect()

launcherService, err := sman.OpenService(launcherServiceName)
if err != nil {
logger.Log(context.TODO(), slog.LevelError,
"opening the launcher service from control manager",
"err", err,
)

return
}

defer launcherService.Close()

recoveryActions := []mgr.RecoveryAction{
{
// first failure
Type: mgr.ServiceRestart,
Delay: 5 * time.Second,
},
{
// second failure
Type: mgr.ServiceRestart,
Delay: 5 * time.Second,
},
{
// subsequent failures
Type: mgr.ServiceRestart,
Delay: 5 * time.Second,
},
}

if err := launcherService.SetRecoveryActions(recoveryActions, 24*60*60); err != nil { // 24 hours
logger.Log(context.TODO(), slog.LevelError,
"setting RecoveryActions",
"err", err,
)
}
}
54 changes: 30 additions & 24 deletions cmd/package-builder/package-builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,11 @@ func runMake(args []string) error {
env.String("LAUNCHER_VERSION", "stable"),
"What TUF channel to download launcher from. Supports filesystem paths",
)
flLauncherArmVersion = flagset.String(
"launcher_arm_version",
env.String("LAUNCHER_ARM_VERSION", "stable"),
"What TUF channel to download launcher from for ARM. Supports filesystem paths",
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this future-proofing in case we want to do more granular releases for launcher in the future (e.g. "bug in launcher version x.y.z on ARM only, so leave stable at x.y.z for all other arches and x.y.y for launcher ARM")? Or are we adding this flag for another reason?

Copy link
Contributor Author

@James-Pickett James-Pickett Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this just so that I could specify a local arm build to package. I can't imagine a situation where we would want the arm and amd binaries to have different code.

This has me thinking though that maybe we should add in some safe guards to ensure the arm and amd versions are the same. If we some how managed to get into a situation where TUF updated one and not the other, we could end up with a package where different arches are on different versions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find a coherent way to verify the versions are the same. However, I set up the flag so that it's default is an empty string and it will just set that flag to what ever launcher_version is if no launcher_arm_version is provided.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure there's a good way to validate a launcher version, without invoking it. Maaaybe we can string parse it, but I'm not sure it's worth it. (And given what Zack recently learned about file versions and MSIs, let's not go there...)

Maybe this is an argument for being less clever. Command line either takes launcher_version= and downloads from TUF (we should have enough metadata there to validate that stable is the same) Or use_local_binary=... and it ignores TUF

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that's kinda what would happen. So maybe I'm over thinking it. Maybe we validate the versions from tuf, and skip that validation on local paths.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invoking it could be troublesome since we'll have 2 binaries with to different arches, An arm64 machine will probably emulate the amd64 binary and run it, but the reverse doesn't work.

I think just having separate flags for paths is probably good with only a single option for launcher_version which is downloaded form tuf for both arches.

flExtensionVersion = flagset.String(
"extension_version",
env.String("EXTENSION_VERSION", "stable"),
Expand Down Expand Up @@ -230,30 +235,31 @@ func runMake(args []string) error {
}

packageOptions := packaging.PackageOptions{
PackageVersion: *flPackageVersion,
OsqueryVersion: *flOsqueryVersion,
OsqueryFlags: flOsqueryFlags,
LauncherVersion: *flLauncherVersion,
ExtensionVersion: *flExtensionVersion,
Hostname: *flHostname,
Secret: *flEnrollSecret,
AppleSigningKey: *flSigningKey,
Transport: *flTransport,
Insecure: *flInsecure,
InsecureTransport: *flInsecureTransport,
UpdateChannel: *flUpdateChannel,
InitialRunner: *flInitialRunner,
Identifier: *flIdentifier,
OmitSecret: *flOmitSecret,
CertPins: *flCertPins,
RootPEM: *flRootPEM,
BinRootDir: *flBinRootDir,
CacheDir: cacheDir,
TufServerURL: *flTufURL,
MirrorURL: *flMirrorURL,
WixPath: *flWixPath,
WixSkipCleanup: *flWixSkipCleanup,
DisableService: *flDisableService,
PackageVersion: *flPackageVersion,
OsqueryVersion: *flOsqueryVersion,
OsqueryFlags: flOsqueryFlags,
LauncherVersion: *flLauncherVersion,
LauncherArmVersion: *flLauncherArmVersion,
ExtensionVersion: *flExtensionVersion,
Hostname: *flHostname,
Secret: *flEnrollSecret,
AppleSigningKey: *flSigningKey,
Transport: *flTransport,
Insecure: *flInsecure,
InsecureTransport: *flInsecureTransport,
UpdateChannel: *flUpdateChannel,
InitialRunner: *flInitialRunner,
Identifier: *flIdentifier,
OmitSecret: *flOmitSecret,
CertPins: *flCertPins,
RootPEM: *flRootPEM,
BinRootDir: *flBinRootDir,
CacheDir: cacheDir,
TufServerURL: *flTufURL,
MirrorURL: *flMirrorURL,
WixPath: *flWixPath,
WixSkipCleanup: *flWixSkipCleanup,
DisableService: *flDisableService,
}

outputDir := *flOutputDir
Expand Down
54 changes: 13 additions & 41 deletions pkg/packagekit/wix/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ type ServiceInstall struct {
Start StartType `xml:",attr,omitempty"`
Type string `xml:",attr,omitempty"`
Vital YesNoType `xml:",attr,omitempty"` // The overall install should fail if this service fails to install
UtilServiceConfig *UtilServiceConfig `xml:",omitempty"`
ServiceConfig *ServiceConfig `xml:",omitempty"`
ServiceDependency *ServiceDependency `xml:",omitempty"`
}
Expand Down Expand Up @@ -90,32 +89,14 @@ type ServiceControl struct {
// This is used needed to set DelayedAutoStart
type ServiceConfig struct {
directionless marked this conversation as resolved.
Show resolved Hide resolved
// TODO: this should need a namespace, and yet. See https://github.com/golang/go/issues/36813
Id string `xml:",attr,omitempty"`
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
XMLName xml.Name `xml:"http://schemas.microsoft.com/wix/2006/wi ServiceConfig"`
DelayedAutoStart YesNoType `xml:",attr,omitempty"`
OnInstall YesNoType `xml:",attr,omitempty"`
OnReinstall YesNoType `xml:",attr,omitempty"`
OnUninstall YesNoType `xml:",attr,omitempty"`
}

// UtilServiceConfig implements
// http://wixtoolset.org/documentation/manual/v3/xsd/util/serviceconfig.html
// This is used to set FailureActions. There are some
// limitations. Notably, reset period is in days here, though the
// underlying `sc.exe` command supports seconds. (See
// https://github.com/wixtoolset/issues/issues/5963)
//
// Docs are a bit confusing. This schema is supported, and should
// work. The non-util ServiceConfig generates unsupported CNDL1150
// errors.
type UtilServiceConfig struct {
XMLName xml.Name `xml:"http://schemas.microsoft.com/wix/UtilExtension ServiceConfig"`
FirstFailureActionType string `xml:",attr,omitempty"`
SecondFailureActionType string `xml:",attr,omitempty"`
ThirdFailureActionType string `xml:",attr,omitempty"`
RestartServiceDelayInSeconds int `xml:",attr,omitempty"`
ResetPeriodInDays int `xml:",attr,omitempty"`
}

// Service represents a wix service. It provides an interface to both
// ServiceInstall and ServiceControl.
type Service struct {
Expand Down Expand Up @@ -194,16 +175,6 @@ func ServiceArgs(args []string) ServiceOpt {

// New returns a service
func NewService(matchString string, opts ...ServiceOpt) *Service {
// Set some defaults. It's not clear we can reset in under a
// day. See https://github.com/wixtoolset/issues/issues/5963
utilServiceConfig := &UtilServiceConfig{
FirstFailureActionType: "restart",
SecondFailureActionType: "restart",
ThirdFailureActionType: "restart",
ResetPeriodInDays: 1,
RestartServiceDelayInSeconds: 5,
}

serviceConfig := &ServiceConfig{
OnInstall: Yes,
OnReinstall: Yes,
Expand All @@ -215,16 +186,16 @@ func NewService(matchString string, opts ...ServiceOpt) *Service {
// probably better to specific a ServiceName, but this might be an
// okay default.
defaultName := cleanServiceName(strings.TrimSuffix(matchString, ".exe") + ".svc")

si := &ServiceInstall{
Name: defaultName,
Id: defaultName,
Account: `[SERVICEACCOUNT]`, // Wix resolves this to `LocalSystem`
Start: StartAuto,
Type: "ownProcess",
ErrorControl: ErrorControlNormal,
Vital: Yes,
UtilServiceConfig: utilServiceConfig,
ServiceConfig: serviceConfig,
Name: defaultName,
Id: defaultName,
Account: `[SERVICEACCOUNT]`, // Wix resolves this to `LocalSystem`
Start: StartAuto,
Type: "ownProcess",
ErrorControl: ErrorControlNormal,
Vital: Yes,
ServiceConfig: serviceConfig,
}

sc := &ServiceControl{
Expand All @@ -237,8 +208,9 @@ func NewService(matchString string, opts ...ServiceOpt) *Service {
}

s := &Service{
matchString: matchString,
expectedCount: 1,
matchString: matchString,
// one count for arm64 and ond for amd64
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
expectedCount: 2,
count: 0,
serviceInstall: si,
serviceControl: sc,
Expand Down
11 changes: 8 additions & 3 deletions pkg/packagekit/wix/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,22 @@ func TestService(t *testing.T) {
require.NoError(t, err)
require.False(t, expectFalse)

// first match
expectTrue, err := service.Match("daemon.exe")
require.NoError(t, err)
require.True(t, expectTrue)

// Should error. count now exceeds expectedCount.
// second match
expectTrue2, err := service.Match("daemon.exe")
require.Error(t, err)
require.NoError(t, err)
require.True(t, expectTrue2)

// third match, should error
expectTrue3, err := service.Match("daemon.exe")
require.Error(t, err)
require.True(t, expectTrue3)

expectedXml := `<ServiceInstall Account="[SERVICEACCOUNT]" ErrorControl="normal" Id="DaemonSvc" Name="DaemonSvc" Start="auto" Type="ownProcess" Vital="yes">
<ServiceConfig xmlns="http://schemas.microsoft.com/wix/UtilExtension" FirstFailureActionType="restart" SecondFailureActionType="restart" ThirdFailureActionType="restart" RestartServiceDelayInSeconds="5" ResetPeriodInDays="1"></ServiceConfig>
<ServiceConfig xmlns="http://schemas.microsoft.com/wix/2006/wi" DelayedAutoStart="no" OnInstall="yes" OnReinstall="yes"></ServiceConfig>
</ServiceInstall>
<ServiceControl Name="DaemonSvc" Id="DaemonSvc" Remove="uninstall" Start="install" Stop="both" Wait="no"></ServiceControl>`
Expand Down
63 changes: 63 additions & 0 deletions pkg/packagekit/wix/wix.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (

"github.com/go-kit/kit/log/level"
"github.com/kolide/kit/fsutil"
"github.com/kolide/kit/ulid"
"github.com/kolide/launcher/pkg/contexts/ctxlog"
)

Expand Down Expand Up @@ -223,19 +224,81 @@ func (wo *wixTool) addServices(ctx context.Context) error {
}
defer heatWrite.Close()

type archSpecificBinDir string

const (
none archSpecificBinDir = ""
amd64 archSpecificBinDir = "amd64"
arm64 archSpecificBinDir = "arm64"
)
currentArchSpecificBinDir := none

baseSvcName := wo.services[0].serviceInstall.Id

lines := strings.Split(string(heatContent), "\n")
for _, line := range lines {

if currentArchSpecificBinDir != none && strings.Contains(line, "</Directory>") {
// were in a arch specific bin dir that we want to remove, don't write closing tag
currentArchSpecificBinDir = none
continue
}

if strings.Contains(line, "Directory") {
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
if strings.Contains(line, string(amd64)) {
// were in a arch specific bin dir that we want to remove so when we hit closing tag, we'll skip it
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
currentArchSpecificBinDir = amd64
continue
}

if strings.Contains(line, string(arm64)) {
// were in a arch specific bin dir that we want to remove so when we hit closing tag, we'll skip it
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
currentArchSpecificBinDir = arm64
continue
}
}

heatWrite.WriteString(line)
heatWrite.WriteString("\n")

for _, service := range wo.services {

isMatch, err := service.Match(line)
if err != nil {
return fmt.Errorf("match error: %w", err)
}

if isMatch {
if currentArchSpecificBinDir == none {
return fmt.Errorf("service found, but not in a bin directory")
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
}

// make sure elements are not duplicated in any service
serviceId := fmt.Sprintf("%s%s", baseSvcName, ulid.New())
service.serviceControl.Id = serviceId
service.serviceInstall.Id = serviceId
service.serviceInstall.ServiceConfig.Id = serviceId

// create a condition based on architecture
// have to format in the "%P" in "%PROCESSOR_ARCHITECTURE"
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
heatWrite.WriteString(fmt.Sprintf(`<Condition> %sROCESSOR_ARCHITECTURE="%s" </Condition>`, "%P", strings.ToUpper(string(currentArchSpecificBinDir))))
heatWrite.WriteString("\n")

if err := service.Xml(heatWrite); err != nil {
return fmt.Errorf("adding service: %w", err)
}

continue
}

if strings.Contains(line, "osqueryd.exe") {
if currentArchSpecificBinDir == none {
return fmt.Errorf("osqueryd.exe found, but not in a bin directory")
}

// create a condition based on architecture
heatWrite.WriteString(fmt.Sprintf(`<Condition> %sROCESSOR_ARCHITECTURE="%s" </Condition>`, "%P", currentArchSpecificBinDir))
heatWrite.WriteString("\n")
}
}
}
Expand Down
8 changes: 7 additions & 1 deletion pkg/packaging/detectLauncherVersion.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,13 @@ func (p *PackageOptions) detectLauncherVersion(ctx context.Context) error {
logger := log.With(ctxlog.FromContext(ctx), "library", "detectLauncherVersion")
level.Debug(logger).Log("msg", "Attempting launcher autodetection")

launcherPath := p.launcherLocation(filepath.Join(p.packageRoot, p.binDir))
binDir := filepath.Join(p.packageRoot, p.binDir)
if p.target.Platform != Darwin {
James-Pickett marked this conversation as resolved.
Show resolved Hide resolved
binDir = filepath.Join(binDir, string(p.target.Arch))
}

launcherPath := p.launcherLocation(binDir)

stdout, err := p.execOut(ctx, launcherPath, "-version")
if err != nil {
return fmt.Errorf("failed to exec -- possibly can't autodetect while cross compiling: out `%s`: %w", stdout, err)
Expand Down
8 changes: 8 additions & 0 deletions pkg/packaging/fetch.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ func FetchBinary(ctx context.Context, localCacheDir, name, binaryName, channelOr
return "", errors.New("empty cache dir argument")
}

// put binaries in arch specific directory to avoid naming collisions in wix msi building
// where a single destination will have multiple, mutally exclusive sources
localCacheDir = filepath.Join(localCacheDir, string(target.Arch))

if err := os.MkdirAll(localCacheDir, fsutil.DirMode); err != nil {
return "", fmt.Errorf("could not create cache directory: %w", err)
}

localBinaryPath := filepath.Join(localCacheDir, fmt.Sprintf("%s-%s-%s", name, target.Platform, channelOrVersion), binaryName)
localPackagePath := filepath.Join(localCacheDir, fmt.Sprintf("%s-%s-%s.tar.gz", name, target.Platform, channelOrVersion))

Expand Down
Loading
Loading