Skip to content

Commit

Permalink
Change gce to gcp (cosmetic only)
Browse files Browse the repository at this point in the history
This change might not break pipelines and auto tests.
  • Loading branch information
HuijingHei authored and cgwalters committed Apr 11, 2023
1 parent 7a1f61e commit 25ff397
Show file tree
Hide file tree
Showing 20 changed files with 79 additions and 79 deletions.
2 changes: 1 addition & 1 deletion docs/building-fcos.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ There are various public cloud options that provide either bare metal or nested
virt, such as:

- [Packet](https://www.packet.com/)
- [GCE nested virt](https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances)
- [GCP nested virt](https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances)
- EC2 `i3.metal` instances
- [IBM Bare Metal](https://www.ibm.com/cloud/bare-metal-servers)
- etc.
Expand Down
4 changes: 2 additions & 2 deletions docs/kola.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Kola is a framework for testing software integration in CoreOS systems
across multiple platforms. It is primarily designed to operate within
the CoreOS Assembler for testing software that has landed in the OS image.

Kola supports running tests on multiple platforms, currently QEMU, GCE,
Kola supports running tests on multiple platforms, currently QEMU, GCP,
AWS, VMware VSphere, Packet, and OpenStack. In the future systemd-nspawn and
other platforms may be added.
Local platforms do not rely on access to the Internet as a design
Expand Down Expand Up @@ -241,5 +241,5 @@ In order to see the logs for these tests you must enter the `tmp/kola/name_of_th

`kola run -p=gce --gce-image=projects/fedora-coreos-cloud/global/images/fedora-coreos-37-20230227-20-2-gcp-x86-64 --gce-json-key=/data/gce.json --gce-project=fedora-coreos-testing basic` This will run the basic tests on GCP using default machine type `n1-standard-1`.
- `gce-image` is in the format of `projects/<GCP Image Project>/global/images/<GCP Image Name>`, to find related info refer to https://builds.coreos.fedoraproject.org/browser?stream=testing-devel&arch=x86_64.
- `gce-json-key` is using a service account's JSON key for authentication, how to create service account keys refer to https://github.com/coreos/coreos-assembler/blob/main/docs/mantle/credentials.md#gce.
- `gce-json-key` is using a service account's JSON key for authentication, how to create service account keys refer to https://github.com/coreos/coreos-assembler/blob/main/docs/mantle/credentials.md#gcp.
- `gce-project` is meant for testing in the specified project, or it will use the same as `<GCP Image Project>`.
2 changes: 1 addition & 1 deletion docs/mantle/credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ via the `--azure-credentials` option on the command line.
}
```

## gce
## gcp

If you want to create a service account's JSON key for authentication, refer to [create service account keys](https://cloud.google.com/iam/docs/).

Expand Down
8 changes: 4 additions & 4 deletions mantle/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ the Container Linux SDK for testing software that has landed in the OS image.
Ideally, all software needed for a test should be included by building
it into the image from the SDK.

Kola supports running tests on multiple platforms, currently QEMU, GCE,
Kola supports running tests on multiple platforms, currently QEMU, GCP,
AWS, VMware VSphere, Packet, and OpenStack. In the future systemd-nspawn and other
platforms may be added.
Local platforms do not rely on access to the Internet as a design
Expand Down Expand Up @@ -167,7 +167,7 @@ each with their own command: pre-release and release. Both of these commands are

#### plume pre-release
The pre-release command does as much of the release process as possible without making anything public.
This includes uploading images to cloud providers (except those like gce which don't allow us to upload
This includes uploading images to cloud providers (except those like gcp which don't allow us to upload
images without making them public).

### plume release
Expand Down Expand Up @@ -312,8 +312,8 @@ The JSON file exported to the variable `AZURE_AUTH_LOCATION` should be generated
}
```

### gce
`gce` uses `~/.config/gce.json`, which contains a JSON-formatted service
### gcp
`gcp` uses `~/.config/gce.json`, which contains a JSON-formatted service
account key. This can be downloaded from the Google Cloud console under
IAM > Service Accounts > [account] > Keys.

Expand Down
6 changes: 3 additions & 3 deletions mantle/auth/google.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ import (
"golang.org/x/oauth2/google"
)

const GCEConfigPath = ".config/gce.json"
const GCPConfigPath = ".config/gce.json"

var scopes = []string{
"https://www.googleapis.com/auth/devstorage.full_control",
"https://www.googleapis.com/auth/compute",
}

// GoogleServiceClient fetchs a token from Google Compute Engine's
// metadata service. This should be used on GCE vms. The Default account
// metadata service. This should be used on GCP vms. The Default account
// is used.
func GoogleServiceClient() *http.Client {
return &http.Client{
Expand All @@ -56,7 +56,7 @@ func GoogleClientFromKeyFile(path string, scope ...string) (*http.Client, error)
if err != nil {
return nil, err
}
path = filepath.Join(user.HomeDir, GCEConfigPath)
path = filepath.Join(user.HomeDir, GCPConfigPath)
}
b, err := os.ReadFile(path)
if err != nil {
Expand Down
12 changes: 6 additions & 6 deletions mantle/cmd/kola/kola.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ func writeProps() error {
Server string `json:"server"`
BaseVMName string `json:"base_vm_name"`
}
type GCE struct {
type GCP struct {
Image string `json:"image"`
MachineType string `json:"type"`
}
Expand Down Expand Up @@ -323,7 +323,7 @@ func writeProps() error {
Azure Azure `json:"azure"`
DO DO `json:"do"`
ESX ESX `json:"esx"`
GCE GCE `json:"gce"`
GCP GCP `json:"gcp"`
OpenStack OpenStack `json:"openstack"`
Packet Packet `json:"packet"`
QEMU QEMU `json:"qemu"`
Expand Down Expand Up @@ -355,9 +355,9 @@ func writeProps() error {
Server: kola.ESXOptions.Server,
BaseVMName: kola.ESXOptions.BaseVMName,
},
GCE: GCE{
Image: kola.GCEOptions.Image,
MachineType: kola.GCEOptions.MachineType,
GCP: GCP{
Image: kola.GCPOptions.Image,
MachineType: kola.GCPOptions.MachineType,
},
OpenStack: OpenStack{
Region: kola.OpenStackOptions.Region,
Expand Down Expand Up @@ -605,7 +605,7 @@ func syncFindParentImageOptions() error {
return err
}
case "gce":
kola.GCEOptions.Image, err = parentCosaBuild.FindGCPImage()
kola.GCPOptions.Image, err = parentCosaBuild.FindGCPImage()
if err != nil {
return err
}
Expand Down
26 changes: 13 additions & 13 deletions mantle/cmd/kola/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,16 +113,16 @@ func init() {
sv(&kola.ESXOptions.Profile, "esx-profile", "", "ESX profile (default \"default\")")
sv(&kola.ESXOptions.BaseVMName, "esx-base-vm", "", "ESX base VM name")

// gce-specific options
sv(&kola.GCEOptions.Image, "gce-image", "", "GCE image, full api endpoints names are accepted if resource is in a different project")
sv(&kola.GCEOptions.Project, "gce-project", "fedora-coreos-devel", "GCE project name")
sv(&kola.GCEOptions.Zone, "gce-zone", "us-central1-a", "GCE zone name")
sv(&kola.GCEOptions.MachineType, "gce-machinetype", "n1-standard-1", "GCE machine type")
sv(&kola.GCEOptions.DiskType, "gce-disktype", "pd-ssd", "GCE disk type")
sv(&kola.GCEOptions.Network, "gce-network", "default", "GCE network")
sv(&kola.GCEOptions.ServiceAcct, "gce-service-account", "", "GCE service account to attach to instance (default project default)")
bv(&kola.GCEOptions.ServiceAuth, "gce-service-auth", false, "for non-interactive auth when running within GCE")
sv(&kola.GCEOptions.JSONKeyFile, "gce-json-key", "", "use a service account's JSON key for authentication (default \"~/"+auth.GCEConfigPath+"\")")
// gcp-specific options
sv(&kola.GCPOptions.Image, "gce-image", "", "GCP image, full api endpoints names are accepted if resource is in a different project")
sv(&kola.GCPOptions.Project, "gce-project", "fedora-coreos-devel", "GCP project name")
sv(&kola.GCPOptions.Zone, "gce-zone", "us-central1-a", "GCP zone name")
sv(&kola.GCPOptions.MachineType, "gce-machinetype", "n1-standard-1", "GCP machine type")
sv(&kola.GCPOptions.DiskType, "gce-disktype", "pd-ssd", "GCP disk type")
sv(&kola.GCPOptions.Network, "gce-network", "default", "GCP network")
sv(&kola.GCPOptions.ServiceAcct, "gce-service-account", "", "GCP service account to attach to instance (default project default)")
bv(&kola.GCPOptions.ServiceAuth, "gce-service-auth", false, "for non-interactive auth when running within GCP")
sv(&kola.GCPOptions.JSONKeyFile, "gce-json-key", "", "use a service account's JSON key for authentication (default \"~/"+auth.GCPConfigPath+"\")")

// openstack-specific options
sv(&kola.OpenStackOptions.ConfigPath, "openstack-config-file", "", "Path to a clouds.yaml formatted OpenStack config file. The underlying library defaults to ./clouds.yaml")
Expand Down Expand Up @@ -364,13 +364,13 @@ func syncCosaOptions() error {
}
case "gce":
// Pick up the GCP image from the build metadata
if kola.GCEOptions.Image == "" && kola.CosaBuild.Meta.Gcp != nil {
kola.GCEOptions.Image =
if kola.GCPOptions.Image == "" && kola.CosaBuild.Meta.Gcp != nil {
kola.GCPOptions.Image =
fmt.Sprintf("projects/%s/global/images/%s",
kola.CosaBuild.Meta.Gcp.ImageProject,
kola.CosaBuild.Meta.Gcp.ImageName)

fmt.Printf("Using GCP image %s\n", kola.GCEOptions.Image)
fmt.Printf("Using GCP image %s\n", kola.GCPOptions.Image)
}
}

Expand Down
20 changes: 10 additions & 10 deletions mantle/cmd/ore/gcloud/create-image.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ import (
var (
cmdCreateImage = &cobra.Command{
Use: "create-image",
Short: "Create GCE image",
Long: "Create GCE image from an existing file in Google Storage",
Short: "Create GCP image",
Long: "Create GCP image from an existing file in Google Storage",
Run: runCreateImage,
}

Expand All @@ -47,7 +47,7 @@ var (
func init() {
user := os.Getenv("USER")
cmdCreateImage.Flags().StringVar(&createImageFamily, "family",
user, "GCE image group and name prefix")
user, "GCP image group and name prefix")
cmdCreateImage.Flags().StringVar(&createImageBoard, "board",
"amd64-usr", "OS board name")
cmdCreateImage.Flags().StringVar(&createImageVersion, "version",
Expand All @@ -56,10 +56,10 @@ func init() {
"gs://users.developer.core-os.net/"+user+"/boards",
"Storage URL prefix")
cmdCreateImage.Flags().StringVar(&createImageName, "source-name",
"coreos_production_gce.tar.gz",
"coreos_production_gcp.tar.gz",
"Storage image name")
cmdCreateImage.Flags().BoolVar(&createImageForce, "force",
false, "overwrite existing GCE images without prompt")
false, "overwrite existing GCP images without prompt")
GCloud.AddCommand(cmdCreateImage)
}

Expand Down Expand Up @@ -91,7 +91,7 @@ func runCreateImage(cmd *cobra.Command, args []string) {
bucket := gsURL.Host
imageNameGS := strings.TrimPrefix(path.Join(gsURL.Path,
createImageBoard, createImageVersion, createImageName), "/")
imageNameGCE := gceSanitize(createImageFamily + "-" + createImageVersion)
imageNameGCP := gcpSanitize(createImageFamily + "-" + createImageVersion)

ctx := context.Background()
storageAPI, err := storage.NewService(ctx)
Expand All @@ -111,19 +111,19 @@ func runCreateImage(cmd *cobra.Command, args []string) {
os.Exit(1)
}

fmt.Printf("Creating image in GCE: %v...\n", imageNameGCE)
fmt.Printf("Creating image in GCP: %v...\n", imageNameGCP)

// create image on gce
// create image on gcp
storageSrc := fmt.Sprintf("https://storage.googleapis.com/%v/%v", bucket, imageNameGS)
_, pending, err := api.CreateImage(&gcloud.ImageSpec{
Name: imageNameGCE,
Name: imageNameGCP,
SourceImage: storageSrc,
}, createImageForce)
if err == nil {
err = pending.Wait()
}
if err != nil {
fmt.Fprintf(os.Stderr, "Creating GCE image failed: %v\n", err)
fmt.Fprintf(os.Stderr, "Creating GCP image failed: %v\n", err)
os.Exit(1)
}
}
2 changes: 1 addition & 1 deletion mantle/cmd/ore/gcloud/delete-images.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
var (
cmdDeleteImage = &cobra.Command{
Use: "delete-images <name>...",
Short: "Delete GCE images",
Short: "Delete GCP images",
Run: runDeleteImage,
}
)
Expand Down
4 changes: 2 additions & 2 deletions mantle/cmd/ore/gcloud/destroy.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import (
var (
cmdDestroy = &cobra.Command{
Use: "destroy-instances --prefix=<prefix> ",
Short: "destroy cluster on GCE",
Long: "Destroy GCE instances based on name prefix.",
Short: "destroy cluster on GCP",
Long: "Destroy GCP instances based on name prefix.",
Run: runDestroy,
}
)
Expand Down
2 changes: 1 addition & 1 deletion mantle/cmd/ore/gcloud/gc.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
var (
cmdGC = &cobra.Command{
Use: "gc",
Short: "GC resources in GCE",
Short: "GC resources in GCP",
Long: `Delete instances created over the given duration ago.`,
RunE: runGC,

Expand Down
2 changes: 1 addition & 1 deletion mantle/cmd/ore/gcloud/gcloud.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ func init() {
sv(&opts.BaseName, "basename", "kola", "instance name prefix")
sv(&opts.Network, "network", "default", "network name")
sv(&opts.JSONKeyFile, "json-key", "", "use a service account's JSON key for authentication")
GCloud.PersistentFlags().BoolVar(&opts.ServiceAuth, "service-auth", false, "use non-interactive auth when running within GCE")
GCloud.PersistentFlags().BoolVar(&opts.ServiceAuth, "service-auth", false, "use non-interactive auth when running within GCP")

cli.WrapPreRun(GCloud, preauth)
}
Expand Down
2 changes: 1 addition & 1 deletion mantle/cmd/ore/gcloud/image.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
var (
cmdImage = &cobra.Command{
Use: "list-images --prefix=<prefix>",
Short: "List images in GCE",
Short: "List images in GCP",
Run: runImage,
}

Expand Down
2 changes: 1 addition & 1 deletion mantle/cmd/ore/gcloud/list.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
var (
cmdList = &cobra.Command{
Use: "list-instances --prefix=<prefix>",
Short: "List instances on GCE",
Short: "List instances on GCP",
Run: runList,
}
)
Expand Down
34 changes: 17 additions & 17 deletions mantle/cmd/ore/gcloud/upload.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ var (
cmdUpload = &cobra.Command{
Use: "upload",
Short: "Upload os image",
Long: "Upload os image to Google Storage bucket and create image in GCE. Intended for use in SDK.",
Long: "Upload os image to Google Storage bucket and create image in GCP. Intended for use in SDK.",
Run: runUpload,
}

Expand Down Expand Up @@ -62,7 +62,7 @@ func init() {
if err := cmdUpload.MarkFlagRequired("file"); err != nil {
panic(err)
}
cmdUpload.Flags().BoolVar(&uploadForce, "force", false, "overwrite existing GS and GCE images without prompt")
cmdUpload.Flags().BoolVar(&uploadForce, "force", false, "overwrite existing GS and GCP images without prompt")
cmdUpload.Flags().StringVar(&uploadWriteUrl, "write-url", "", "output the uploaded URL to the named file")
cmdUpload.Flags().StringVar(&uploadImageFamily, "family", "", "GCP image family to attach image to")
cmdUpload.Flags().StringVar(&uploadImageDescription, "description", "", "The description that should be attached to the image")
Expand Down Expand Up @@ -101,8 +101,8 @@ func runUpload(cmd *cobra.Command, args []string) {
uploadBucket = gsURL.Host
imageNameGS := strings.TrimPrefix(gsURL.Path+"/"+uploadImageName, "/") + ".tar.gz"

// Sanitize the image name for GCE
imageNameGCE := gceSanitize(uploadImageName)
// Sanitize the image name for GCP
imageNameGCP := gcpSanitize(uploadImageName)

ctx := context.Background()
storageAPI, err := storage.NewService(ctx, option.WithHTTPClient(api.Client()))
Expand Down Expand Up @@ -143,9 +143,9 @@ func runUpload(cmd *cobra.Command, args []string) {
imageStorageURL := fmt.Sprintf("https://storage.googleapis.com/%v/%v", uploadBucket, imageNameGS)

if uploadCreateImage {
fmt.Printf("Creating image in GCE: %v...\n", imageNameGCE)
fmt.Printf("Creating image in GCP: %v...\n", imageNameGCP)
spec := &gcloud.ImageSpec{
Name: imageNameGCE,
Name: imageNameGCP,
Family: uploadImageFamily,
SourceImage: imageStorageURL,
Description: uploadImageDescription,
Expand All @@ -161,7 +161,7 @@ func runUpload(cmd *cobra.Command, args []string) {
// if image already exists ask to delete and try again
if err != nil && strings.HasSuffix(err.Error(), "alreadyExists") {
var ans string
fmt.Printf("Image %v already exists on GCE. Overwrite? (y/n):", imageNameGCE)
fmt.Printf("Image %v already exists on GCP. Overwrite? (y/n):", imageNameGCP)
if _, err = fmt.Scan(&ans); err != nil {
fmt.Fprintf(os.Stderr, "Scanning overwrite input: %v", err)
os.Exit(1)
Expand All @@ -174,25 +174,25 @@ func runUpload(cmd *cobra.Command, args []string) {
err = pending.Wait()
}
if err != nil {
fmt.Fprintf(os.Stderr, "Creating GCE image failed: %v\n", err)
fmt.Fprintf(os.Stderr, "Creating GCP image failed: %v\n", err)
os.Exit(1)
}
fmt.Printf("Image %v sucessfully created in GCE\n", imageNameGCE)
fmt.Printf("Image %v sucessfully created in GCP\n", imageNameGCP)
default:
fmt.Println("Skipped GCE image creation")
fmt.Println("Skipped GCP image creation")
}
}
if err != nil {
fmt.Fprintf(os.Stderr, "Creating GCE image failed: %v\n", err)
fmt.Fprintf(os.Stderr, "Creating GCP image failed: %v\n", err)
os.Exit(1)
}

// If requested, set the image ACL to public
if uploadPublic {
fmt.Printf("Setting image to have public access: %v\n", imageNameGCE)
err = api.SetImagePublic(imageNameGCE)
fmt.Printf("Setting image to have public access: %v\n", imageNameGCP)
err = api.SetImagePublic(imageNameGCP)
if err != nil {
fmt.Fprintf(os.Stderr, "Marking GCE image with public ACLs failed: %v\n", err)
fmt.Fprintf(os.Stderr, "Marking GCP image with public ACLs failed: %v\n", err)
os.Exit(1)
}
}
Expand All @@ -208,11 +208,11 @@ func runUpload(cmd *cobra.Command, args []string) {

}

// Converts an image name from Google Storage to an equivalent GCE image
// name. NOTE: Not a fully generlized sanitizer for GCE. Designed for
// Converts an image name from Google Storage to an equivalent GCP image
// name. NOTE: Not a fully generlized sanitizer for GCP. Designed for
// the default version.txt name (ex: 633.1.0+2015-03-31-1538). See:
// https://godoc.org/google.golang.org/api/compute/v1#Image
func gceSanitize(name string) string {
func gcpSanitize(name string) string {
if name == "" {
return name
}
Expand Down
Loading

0 comments on commit 25ff397

Please sign in to comment.