Add SAML support with ADFS as IdP

CHANGELOG.md

@@ -8,8 +8,12 @@
 * Added `vdc.QueryVappVmTemplate` and changed `vapp.AddNewVMWithStorageProfile` to allow creating VM from VM template.
 * Enhanced tests command line with flags that can be used instead of environment variables. [#305](https://github.com/vmware/go-vcloud-director/pull/305)
 * Improve logging security of debug output for API requests and responses [#306](https://github.com/vmware/go-vcloud-director/pull/306)
-
-## 2.7.0 (April 10,2020)
+* Add configuration option `WithSamlAdfs` to `NewVCDClient()` to support SAML authentication using
+  Active Directory Federations Services (ADFS) as IdP using WS-TRUST auth endpoint
+  "/adfs/services/trust/13/usernamemixed"
+  [#304](https://github.com/vmware/go-vcloud-director/pull/304)
+
+## 2.7.0 (April 10, 2020)
 
 * Added methods `OrgVdcNetwork.Update`, `OrgVdcNetwork.UpdateAsync`, and `OrgVdcNetwork.Rename` [#292](https://github.com/vmware/go-vcloud-director/pull/292)
 * Added methods `EdgeGateway.Update` and `EdgeGateway.UpdateAsync` [#292](https://github.com/vmware/go-vcloud-director/pull/292)

Makefile

@@ -51,7 +51,7 @@ testnsxv:
 # any common errors.
 vet:
 	@echo "==> Running Go Vet"
-	@cd govcd && go vet ; if [ $$? -ne 0 ] ; then echo "vet error!" ; exit 1 ; fi && cd -
+	@go vet ./... ; if [ $$? -ne 0 ] ; then echo "vet error!" ; exit 1 ; fi
 
 # static runs the source code static analysis tool `staticcheck`
 static: fmtcheck

README.md

@@ -114,25 +114,35 @@ func main() {
 
 ## Authentication
 
-You can authenticate to the vCD in three ways:
+You can authenticate to the vCD in four ways:
 
 * With a System Administration user and password (`administrator@system`)
 * With an Organization user and password (`tenant-admin@org-name`)
-* With an authorization token
-
-For the first two methods, you use:
 
+For the above two methods, you use: 
 ```go
 	err := vcdClient.Authenticate(User, Password, Org)
 	// or
 	resp, err := vcdClient.GetAuthResponse(User, Password, Org)
 ```
 
-For the token, you use:
-
+* With an authorization token
 ```go
 	err := vcdClient.SetToken(Org, govcd.AuthorizationHeader, Token)
 ```
+The file `scripts/get_token.sh` provides a handy method of extracting the token
+(`x-vcloud-authorization` value) for future use.
+
+* SAML user and password (works with ADFS as IdP using WS-TRUST endpoint
+  "/adfs/services/trust/13/usernamemixed"). One must pass `govcd.WithSamlAdfs(true,customAdfsRptId)`
+  and username must be formatted so that ADFS understands it ('user@contoso.com' or
+  'contoso.com\user') You can find usage example in
+  [samples/saml_auth_adfs](/samples/saml_auth_adfs). 
+```go
+vcdCli := govcd.NewVCDClient(*vcdURL, true, govcd.WithSamlAdfs(true, customAdfsRptId))
+err = vcdCli.Authenticate(username, password, org)
+```
 
-The file `scripts/get_token.sh` provides a handy method of extracting the token (`x-vcloud-authorization` value) for future use.
-
+More information about inner workings of SAML auth flow in this codebase can be found in
+`saml_auth.go:authorizeSamlAdfs(...)`. Additionaly this flow is documented in [vCD
+documentation](https://code.vmware.com/docs/10000/vcloud-api-programming-guide-for-service-providers/GUID-335CFC35-7AD8-40E5-91BE-53971937A2BB.html).

TESTING.md

@@ -335,6 +335,22 @@ While running tests, the following environment variables can be used:
 
 When both the environment variable and the command line option are possible, the environment variable gets evaluated first.
 
+# SAML auth testing with Active Directory Federation Services (ADFS) as Identity Provider (IdP)
+
+This package supports SAML authentication with ADFS. It can be achieved by supplying
+`WithSamlAdfs()` function to `NewVCDClient`. Testing framework also supports SAML auth and there are
+a few ways to test it:
+* There is a unit test `TestSamlAdfsAuthenticate` which spawns mock servers and does not require
+  ADFS or SAML being configured. It tests the flow based on mock endpoints.
+* Using regular `user` and `password` to supply SAML credentials and `true` for `useSamlAdfs`
+  variable (optionally one can override Relaying Party Trust ID with variable `customAdfsRptId`).
+  That way all tests would run using SAML authentication flow.
+* Using `samlUser`, `samlPassword` and optionally `samlCustomRptId` variables will enable
+  `Test_SamlAdfsAuth` test run. Test_SamlAdfsAuth will test and compare VDC retrieved using main
+  authentication credentials vs the one retrieved using specific SAML credentials.
+
+All these tests can run in combination.
+
 # Final Words
 Be careful about using our tests as these tests run on a real vcd. If you don't have 1 gb of ram and 2 vcpus available then you should not be running tests that deploy your vm/change memory and cpu. However everything created will be removed at the end of testing.
 

govcd/api.go

@@ -14,7 +14,6 @@ import (
 	"net/http"
 	"net/url"
 	"os"
-	"reflect"
 	"strings"
 	"time"
 
@@ -36,10 +35,20 @@ type Client struct {
 	// This must be >0 to avoid instant timeout errors.
 	MaxRetryTimeout int
 
+	// UseSamlAdfs specifies if SAML auth is used for authenticating vCD instead of local login.
+	// The following conditions must be met so that authentication SAML authentication works:
+	// * SAML IdP (Identity Provider) is Active Directory Federation Service (ADFS)
+	// * Authentication endpoint "/adfs/services/trust/13/usernamemixed" must be enabled on ADFS
+	// server
+	UseSamlAdfs bool
+	// CustomAdfsRptId allows to set custom Relaying Party Trust identifier. By default vCD Entity
+	// ID is used as Relaying Party Trust identifier.
+	CustomAdfsRptId string
+
 	supportedVersions SupportedVersions // Versions from /api/versions endpoint
 }
 
-// The header key used by default to set the authorization token.
+// AuthorizationHeader header key used by default to set the authorization token.
 const AuthorizationHeader = "X-Vcloud-Authorization"
 
 // General purpose error to be used whenever an entity is not found from a "GET" request
@@ -147,6 +156,12 @@ func (cli *Client) NewRequestWitNotEncodedParams(params map[string]string, notEn
 // * body - request body
 // * apiVersion - provided Api version overrides default Api version value used in request parameter
 func (cli *Client) NewRequestWitNotEncodedParamsWithApiVersion(params map[string]string, notEncodedParams map[string]string, method string, reqUrl url.URL, body io.Reader, apiVersion string) *http.Request {
+	return cli.newRequest(params, notEncodedParams, method, reqUrl, body, apiVersion, nil)
+}
+
+// newRequest is the parent of many "specific" "NewRequest" functions.
+// Note. It is kept private to avoid breaking public API on every new field addition.
+func (cli *Client) newRequest(params map[string]string, notEncodedParams map[string]string, method string, reqUrl url.URL, body io.Reader, apiVersion string, additionalHeader http.Header) *http.Request {
 	reqValues := url.Values{}
 
 	// Build up our request parameters
@@ -163,6 +178,14 @@ func (cli *Client) NewRequestWitNotEncodedParamsWithApiVersion(params map[string
 		}
 	}
 
+	// If the body contains data - try to read all contents for logging and re-create another
+	// io.Reader with all contents to use it down the line
+	var readBody []byte
+	if body != nil {
+		readBody, _ = ioutil.ReadAll(body)
+		body = bytes.NewReader(readBody)
+	}
+
 	// Build the request, no point in checking for errors here as we're just
 	// passing a string version of an url.URL struct and http.NewRequest returns
 	// error only if can't process an url.ParseRequestURI().
@@ -171,39 +194,38 @@ func (cli *Client) NewRequestWitNotEncodedParamsWithApiVersion(params map[string
 	if cli.VCDAuthHeader != "" && cli.VCDToken != "" {
 		// Add the authorization header
 		req.Header.Add(cli.VCDAuthHeader, cli.VCDToken)
+	}
+	if (cli.VCDAuthHeader != "" && cli.VCDToken != "") ||
+		(additionalHeader != nil && additionalHeader.Get("Authorization") != "") {
 		// Add the Accept header for VCD
 		req.Header.Add("Accept", "application/*+xml;version="+apiVersion)
 	}
 
+	// Merge in additional headers before logging if any where specified in additionalHeader
+	// paramter
+	if additionalHeader != nil && len(additionalHeader) > 0 {
+		for headerName, headerValueSlice := range additionalHeader {
+			for _, singleHeaderValue := range headerValueSlice {
+				req.Header.Add(headerName, singleHeaderValue)
+			}
+		}
+	}
+
 	// Avoids passing data if the logging of requests is disabled
 	if util.LogHttpRequest {
-		// Makes a safe copy of the request body, and passes it
-		// to the processing function.
 		payload := ""
 		if req.ContentLength > 0 {
-			// We try to convert body to a *bytes.Buffer
-			var ibody interface{} = body
-			bbody, ok := ibody.(*bytes.Buffer)
-			// If the inner object is a bytes.Buffer, we get a safe copy of the data.
-			// If it is really just an io.Reader, we don't, as the copy would empty the reader
-			if ok {
-				payload = bbody.String()
-			} else {
-				// With this content, we'll know that the payload is not really empty, but
-				// it was unavailable due to the body type.
-				payload = fmt.Sprintf("<Not retrieved from type %s>", reflect.TypeOf(body))
-			}
+			payload = string(readBody)
 		}
 		util.ProcessRequestOutput(util.FuncNameCallStack(), method, reqUrl.String(), payload, req)
-
 		debugShowRequest(req, payload)
 	}
+
 	return req
 
 }
 
-// NewRequest creates a new HTTP request and applies necessary auth headers if
-// set.
+// NewRequest creates a new HTTP request and applies necessary auth headers if set.
 func (cli *Client) NewRequest(params map[string]string, method string, reqUrl url.URL, body io.Reader) *http.Request {
 	return cli.NewRequestWitNotEncodedParams(params, nil, method, reqUrl, body)
 }
@@ -237,9 +259,13 @@ func decodeBody(resp *http.Response, out interface{}) error {
 	}
 
 	debugShowResponse(resp, body)
-	// Unmarshal the XML.
-	if err = xml.Unmarshal(body, &out); err != nil {
-		return err
+
+	// only attempty to unmarshal if body is not empty
+	if len(body) > 0 {
+		// Unmarshal the XML.
+		if err = xml.Unmarshal(body, &out); err != nil {
+			return err
+		}
 	}
 
 	return nil
@@ -266,7 +292,8 @@ func checkRespWithErrType(resp *http.Response, err, errType error) (*http.Respon
 		http.StatusOK,        // 200
 		http.StatusCreated,   // 201
 		http.StatusAccepted,  // 202
-		http.StatusNoContent: // 204
+		http.StatusNoContent, // 204
+		http.StatusFound:     // 302
 		return resp, nil
 	// Invalid request, parse the XML error returned and return it.
 	case

govcd/api_vcd.go

@@ -117,7 +117,7 @@ func NewVCDClient(vcdEndpoint url.URL, insecure bool, options ...VCDClientOption
 	return vcdClient
 }
 
-// Authenticate is an helper function that performs a login in vCloud Director.
+// Authenticate is a helper function that performs a login in vCloud Director.
 func (vcdCli *VCDClient) Authenticate(username, password, org string) error {
 	_, err := vcdCli.GetAuthResponse(username, password, org)
 	return err
@@ -132,11 +132,24 @@ func (vcdCli *VCDClient) GetAuthResponse(username, password, org string) (*http.
 	if err != nil {
 		return nil, fmt.Errorf("error finding LoginUrl: %s", err)
 	}
-	// Authorize
-	resp, err := vcdCli.vcdAuthorize(username, password, org)
-	if err != nil {
-		return nil, fmt.Errorf("error authorizing: %s", err)
+
+	// Choose correct auth mechanism based on what type of authentication is used. The end result
+	// for each of the below functions is to set authorization token vcdCli.Client.VCDToken.
+	var resp *http.Response
+	switch {
+	case vcdCli.Client.UseSamlAdfs:
+		err = vcdCli.authorizeSamlAdfs(username, password, org, vcdCli.Client.CustomAdfsRptId)
+		if err != nil {
+			return nil, fmt.Errorf("error authorizing SAML: %s", err)
+		}
+	default:
+		// Authorize
+		resp, err = vcdCli.vcdAuthorize(username, password, org)
+		if err != nil {
+			return nil, fmt.Errorf("error authorizing: %s", err)
+		}
 	}
+
 	return resp, nil
 }
 
@@ -217,3 +230,18 @@ func WithHttpTimeout(timeout int64) VCDClientOption {
 		return nil
 	}
 }
+
+// WithSamlAdfs specifies if SAML auth is used for authenticating to vCD instead of local login.
+// The following conditions must be met so that SAML authentication works:
+// * SAML IdP (Identity Provider) is Active Directory Federation Service (ADFS)
+// * WS-Trust authentication endpoint "/adfs/services/trust/13/usernamemixed" must be enabled on
+// ADFS server
+// By default vCD SAML Entity ID will be used as Relaying Party Trust Identifier unless
+// customAdfsRptId is specified
+func WithSamlAdfs(useSaml bool, customAdfsRptId string) VCDClientOption {
+	return func(vcdClient *VCDClient) error {
+		vcdClient.Client.UseSamlAdfs = useSaml
+		vcdClient.Client.CustomAdfsRptId = customAdfsRptId
+		return nil
+	}
+}

govcd/api_vcd_test.go

@@ -1,4 +1,4 @@
-// +build api functional catalog vapp gateway network org query extnetwork task vm vdc system disk lb lbAppRule lbAppProfile lbServerPool lbServiceMonitor lbVirtualServer user search nsxv ALL
+// +build api functional catalog vapp gateway network org query extnetwork task vm vdc system disk lb lbAppRule lbAppProfile lbServerPool lbServiceMonitor lbVirtualServer user search nsxv auth ALL
 
 /*
  * Copyright 2019 VMware, Inc.  All rights reserved.  Licensed under the Apache v2 License.
@@ -86,6 +86,17 @@ type TestConfig struct {
 		User            string `yaml:"user"`
 		Password        string `yaml:"password"`
 		Token           string `yaml:"token"`
+		UseSamlAdfs     bool   `yaml:"useSamlAdfs"`
+		CustomAdfsRptId string `yaml:"customAdfsRptId"`
+
+		// The variables `SamlUser`, `SamlPassword` and `SamlCustomRptId` are optional and are
+		// related to an additional test run specifically with SAML user/password. It is useful in
+		// case local user is used for test run (defined by above 'User', 'Password' variables).
+		// SamlUser takes ADFS friendly format ('contoso.com\username' or 'username@contoso.com')
+		SamlUser        string `yaml:"samlUser,omitempty"`
+		SamlPassword    string `yaml:"samlPassword,omitempty"`
+		SamlCustomRptId string `yaml:"samlCustomRptId,omitempty"`
+
 		Url             string `yaml:"url"`
 		SysOrg          string `yaml:"sysOrg"`
 		MaxRetryTimeout int    `yaml:"maxRetryTimeout,omitempty"`
@@ -369,6 +380,10 @@ func GetTestVCDFromYaml(testConfig TestConfig, options ...VCDClientOption) (*VCD
 		options = append(options, WithHttpTimeout(testConfig.Provider.HttpTimeout))
 	}
 
+	if testConfig.Provider.UseSamlAdfs {
+		options = append(options, WithSamlAdfs(true, testConfig.Provider.CustomAdfsRptId))
+	}
+
 	return NewVCDClient(*configUrl, true, options...), nil
 }
 
@@ -439,6 +454,9 @@ func (vcd *TestVCD) SetUpSuite(check *C) {
 	} else {
 		err = vcd.client.Authenticate(config.Provider.User, config.Provider.Password, config.Provider.SysOrg)
 	}
+	if config.Provider.UseSamlAdfs {
+		authenticationMode = "SAML password"
+	}
 	if err != nil {
 		panic(err)
 	}

govcd/api_vcd_versions.go

@@ -8,6 +8,7 @@ import (
 	"fmt"
 	"net/http"
 	"sort"
+	"strings"
 
 	semver "github.com/hashicorp/go-version"
 
@@ -122,6 +123,13 @@ func (cli *Client) vcdFetchSupportedVersions() error {
 
 	cli.supportedVersions = *suppVersions
 
+	// Log all supported API versions in one line to help identify vCD version from logs
+	allApiVersions := make([]string, len(cli.supportedVersions.VersionInfos))
+	for versionIndex, version := range cli.supportedVersions.VersionInfos {
+		allApiVersions[versionIndex] = version.Version
+	}
+	util.Logger.Printf("[DEBUG] supported API versions : %s", strings.Join(allApiVersions, ","))
+
 	return err
 }