Performance Testing

This summarizes the very first performance testing done with MSAL Go. Below is a description of test environments and findings.

Test set up

Method used for testing is AcquireTokenForClient() and AcquireTokenSilent(). An external storage was used which was partitioned based on the suggested Cache key given out by MSAL Go. The external storage used is this https://github.com/patrickmn/go-cache This scenario is relevant for multi tenant applications. The cache key is of the format clientid_tenant_id_AppTokenCache

The input scenario is there are n number of tenants and t is the total number of tokens. So, as an example, 8 tenants will create 8 partitions and t number of tokens are distributed evenly in these partitions. It was tested on a SurfaceBook2(Intel(R)Core(TM) i7-8650U CPU @ 1.90GHz, 16GB RAM, 4 cores) and a Windows 10 Enterprise

Results

The results below suggest that using the recommended cache partitioning key, it shows constant time lookup regardless of how many items are in the cache. Ideally the number of tokens per cache partition would be less than 10 tokens but the results are comparable even when the number of tokens per cache partition goes upto a 100.

Action	Function	# of Tokens	# of Tenants	Tokens per cache partition	Avg Time	P50	P95
500
Populate Cache	AcqTokenClient	500	10	50	1.091997ms	999.9µs	2.9999ms
Retrieve from Cache	AcqTokenSilent	500	10	50	1.687641ms	1.9912ms	2.9994ms
1,000
Populate Cache	AcqTokenClient	1000	10	100	1.705192ms	1.9984ms	3.0458ms
Retrieve from Cache	AcqTokenSilent	1000	10	100	3.883097ms	3.9985ms	5.0484ms
5,000
Populate Cache	AcqTokenClient	5000	100	50	948.985µs	999.8µs	2.0287ms
Retrieve from Cache	AcqTokenSilent	5000	100	50	1.994151ms	1.9999ms	3.0019ms
50,000
Populate Cache	AcqTokenClient	50,000	1000	50	896.284µs	999.8µs	2.0011ms
Retrieve from Cache	AcqTokenSilent	50,000	1000	50	2.209502ms	2.0001ms	3.0044ms
200,000
Populate Cache	AcqTokenClient	200,000	10,000	20	716.974µs	996.8µs	2.0011ms
Retrieve from Cache	AcqTokenSilent	200,000	10,000	20	1.304306ms	1.0014ms	2.6376ms
100,000
Populate Cache	AcqTokenClient	100,000	1000	100	4.069824ms	3.9866ms	8.9979ms
Retrieve from Cache	AcqTokenSilent	100,000	1000	100	8.455385ms	8.011ms	13.2243ms

Code snippet on how to use the suggested cache partitioning key

Suggested cache key is returned as part of our cache interface. Apps will implement a cache accessor and pass it to application object. This is the interface that needs to be implemented:

// ExportReplace is used export or replace what is in the cache.
type ExportReplace interface {
	// Replace replaces the cache with what is in external storage.
	// key is the suggested key which can be used for partioning the cache
	Replace(cache Unmarshaler, key string)
	// Export writes the binary representation of the cache (cache.Marshal()) to
	// external storage. This is considered opaque.
	// key is the suggested key which can be used for partioning the cache
	Export(cache Marshaler, key string)
}

Below is an example of how a sample cache accessor using the partitioning key will look like. The external storage used in this example is https://github.com/patrickmn/go-cache

package main

import (
	"fmt"
	"log"

	"github.com/AzureAD/microsoft-authentication-library-for-go/apps/cache"
	inmemory "github.com/patrickmn/go-cache"
)

type TokenCache struct {
	cache *inmemory.Cache
}

func (t *TokenCache) Replace(cache cache.Unmarshaler, key string) {
	data, found := t.cache.Get(key) // This gets the correct partition from the external storage 
	if !found {
		log.Println("Didn't find item in cache")
	}
	buf, ok := data.([]byte)
	if !ok {
		log.Println("Byte conversion didn't work as expected")
	}

	err := cache.Unmarshal(buf)
	if err != nil {
		log.Println(err)
	}
}

func (t *TokenCache) Export(cache cache.Marshaler, key string) {
	data, err := cache.Marshal()
	if err != nil {
		log.Println(err)
	}
	t.cache.Set(key, data, -1) // This sets the cache key back to data received from MSAL
}

This is how you can initialize the Token Cache object

var cacheAccessor = &TokenCache{cache: inmemory.New(5*time.Minute, 10*time.Minute)}

And finally you pass this cacheAccessor to the Application Object

app = confidential.New("client_id", credential, confidential.WithAuthority("authority_url", confidential.WithCacheAccessor(cacheAccessor)) 

Future work

We continue to invest in the performance testing. Future work includes increasing the number of metrics (ex. cache size growth) and test cases, running the performance tests on a regular schedule, creating more reports (ex. via App Insights).