This is a proof of concept of Container Credential Guard plugin that allows launching containers for IWA (Kerberos) on non-domain joined machines.
Docker runtime on Windows allow containers to be launched under gMSA account. You need a special user account that has been granted access to retrieve password for gMSA account (lets call it gmsa.plugin
). A plugin is registered on the host, which provides docker runtime with credentials to gmsa.plugin
. A special json launch file (CredentialsSpec) is used to instruct docker runtime of the domain controller addresses, the gMSA account to assign to the container as the identity, the plugin to be used. When container is lauched, an optional parameter can be passed into docker run
to instruct it to use credential spec json file in order to authenticate under gMSA and set it as container identity.
-
Compile the plugin (Requires windows machine with .NET 6 SDK installed)
dotnet build
-
Register the plugin on the non-domain joined host
- From the resulting DLL directory (
bin\Debug
), run the following
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\regsvcs /fc CcgPlugin.dll
- Add the following registry key (You may need to change permissions first before being able to modify
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CCG
registry path)
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CCG\COMClasses\{DEFFF03C-3245-465F-8391-CC586A2D1F32}]
- From the resulting DLL directory (
-
Perform steps for non domain-joined hosts in this article to setup gMSA account, gMSA plugin account, and create credentials spec.
- You should run
New-CredentialSpec
powershell commandlet on domain joined machine to ensure correct values are generated. This commandlet requires that you have an existing directoryC:\ProgramData\Docker\CredentialSpecs
.
- You should run
-
Copy the resulting
credspec.json
to the non domain joined host'sC:\ProgramData\Docker\CredentialSpecs
directory -
Edit
credspec.json
and MERGE in the following values (under existingActiveDirectoryConfig
section){ "ActiveDirectoryConfig": { "HostAccountConfig": { "PortableCcgVersion": "1", "PluginGUID": "{DEFFF03C-3245-465F-8391-CC586A2D1F32}", "PluginInput": "gmsa.plugin@mydomain.com:P@ssw0rd" } } }
Change
PluginInput
to use account you created in step 3 that is allowed to retrieve gMSA credentials. -
Change COM-component settings to run as NETWORK SERVICE LOCAL SYSTEM. Run this:
$comAdmin = New-Object -comobject COMAdmin.COMAdminCatalog $apps = $comAdmin.GetCollection("Applications") $apps.Populate() $app = $apps | Where-Object {$_.Name -eq "CcgPlugin"} $app.Value("Identity") = "NT AUTHORITY\NetworkService" $apps.SaveChanges()
-
Ensure that the domain controller is reachable on the following ports
Protocol and port Purpose TCP and UDP 53 DNS TCP and UDP 88 Kerberos TCP 139 NetLogon TCP and UDP 389 LDAP TCP 636 LDAP SSL -
Launch container by passing in name of credential spec json as parameter like this (only filename - not path)
docker run --security-opt "credentialspec=file://credspec.json" -it mcr.microsoft.com/windows/servercore:ltsc2019 powershell
-
Verify that Kerberos ticket can be obtained by invoking.
klist get <MY_GMSA_ACCOUNT>
Cannot get CCM to activate COM plugin component
-
Implemented COM interface as C# project
-
Compile in x64 mode (Debug config)
-
Register to COM from
bin\Debug
folder viaC:\Windows\Microsoft.NET\Framework64\v4.0.30319\RegAsm.exe CcgPlugin.dll /tlb /codebase
-
Add COM registration to CCG
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\CCG\COMClasses\{DEFFF03C-3245-465F-8391-CC586A2D1F32}]
-
Place
ccg.json
intoC:\ProgramData\Docker\credentialspecs
-
Try to start container with
docker run --security-opt "credentialspec=file://ccg.json" --hostname webapp01 -it mcr.microsoft.com/windows/servercore:ltsc2019 powershell
- Some kinda logs created in
c:\temp
(first on component activation, and method invocation)
- No logs in
c:\temp
COM registration seems to be successful, as it can be activated like this:
var otherType = Type.GetTypeFromCLSID(new Guid("DEFFF03C-3245-465F-8391-CC586A2D1F32"));
var otherInstance = Activator.CreateInstance(otherType).Dump();
object[] args = new object[] { "test","","","" };
ParameterModifier pMod = new ParameterModifier(4);
pMod[1] = true;
pMod[2] = true;
pMod[3] = true;
ParameterModifier[] mods = { pMod };
otherType.InvokeMember("GetPasswordCredentials", BindingFlags.InvokeMethod | BindingFlags.NonPublic | BindingFlags.Public , null, otherInstance, args, mods, null, null);
-
Windows SDK installed to get original
IDL
forccgplugins
. Necessary files are inC:\Program Files (x86)\Windows Kits\10\Include\10.0.20348.0\um
-
IDL converted to
TLB
as following:midl ccgplugins-lib.idl /tlb ccgplugins.tlb
-
TLB
converted to.NET
assembly DLL viatlbimp ccgplugins.tlb /out:ccgplugins-net.dll
tlbimp ccgplugins.tlb /out:ccgplugins-net.dll
-
Reverse decompilation of the resulting DLL via Jetbrains DotPeek confirms that the correct signature for the interface as:
namespace ccgplugins\u002Dnet { [Guid("6ECDA518-2010-4437-8BC3-46E752B7B172")] [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] [ComImport] public interface ICcgDomainAuthCredentials { [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)] void GetPasswordCredentials( [MarshalAs(UnmanagedType.LPWStr), In] string pluginInput, [MarshalAs(UnmanagedType.LPWStr)] out string domainName, [MarshalAs(UnmanagedType.LPWStr)] out string username, [MarshalAs(UnmanagedType.LPWStr)] out string password); } }
It was confirmed that AKS plugin that implements the above interface does work and is able to be spun up and start container with GMSA creds as per this article. https://docs.microsoft.com/en-us/azure/aks/use-group-managed-service-accounts
Given that COM plugin has constructor logic that writes a marker file to c:\temp
, the lack of said file when attempting to start container with --security-opt
would imply that COM method signature conforming to an interface is not a problem, as COM object would be instantiated before the method call. Since it's not being created, seems like CCG subsystem is not picking up the plugin registration properly.
The experiment was repeated by copying the DLL to an AKS node where Azure's Vault CCG plugin is installed, registering it and adding the necessary CCG registry entries. No COM activation was observed in that environment either.
PS C:\projects> [Environment]::OSVersion
Platform ServicePack Version VersionString
-------- ----------- ------- -------------
Win32NT 10.0.19043.0 Microsoft Windows NT 10.0.19043.0
PS C:\projects> docker version
Client:
Cloud integration: v1.0.20
Version: 20.10.10
API version: 1.41
Go version: go1.16.9
Git commit: b485636
Built: Mon Oct 25 07:47:53 2021
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.10
API version: 1.41 (minimum version 1.24)
Go version: go1.16.9
Git commit: e2f740d
Built: Mon Oct 25 07:43:13 2021
OS/Arch: windows/amd64
Experimental: false
PS C:\projects>