Waggle-dance does not support DIGEST protocol

[zzzzming95]

Hey, I have a problem. Our environment is based on kerberos. I backport this patchhttps://github.com/ExpediaGroup/waggle-dance/commit/a4ea6950eb755ca279e405b3730a6f9a0a28f1dc to 3.10.10 .

Now we can using waggle-dance via kerberos protocol like hive-client or spark-client.

However, when I want to use hiveserver2 or trino based on the delegation-token mechanism, the connection fails.

The authentication mechanism of hiveserver2 is to use the kerberos protocol connection for the first time, obtain the delegation-token, and then add the token to the UGI, ugi. addToken (delegationToken); Later, the connection will be made through the 'DIGEST' protocol.

The following is the code for the client to create different protocols.

        public TTransport createClientTransport(String principalConfig, String host, String methodStr, String tokenStrForm, final TTransport underlyingTransport, final Map<String, String> saslProps) throws IOException {
            final AuthMethod method = (AuthMethod)AuthMethod.valueOf(AuthMethod.class, methodStr);
            TTransport saslTransport = null;
            switch(method) {
            case DIGEST:
                Token<DelegationTokenIdentifier> t = new Token();
                t.decodeFromUrlString(tokenStrForm);
                saslTransport = new TSaslClientTransport(method.getMechanismName(), (String)null, (String)null, "default", saslProps, new HadoopThriftAuthBridge.Client.SaslClientCallbackHandler(t), underlyingTransport);
                return new TUGIAssumingTransport(saslTransport, UserGroupInformation.getCurrentUser());
            case KERBEROS:
                String serverPrincipal = SecurityUtil.getServerPrincipal(principalConfig, host);

In the server log of waggle-dance, there will be the following error.

java.lang.NullPointerException: null
    at org.apache.hadoop.security.SaslRpcServer.getIdentifier(SaslRpcServer.java:198) ~[hadoop-common-2.10.1.jar!/:?]
    at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:573) ~[hive-shims-common-2.3.7.jar!/:2.3.7]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) [libthrift-0.9.3.jar!/:0.9.3]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

Are there any suggestions for solution? Thanks very much~

[zzzzming95]

@javsanbel2

Do you have any suggestions? thanks~

[zzzzming95]

The problem has been solved. We add new DelegationTokenStore , and then make the delegation-token request route to primay MS.