sql/pgwire: TestAuthenticationAndHBARules failed

[cockroach-teamcity]

sql/pgwire.TestAuthenticationAndHBARules failed with artifacts on release-23.1 @ e47b24c6c2c31ff31cab7a68533ccb25fb430718:

=== CONT  TestAuthenticationAndHBARules/insecure=false/scram
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/9df7e2b43c6703c48b37b40add1a12cd/logTestAuthenticationAndHBARules_insecure=false_scram3454717974
        --- FAIL: TestAuthenticationAndHBARules/insecure=false/scram (4.09s)
=== RUN   TestAuthenticationAndHBARules/insecure=false/scram/autodetect
    datadriven.go:242: 
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/13039/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/pgwire/pgwire_test_/pgwire_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/pgwire/testdata/auth/scram:248:
        set_hba [0 args]
        host all foo all cert-password
        host all abc all cert-password
        ----
        # Active authentication configuration on this node:
        # Original configuration:
        # loopback all all all trust       # built-in CockroachDB default
        # host  all root all cert-password # CockroachDB mandatory rule
        # host all foo all cert-password
        # host all abc all cert-password
        #
        # Interpreted configuration:
        # TYPE   DATABASE USER ADDRESS METHOD        OPTIONS
        loopback all      all  all     trust
        host     all      root all     cert-password
        host     all      foo  all     cert-password
        host     all      abc  all     cert-password
    datadriven.go:242: 
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/13039/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/pgwire/pgwire_test_/pgwire_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/pgwire/testdata/auth/scram:267:
        connect [2 args]
        <no input to command>
        ----
        ok defaultdb
    datadriven.go:242: 
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/13039/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/pgwire/pgwire_test_/pgwire_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/pgwire/testdata/auth/scram:272:
         .*client_connection_end
        output didn't match expected:
        @@ -1,8 +1,8 @@
        -64 {"EventType":"client_connection_start","InstanceID":1,"Network":"tcp","RemoteAddress":"XXX","Timestamp":"XXX"}
        -65 {"EventType":"client_authentication_info","Info":"HBA rule: host all foo all cert-password","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl"}
        -66 {"EventType":"client_authentication_info","Info":"client did not present TLS certificate","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl"}
        -67 {"EventType":"client_authentication_info","Info":"found stored crdb-bcrypt credentials; requesting cleartext password","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl","User":"foo"}
        +64 {"EventType":"client_authentication_info","Info":"client did not present TLS certificate","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl"}
        +65 {"EventType":"client_authentication_info","Info":"found stored crdb-bcrypt credentials; requesting cleartext password","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl","User":"foo"}
        +66 {"Duration":"NNN","EventType":"client_session_end","InstanceID":1,"Network":"tcp","RemoteAddress":"XXX","Timestamp":"XXX"}
        +67 {"Duration":"NNN","EventType":"client_connection_end","InstanceID":1,"Network":"tcp","RemoteAddress":"XXX","Timestamp":"XXX"}
         68 {"EventType":"client_authentication_ok","InstanceID":1,"Method":"cert-password","Network":"tcp","RemoteAddress":"XXX","SystemIdentity":"foo","Timestamp":"XXX","Transport":"hostssl","User":"foo"}
         69 {"Duration":"NNN","EventType":"client_session_end","InstanceID":1,"Network":"tcp","RemoteAddress":"XXX","Timestamp":"XXX"}
         70 {"Duration":"NNN","EventType":"client_connection_end","InstanceID":1,"Network":"tcp","RemoteAddress":"XXX","Timestamp":"XXX"}
         
            --- FAIL: TestAuthenticationAndHBARules/insecure=false/scram/autodetect (0.16s)
=== RUN   TestAuthenticationAndHBARules/insecure=false
    --- FAIL: TestAuthenticationAndHBARules/insecure=false (49.35s)

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• sql/pgwire: TestAuthenticationAndHBARules failed #100353 sql/pgwire: TestAuthenticationAndHBARules failed [C-test-failure O-robot branch-master]

/cc @cockroachdb/sql-foundations @cockroachdb/server

This test on roachdash | Improve this report!

Jira issue: CRDB-28495

[rafiss]

The test resulted in client_connection_end and client_session_end logs, which weren't expected. @andyyang890 can you try to reproduce this?