- release 2.0

- minor edits in docs - more info on TLS configuration - allow -tls_opts to override defaults
Kazmirchuk · Mar 5, 2023 · d1d8839 · d1d8839
1 parent 6235e97
commit d1d8839
Show file tree

Hide file tree

Showing 7 changed files with 77 additions and 55 deletions.
diff --git a/CoreAPI.md b/CoreAPI.md
diff --git a/JsAPI.md b/JsAPI.md
@@ -1,6 +1,6 @@
 # JetStream API
 
-JetStream functionality of NATS can be accessed by creating the `nats::jet_stream` TclOO object. Do not create it directly - instead, call the [jet_stream](CoreAPI.md#objectname-jet_stream--timeout-ms--domain-domain) method of your `nats::connection`. You can have multiple jet_stream objects created from the same connection, each having its own timeout and domain.
+JetStream functionality of NATS can be accessed by creating the `nats::jet_stream` TclOO object. Do not create it directly - instead, call the [jet_stream](CoreAPI.md#objectname-jet_stream--timeout-ms--domain-domain) method of your `nats::connection`. You can have multiple JS objects created from the same connection, each having its own timeout and domain.
 
 ## Synopsis
 
@@ -64,7 +64,7 @@ Note that API of official NATS clients (`JetStreamContext`) is designed in a way
 
 Also, official NATS clients often provide an auto-acknowledgment option (and sometimes even default to it!) - I find it potentially harmful, so it's missing from this client. Always remember to acknoledge JetStream messages according to your policy.
 
-The Tcl client does not provide a dedicated method to subscribe to push consumers. The core NATS subscription is perfectly adequate for the task. If your push consumer is configured with idle heartbeats, you will need to filter them out based on the status header = 100. You can find an example of such subscription in [js_msg.tcl](examples/js_msg.tcl)
+The Tcl client does not provide a dedicated method to subscribe to push consumers. The core NATS subscription is perfectly adequate for the task. If your push consumer is configured with idle heartbeats, you will need to filter them out based on the status header = 100. You can find an example of such subscription in [js_msg.tcl](examples/js_msg.tcl).
 
 ## JetStream wire format
 The JetStream wire format uses nanoseconds for timestamps and durations in all requests and replies. To be consistent with the rest of the Tcl API, the client converts them to milliseconds before returning to a user. And vice versa: all function arguments are accepted as ms and converted to ns before sending.
@@ -118,7 +118,7 @@ Note that you can publish messages to a stream using [nats::connection publish](
 ### js publish_msg *message ?args?*
 Publishes `message` (created with [nats::msg create](CoreAPI.md#msg-create-subject--data-payload--reply-replysubj)) to a stream. Other options are the same as above. Use this method to publish a message with headers.
 ### js consume *stream consumer ?args?*
-Consumes a number of messages from a [pull consumer](https://docs.nats.io/jetstream/concepts/consumers) defined on a [stream](https://docs.nats.io/jetstream/concepts/streams). This is the analogue of PullSubscribe + [fetch](https://pkg.go.dev/github.com/nats-io/nats.go#Subscription.Fetch) in official NATS clients.
+Consumes a number of messages from a [pull consumer](https://docs.nats.io/jetstream/concepts/consumers) defined on a [stream](https://docs.nats.io/jetstream/concepts/streams). This is the analogue of `PullSubscribe` + `fetch` in official NATS clients.
 
 You can provide the following options:
 - `-batch_size int` - number of messages to consume. Default batch is 1.
@@ -133,18 +133,19 @@ If `-timeout` is omitted, the client sends a `no_wait` request, asking NATS to d
 
 If `-timeout` is given, it defines both the client-side and server-side timeouts for the pull request:
 - the client-side timeout is the timeout for the underlying `request`
-- the server-side timeout is 10ms shorter than `timeout`, and it is sent in the `expires` JSON field. This behaviour is consistent with nats.go
+- the server-side timeout is 10ms shorter than `timeout`, and it is sent in the `expires` JSON field. This behaviour is consistent with `nats.go`.
 
 *Note:* you can specify the `-expires` option explicitly (ms), but this is an advanced use case and normally should not be needed.
 
 If a callback is not given, the request is synchronous and blocks in a (coroutine-aware) `vwait` until all expected messages are received or the pull request expires. If the client-side timeout fires before the server-side timeout, and no messages have been received, the method raises `ErrTimeout`. In all other cases the method returns a list with as many messages as currently avaiable, but not more than `batch_size`.
 
-If a callback is given, the call returns immediately. Return value is a unique ID that can be used to cancel the pull request. When a message is pulled or a timeout fires, the callback will be invoked from the event loop. It must have the following signature:<br/>
-**asyncRequestCallback** *timedOut message*
+If a callback is given, the call returns immediately. Return value is a unique ID that can be used to cancel the pull request. When a message is pulled or a timeout fires, the callback will be invoked from the event loop. It must have the following signature:
+
+**cmdPrefix** *timedOut message*
 
 If less than `batch_size` messages are pulled before the pull request times out, the callback is invoked one last time with `timedOut=1`.
 
-The client handles status messages 404, 408 and 409 transparently. You can see them in the debug log, if needed. Also, they are passed to `asyncRequestCallback`, in case you need to distinguish between a client-side and server-side timeout.
+The client handles status messages 404, 408 and 409 transparently. You can see them in the debug log, if needed. Also, they are passed to the callback, in case you need to distinguish between a client-side and server-side timeout.
 
 Depending on the consumer's [AckPolicy](https://docs.nats.io/nats-concepts/jetstream/consumers#ackpolicy), you might need to acknowledge the received messages with one of the methods below. [This](https://docs.nats.io/using-nats/developer/develop_jetstream/consumers#delivery-reliability) official doc explains all different kinds of ACKs.
 
@@ -167,7 +168,7 @@ Create or update a `stream` with configuration specified as option-value pairs.
 | Option        | Type   | Default |
 | ------------- |--------|---------|
 | -description  | string |         |
-| -subjects     | list of strings  | required|
+| -subjects     | list of strings  | (required)|
 | -retention    | one of: limits, interest,<br/> workqueue |limits |
 | -max_consumers  | int |         |
 | -max_msgs  | int |         |
@@ -206,14 +207,14 @@ Create or update a pull or push consumer defined on `stream`. See the [official
 | -name | string | |
 | -durable_name | string | |
 | -description | string | |
-| -deliver_policy | one of: all last new by_start_sequence<br/> by_start_time last_per_subject | all|
+| -deliver_policy | one of: all, last, new, by_start_sequence<br/> by_start_time last_per_subject | all|
 | -opt_start_seq | int | |
 | -opt_start_time | string | |
-| -ack_policy |one of: none all explicit | explicit |
+| -ack_policy |one of: none, all, explicit, | explicit |
 | -ack_wait | ms | |
 | -max_deliver | int | |
 | -filter_subject | string | |
-| -replay_policy | one of: instant original | instant|
+| -replay_policy | one of: instant, original | instant|
 | -rate_limit_bps | int | |
 | -sample_freq | string | |
 | -max_waiting | int | |

diff --git a/README.md b/README.md
@@ -16,15 +16,15 @@ The package is written in pure Tcl, without any C code, and so will work anywher
 
 ## Installing
 The package is available in two forms:
-- as a classic Tcl package using pkgIndex.tcl. Download/clone the repository to one of places listed in your `$auto_path`. Or you can extend `$auto_path` with a new folder, e.g.:
+1. As a classic Tcl package using `pkgIndex.tcl`. Download/clone the repository to one of places listed in your `$auto_path`. Or you can extend `$auto_path` with a new folder, e.g.:
 ```Tcl
 lappend auto_path <path>
 ```
 or using an environment variable:
 ```bash
 export TCLLIBPATH=<path>
 ```
-- as a Tcl module, where all implementation is put into a single *.tm file. This improves package loading time. Note that Tcl modules are loaded from different locations than `$auto_path`. You can check them with the following command:
+2. As a Tcl module, where all implementation is put into a single *.tm file. This improves package loading time. Note that Tcl modules are loaded from different locations than `$auto_path`. You can check them with the following command:
 ```Tcl
 tcl::tm::path list
 ```
@@ -39,12 +39,12 @@ package require nats
 ```
 If you are using a "batteries-included" Tcl distribution, like [Magicsplat](https://www.magicsplat.com/tcl-installer/index.html) or [AndroWish](https://www.androwish.org/home/wiki?name=Batteries+Included), you might already have the package.
 ## Supported features
-- Publish and subscribe to messages, also with headers (NATS version 2.2+)
+- Publish and receive messages, also with headers (NATS version 2.2+)
 - Synchronous and asynchronous requests (optimized: under the hood a single wildcard subscription is used for all requests)
 - Queue groups
 - Gather multiple responses to a request
 - Publishing and consuming messages from JetStream, providing "at least once" or "exactly once" delivery guarantees
-- JetStream management of streams and consumers
+- Management of JetStream streams and consumers
 - Standard `configure` method with many options
 - Protected connections using TLS
 - Automatic reconnection in case of network or server failure
@@ -59,5 +59,5 @@ If you are using a "batteries-included" Tcl distribution, like [Magicsplat](http
 Look into the [examples](examples) folder.
 
 ## Missing features (in comparison to official NATS clients)
-- The new authentication mechanism using NKey & [JWT](https://docs.nats.io/developing-with-nats/security/creds). This one will be difficult to do, because it requires support for _ed25519_ cryptography that is missing in Tcl AFAIK. Please let me know if you need it.
+- The new authentication mechanism using NKey & JWT.
 - WebSocket is not supported. The only available transport is TCP.
diff --git a/examples/hello_nats.tcl b/examples/hello_nats.tcl
@@ -12,8 +12,10 @@ package require nats
 set conn [nats::connection new "MyNats"]
 
 # as a minimum configuration, you need to specify the URL of your NATS server. Here we assume it runs on the same host and on the default port TCP/4222
-# if NATS is configured with TLS, use tls:// instead of nats://
-# if you need to authenticate with NATS, you can provide a username and password in the URL or using 'configure'
+# if NATS is configured with TLS, you can use either `tls://` or `nats://`.
+# In the former case, the TLS connection is mandatory, i.e. a client will reject a NATS server that doesn't have TLS configured.
+
+# If you need to authenticate with NATS, you can provide a username and password in the URL or using 'configure'
 # in the latter case, the credentials will apply to all servers in the pool that don't have them in URL
 $conn configure -servers nats://localhost:4222
 

diff --git a/nats_client.tcl b/nats_client.tcl
@@ -35,7 +35,7 @@ set ::nats::_option_spec {
     randomize bool true
     connect_timeout timeout 2000
     reconnect_time_wait timeout 2000
-    max_reconnect_attempts pos_int 60
+    max_reconnect_attempts int 60
     ping_interval timeout 120000
     max_outstanding_pings pos_int 2
     echo bool true
@@ -190,6 +190,7 @@ oo::class create ::nats::connection {
     }
 
     method current_server {} {
+        my CheckConnection
         return [lrange [$serverPool current_server] 0 1]  ;# drop the last element - schema (nats/tls)
     }
 
@@ -734,9 +735,9 @@ oo::class create ::nats::connection {
                 }
                 chan event $sock writable [list $coro connected]
                 return
-            } on error {msg opt} {
+            } on error err {
                 $serverPool current_server_connected false
-                my AsyncError ErrConnectionRefused "Failed to connect to $host:$port: $msg"
+                my AsyncError ErrConnectionRefused "Failed to connect to $host:$port: $err"
             }
         }
     }
@@ -750,7 +751,7 @@ oo::class create ::nats::connection {
                                     tls_required $tls_done \
                                     name [json::write::string $config(name)] \
                                     lang [json::write::string Tcl] \
-                                    version [json::write::string 1.0] \
+                                    version [json::write::string 2.0] \
                                     protocol 1 \
                                     echo $config(echo)]
 
@@ -833,11 +834,11 @@ oo::class create ::nats::connection {
             # there's no need to check for tls_verify in INFO
             # a user needs to provide -certfile and -keyfile options to tls::import anyway
             # and if they are not needed, NATS server will ignore them
-
             # for simplicity, let's switch to the blocking mode just for the handshake
             chan configure $sock -blocking 1
             try {
-                tls::import $sock -require 1 -command ::nats::tls_callback {*}$config(tls_opts)
+                log::debug "Performing TLS handshake..."
+                tls::import $sock {*}[dict merge {-require 1 -command ::nats::tls_callback} $config(tls_opts)]
                 tls::handshake $sock
                 set tls_done true
             } on error err {
@@ -1215,7 +1216,6 @@ oo::class create ::nats::connection {
 
 # by default, when a handshake fails, the TLS library reports it to stderr AND raises an error - see tls.c, function Tls_Error
 # so I end up with the same message logged twice. Let's suppress stderr altogether
-# keep this proc "public" in case a user needs to override it
 proc ::nats::tls_callback {args} { }
 
 namespace eval ::nats::msg {

diff --git a/pkgIndex.tcl b/pkgIndex.tcl
@@ -1,7 +1,7 @@
-package ifneeded nats 1.0 \
+package ifneeded nats 2.0 \
 [list apply {{dir} {
     source [file join $dir server_pool.tcl]
     source [file join $dir nats_client.tcl]
     source [file join $dir jet_stream.tcl]
-    package provide nats 1.0
+    package provide nats 2.0
 }} $dir]
diff --git a/tests/tls.test b/tests/tls.test
@@ -17,6 +17,9 @@ set tls_failed [catch {package require tls}]
 set certs_present [file isdirectory cert]
 tcltest::testConstraint tls [expr {!$tls_failed && $certs_present}]
 
+# due to a bug in TclTLS, SAN in the server's certificate is not checked https://core.tcl-lang.org/tcltls/tktview/3c42b2ba11
+# it also doesn't pass checks at badssl.com : https://core.tcl-lang.org/tcltls/tktview/9773973cfc
+
 # my certificates are self-signed and not known by the system, so Tcl should reject them
 test tls-1.1 "Try connecting to NATS without CA" -constraints tls -setup {
     cd cert
@@ -28,7 +31,7 @@ test tls-1.1 "Try connecting to NATS without CA" -constraints tls -setup {
 } -result {*certificate verify failed*} -match glob -errorCode {NATS ErrTLS}
 
 test tls-1.2 "Connect to NATS using CA" -constraints tls -body {
-    $conn configure -tls_opts [list -cafile rootCA.pem]
+    $conn configure -tls_opts {-cafile rootCA.pem}
     $conn connect
     assert { [dict get [$conn server_info] tls_required]} 1
     set ${conn}::status
@@ -41,7 +44,7 @@ test tls-1.2 "Connect to NATS using CA" -constraints tls -body {
 test tls-2.1 "Connect to NATS with client verification" -constraints tls -setup {
     startNats NATS --tlsverify --tlscert=server-cert.pem --tlskey=server-key.pem --tlscacert=rootCA.pem
 } -body {
-    $conn configure -tls_opts [list -cafile rootCA.pem -certfile client-cert.pem -keyfile client-key.pem]
+    $conn configure -tls_opts {-cafile rootCA.pem -certfile client-cert.pem -keyfile client-key.pem}
     $conn connect
     assert {[dict get [$conn server_info] tls_verify]} 1
     set ${conn}::status
@@ -51,7 +54,7 @@ test tls-2.1 "Connect to NATS with client verification" -constraints tls -setup
 }
 
 test tls-2.2 "Connect to NATS with client verification - failed" -constraints tls -body {
-    $conn configure -tls_opts [list -cafile rootCA.pem]
+    $conn configure -tls_opts {-cafile rootCA.pem}
     catch {$conn connect} err errOpts
     # depending on Windows/Linux and Tcl/TLS patch version I get either ErrTLS or ErrBrokenSocket
     set errCode [lindex [dict get $errOpts -errorcode] 1]
@@ -67,15 +70,10 @@ test tls-3.1 "Connect to NATS without TLS" -constraints tls -setup {
 } -body {
     # explicit TLS scheme in the URL means that client will require TLS from NATS
     # or you can specify secure=true in "configure"
-    $conn configure -servers tls://localhost:4222 -tls_opts [list -cafile rootCA.pem]
+    $conn configure -servers tls://localhost:4222 -tls_opts {-cafile rootCA.pem}
     $conn connect
 } -result {Server localhost:4222 does not provide TLS} -errorCode {NATS ErrSecureConnWanted} -cleanup {
     stopNats NATS
-}
-
-# This should be the last test case in the file!
-test tls-99 "Finalize the tests" -constraints tls -body {
-    # because of the tls constraint, this cleanup can't be done in the global scope
     cd ..
     $conn destroy
 }