Skip to content

Commit

Permalink
Updated tracing readme's and removed deprecated tracing/zipkin package
Browse files Browse the repository at this point in the history
  • Loading branch information
@basvanbeek
basvanbeek committed May 21, 2016
1 parent 7de0f49 commit d4b020d
Show file tree
Hide file tree
Showing 37 changed files with 127 additions and 17,350 deletions.
54 changes: 49 additions & 5 deletions tracing/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# package tracing

`package tracing` provides [Dapper][]-style request tracing to services.
An implementation exists for [Zipkin][]; [Appdash][] support is planned.

[Dapper]: http://research.google.com/pubs/pub36356.html
[Zipkin]: https://blog.twitter.com/2012/distributed-systems-tracing-with-zipkin
[Appdash]: https://github.com/sourcegraph/appdash

## Rationale

Expand All @@ -14,3 +9,52 @@ applications. It's instrumental in understanding request flows, identifying
hot spots, and diagnosing errors. All microservice infrastructures will
benefit from request tracing; sufficiently large infrastructures will require
it.

## OpenTracing

Go kit builds on top of the [OpenTracing] API and uses the [opentracing-go]
package to provide tracing middlewares for its servers and clients. Currently
`kit/transport/http` and `kit/transport/grpc` transports are supported.

Since [OpenTracing] is an upcoming standard API, Go kit should support a
multitude of tracing backends. If a Tracer implementation in Go for your
back-end exists, it should work out of the box. The following tracing back-ends
are known to work with Go kit through the OpenTracing interface and are
highlighted in the [addsvc] example.


### LightStep

[LightStep] support is available through their standard Go package
[lightstep-tracer-go].

### AppDash

[Appdash] support is available straight from their system repository in the
[appdash/opentracing] directory.

### Zipkin

[Zipkin] support is now available from the [zipkin-go-opentracing] package which
can be found at the [Open Zipkin GitHub] page. This means our old custom
`tracing/zipkin` package is now deprecated. In the `kit/tracing/zipkin`
directory you can still find the `docker-compose` script to bootstrap a Zipkin
development environment and a [README] detailing how to transition from the
old package to the new.

[Dapper]: http://research.google.com/pubs/pub36356.html
[addsvc]:https://github.com/go-kit/kit/tree/master/examples/addsvc
[README]: https://github.com/go-kit/kit/blob/master/tracing/zipkin/README.md

[OpenTracing]: http://opentracing.io
[opentracing-go]: https://github.com/opentracing/opentracing-go

[Zipkin]: http://zipkin.io/
[Open Zipkin GitHub]: https://github.com/openzipkin
[zipkin-go-opentracing]: https://github.com/openzipkin/zipkin-go-opentracing

[Appdash]: https://github.com/sourcegraph/appdash
[appdash/opentracing]: https://github.com/sourcegraph/appdash/tree/master/opentracing

[LightStep]: http://lightstep.com/
[lightstep-tracer-go]: https://github.com/lightstep/lightstep-tracer-go
191 changes: 78 additions & 113 deletions tracing/zipkin/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@

Setting up [Zipkin] is not an easy thing to do. It will also demand quite some
resources. To help you get started with development and testing we've made a
docker-compose file available for running a full Zipkin stack. See the
`kit/tracing/zipkin/_docker` subdirectory.
docker-compose file available for running a full Zipkin stack.

You will need [docker-compose] 1.6.0+ and [docker-engine] 1.10.0+.

Expand All @@ -14,7 +13,7 @@ or Windows you probably need to set the hostname environment variable to the
hostname of the VM running the docker containers.

```sh
cd tracing/zipkin/_docker
cd tracing/zipkin
HOSTNAME=localhost docker-compose -f docker-compose-zipkin.yml up
```

Expand All @@ -36,131 +35,97 @@ The following services have been set-up to run:

## Middleware Usage

Wrap a server- or client-side [endpoint][] so that it emits traces to a Zipkin
collector. Make sure the host given to `MakeNewSpanFunc` resolves to an IP. If
not your span will silently fail!
Follow the [addsvc] example to check out how to wire the Zipkin Middleware. The
changes should be relatively minor.

[endpoint]: http://godoc.org/github.com/go-kit/kit/endpoint#Endpoint
The [zipkin-go-opentracing] package has support for Kafka and Scribe collectors
as well as using Go Kit's [Log] package for logging.

If needing to create child spans in methods or calling another service from your
service method, it is highly recommended to request a context parameter so you
can transfer the needed metadata for traces across service boundaries.
### Span per Node vs. Span per RPC
OpenTracing tends to support the `span per node` model, meaning the server and
client side of an RPC call are dealt with in separate spans. This diverts from
the Zipkin V1 model which is more tailored towards the `span per RPC call` model
in which the server and client side annotations of the same RPC call are part of
the same span. The [zipkin-go-opentracing] implementation allows you to choose
which model you wish to use. Make sure you select the same model consistently
for all your services that are required to talk to each other or you will have
trace propagation issues. If using non OpenTracing / legacy instrumentation use
the `span per RPC call` model.

It is also wise to always return error parameters with your service method
calls, even if your service method implementations will not throw errors
themselves. The error return parameter can be wired to pass the potential
transport errors when consuming your service API in a networked environment.
To adhere to the OpenTracing philosophy the Tracer defaults to `span per node`.
To set the `span per RPC call` mode start your tracer like this:

```go
func main() {
tracer, err = zipkin.NewTracer(
zipkin.NewRecorder(...),
zipkin.ClientServerSameSpan(true),
)
```

[zipkin-go-opentracing]: https://github.com/openzipkin/zipkin-go-opentracing
[addsvc]:https://github.com/go-kit/kit/tree/master/examples/addsvc
[Log]: https://github.com/go-kit/kit/tree/master/log

### Tracing Resources

In our legacy implementation we had the `NewChildSpan` method to allow
annotation of resources such as databases, caches and other services that do not
have server side tracing support. Since OpenTracing has no specific method of
dealing with these items explicitely that is compatible with Zipkin's `SA`
annotation the [zipkin-go-opentracing] has implemented support using the
OpenTracing Tags system. Here is an example of how one would be able to record
a resource span compatible with standard OpenTracing and triggering an `SA`
annotation in [zipkin-go-opentracing]:

```go
// you need to import the ext package for the Tag helper functions
import (
"github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
)

func (svc *Service) GetMeSomeExamples(ctx context.Context, ...) ([]Examples, error) {
// Example of annotating a database query:
var (
// myHost MUST resolve to an IP or your span will not show up in Zipkin.
myHost = "instance01.addsvc.internal.net:8000"
myService = "AddService"
myMethod = "Add"
url = myHost + "/add/"
kafkaHost = []string{"kafka.internal.net:9092"}
serviceName = "MySQL"
serviceHost = "mysql.example.com"
servicePort = uint16(3306)
queryLabel = "GetExamplesByParam"
query = "select * from example where param = 'value'"
)

ctx := context.Background()

// Set Up Zipkin Collector and Span factory
spanFunc := zipkin.MakeNewSpanFunc(myHost, myService, myMethod)
collector, _ := zipkin.NewKafkaCollector(kafkaHost)

// Server-side Wiring
var server endpoint.Endpoint
server = makeEndpoint() // for your service
// wrap endpoint with Zipkin tracing middleware
server = zipkin.AnnotateServer(spanFunc, collector)(server)

http.Handle(
"/add/",
httptransport.NewServer(
ctx,
server,
decodeRequestFunc,
encodeResponseFunc,
httptransport.ServerBefore(
zipkin.ToContext(spanFunc),
),
),
)
...
// retrieve the parent span, if not found create a new trace
parentSpan := opentracing.SpanFromContext(ctx)
if parentSpan == nil {
parentSpan = opentracing.StartSpan(queryLabel)
}

// Client-side
var client endpoint.Endpoint
client = httptransport.NewClient(
"GET",
URL,
encodeRequestFunc,
decodeResponseFunc,
httptransport.ClientBefore(zipkin.ToRequest(spanFunc)),
).Endpoint()
client = zipkin.AnnotateClient(spanFunc, collector)(client)

ctx, cancel := context.WithTimeout(ctx, myTimeout)
defer cancel()

reply, err := client(ctx, param1, param2)
// do something with the response/error
...
}
```
// create a new span to record the resource interaction
span := opentracing.StartChildSpan(parentSpan, queryLabel)

## Annotating Remote Resources
// span.kind "resource" triggers SA annotation
ext.SpanKind.Set(span, "resource")

Next to the above shown examples of wiring server-side and client-side tracing
middlewares, you can also span resources called from your service methods.
// this will label the span's service & hostPort (called Endpoint in Zipkin)
ext.PeerService.Set(span, serviceName)
ext.PeerHostname,Set(span, serviceHost)
ext.PeerPort.Set(span, servicePort)

To do this, the service method needs to include a context parameter. From your
endpoint wrapper you can inject the endpoint context which will hold the parent
span already created by the server-side middleware. If the resource is a remote
database you can use the `zipkin.ServerAddr` spanOption to identify the remote
host:port and the display name of this resource.
// a Tag is the equivalent of a Zipkin Binary Annotation (key:value pair)
span.SetTag("query", query)

```go
type MyService struct {
// add a Zipkin Collector to your service implementation's properties.
Collector zipkin.Collector
}
// a LogEvent is the equivalent of a Zipkin Annotation (timestamped)
span.LogEvent("query:start")

// Example of the endpoint.Endpoint to service method wrapper, injecting the
// context provided by the transport server.
func makeComplexEndpoint() endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(ComplexRequest)
v, err := svc.Complex(ctx, req.A, req.B)
return ComplexResponse{V: v, Err: err}, nil
}
}
// do the actual query...

// Complex is an example method of our service, displaying the tracing of a
// remote database resource.
func (s *MyService) Complex(ctx context.Context, A someType, B otherType) (returnType, error) {
// we've parsed the incoming parameters and now we need to query the database.
// we wish to include this action into our trace.
span, collect := zipkin.NewChildSpan(
ctx,
s.Collector,
"complexQuery",
zipkin.ServerAddr(
"mysql01.internal.net:3306",
"MySQL",
),
)
// you probably want to binary annotate your query
span.AnnotateBinary("query", "SELECT ... FROM ... WHERE ... ORDER BY ..."),
// annotate the start of the query
span.Annotate("complexQuery:start")
// do the query and handle resultset
...
// annotate we are done with the query
span.Annotate("complexQuery:end")
// maybe binary annotate some items returned by the resultset
...
// when done with all annotations, collect the span
collect()
// let's annotate the end...
span.LogEvent("query:end")

// we're done with the span. send it to the SpanRecorder for collection...
span.Finish()

// do other stuff
...
}
```
11 changes: 0 additions & 11 deletions tracing/zipkin/_thrift/compile.sh
View file

This file was deleted.

18 changes: 0 additions & 18 deletions tracing/zipkin/_thrift/gen-go/scribe/constants.go
View file

This file was deleted.

Loading

0 comments on commit d4b020d

Please sign in to comment.