Skip to content

Commit

Permalink
Update Otel.md
Browse files Browse the repository at this point in the history
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
  • Loading branch information
ronensc committed Jun 24, 2024
1 parent 4645cbc commit 6174e97
Showing 1 changed file with 17 additions and 2 deletions.
19 changes: 17 additions & 2 deletions examples/production_monitoring/Otel.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,36 @@
export JAEGER_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' jaeger)
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=grpc://$JAEGER_IP:4317
```
Then set vLLM's service name for OpenTelemetry, enable insecure connections to Jaeger and run vLLM:
Then set vLLM's service name for OpenTelemetry, enable insecure connections to Jaeger and run vLLM with the OpenAI endpoint:
```
export OTEL_SERVICE_NAME="vllm-server"
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
python -m vllm.entrypoints.openai.api_server --model="facebook/opt-125m" --otlp-traces-endpoint="$OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"
```
or run vLLM with the grpc endpoint:
```
export OTEL_SERVICE_NAME="vllm-server"
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
python -m vllm.entrypoints.openai.api_server --model="facebook/opt-125m" --otlp-traces-endpoint="$OTEL_EXPORTER_OTLP_TRACES_ENDPOINT" --grpc-port 50051
```

1. In a new shell, send requests with trace context from a dummy client
1. In a new shell, send requests with trace context from a dummy http client
```
export JAEGER_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' jaeger)
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=grpc://$JAEGER_IP:4317
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
export OTEL_SERVICE_NAME="client-service"
python dummy_client.py
```
or a dummy grpc client:
```
export JAEGER_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' jaeger)
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=grpc://$JAEGER_IP:4317
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
export OTEL_SERVICE_NAME="client-service"
python dummy_client_grpc.py
```


1. Open Jaeger webui: http://localhost:16686/

Expand Down

0 comments on commit 6174e97

Please sign in to comment.