Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cart] - fix cartServiceFailure flag #1313

Merged
merged 4 commits into from
Jan 2, 2024
Merged

[cart] - fix cartServiceFailure flag #1313

merged 4 commits into from
Jan 2, 2024

Conversation

puckpuck
Copy link
Contributor

@puckpuck puckpuck commented Dec 22, 2023

Changes

Fixes: #1306

Adds the environment variable to enable the cart to find the feature flag service. It also removes code from the cart service to do a random check 10% of the time. Now that the feature flag service supports this functionality, we should leverage it there instead. Set the feature flag to 0.1 to have it fail 10% of the time.

  • CHANGELOG.md updated to document new feature additions

Signed-off-by: Pierre Tessier <pierre@pierretessier.com>
@puckpuck puckpuck requested a review from a team as a code owner December 22, 2023 20:55
puckpuck and others added 2 commits December 22, 2023 15:56
Signed-off-by: Pierre Tessier <pierre@pierretessier.com>
Copy link
Member

@julianocosta89 julianocosta89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are still missing something here.

I've set the flag cartServiceFailure to 1.0 and I didn't get the errors on the CartService, but in the Frontend Proxy.

Here is the full Trace:

{
    "data": [
        {
            "traceID": "864f33c21e7388fc27615dd9f45cb499",
            "spans": [
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "9a241ef3bbcd3bec",
                    "operationName": "CurrencyService/Convert",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "dc1d62bcbafb1410"
                        }
                    ],
                    "startTime": 1703716193160551,
                    "duration": 117,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "CurrencyService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Convert"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "app.currency.conversion.from",
                            "type": "string",
                            "value": "USD"
                        },
                        {
                            "key": "app.currency.conversion.to",
                            "type": "string",
                            "value": "BRL"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "otel.status_code",
                            "type": "string",
                            "value": "OK"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193160575,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Processing currency conversion request"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193160638,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Conversion successful, response sent back"
                                }
                            ]
                        }
                    ],
                    "processID": "p1",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "a0708f9b87e22858",
                    "operationName": "CurrencyService/Convert",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "1e98c7af67d04cd0"
                        }
                    ],
                    "startTime": 1703716193179759,
                    "duration": 96,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "CurrencyService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Convert"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "app.currency.conversion.from",
                            "type": "string",
                            "value": "USD"
                        },
                        {
                            "key": "app.currency.conversion.to",
                            "type": "string",
                            "value": "BRL"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "otel.status_code",
                            "type": "string",
                            "value": "OK"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193179777,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Processing currency conversion request"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193179831,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Conversion successful, response sent back"
                                }
                            ]
                        }
                    ],
                    "processID": "p1",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "80164af40fe94231",
                    "operationName": "calculate-quote",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "3a37e0ccc3a05556"
                        }
                    ],
                    "startTime": 1703716193174014,
                    "duration": 22,
                    "tags": [
                        {
                            "key": "app.quote.items.count",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "app.quote.cost.total",
                            "type": "float64",
                            "value": 8.9
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "internal"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193174020,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Calculating quote"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193174033,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Quote calculated, returning its value"
                                }
                            ]
                        }
                    ],
                    "processID": "p2",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "3a37e0ccc3a05556",
                    "operationName": "{closure}",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "175eb70e0f707a49"
                        }
                    ],
                    "startTime": 1703716193173908,
                    "duration": 312,
                    "tags": [
                        {
                            "key": "code.function",
                            "type": "string",
                            "value": "__invoke"
                        },
                        {
                            "key": "code.namespace",
                            "type": "string",
                            "value": "DI\\Bridge\\Slim\\ControllerInvoker"
                        },
                        {
                            "key": "code.filepath",
                            "type": "string",
                            "value": "/var/www/vendor/php-di/slim-bridge/src/ControllerInvoker.php"
                        },
                        {
                            "key": "code.lineno",
                            "type": "int64",
                            "value": 31
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "internal"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193173964,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Received get quote request, processing it"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193174067,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Quote processed, response sent back"
                                },
                                {
                                    "key": "app.quote.cost.total",
                                    "type": "float64",
                                    "value": 8.9
                                }
                            ]
                        }
                    ],
                    "processID": "p3",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "175eb70e0f707a49",
                    "operationName": "POST /getquote",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "a6d8a4f7ad43e18e"
                        }
                    ],
                    "startTime": 1703716193173731,
                    "duration": 529,
                    "tags": [
                        {
                            "key": "code.function",
                            "type": "string",
                            "value": "handle"
                        },
                        {
                            "key": "code.namespace",
                            "type": "string",
                            "value": "Slim\\App"
                        },
                        {
                            "key": "code.filepath",
                            "type": "string",
                            "value": "/var/www/vendor/slim/slim/Slim/App.php"
                        },
                        {
                            "key": "code.lineno",
                            "type": "int64",
                            "value": 197
                        },
                        {
                            "key": "url.full",
                            "type": "string",
                            "value": "http://quoteservice:8090/getquote"
                        },
                        {
                            "key": "http.request.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "http.request.body.size",
                            "type": "string",
                            "value": "19"
                        },
                        {
                            "key": "url.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.response.status_code",
                            "type": "int64",
                            "value": 200
                        },
                        {
                            "key": "network.protocol.version",
                            "type": "string",
                            "value": "1.1"
                        },
                        {
                            "key": "http.response.body.size",
                            "type": "string",
                            "value": ""
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p3",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "e738af9aeb1a1541",
                    "operationName": "POST",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "a6d8a4f7ad43e18e"
                        }
                    ],
                    "startTime": 1703716193171951,
                    "duration": 3066,
                    "tags": [
                        {
                            "key": "code.filepath",
                            "type": "string",
                            "value": "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/reqwest-tracing-0.4.6/src/reqwest_otel_span_builder.rs"
                        },
                        {
                            "key": "code.namespace",
                            "type": "string",
                            "value": "reqwest_tracing::reqwest_otel_span_builder"
                        },
                        {
                            "key": "code.lineno",
                            "type": "int64",
                            "value": 138
                        },
                        {
                            "key": "thread.id",
                            "type": "int64",
                            "value": 5
                        },
                        {
                            "key": "thread.name",
                            "type": "string",
                            "value": "tokio-runtime-worker"
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "http.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.host",
                            "type": "string",
                            "value": "quoteservice"
                        },
                        {
                            "key": "net.host.port",
                            "type": "string",
                            "value": "8090"
                        },
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://quoteservice:8090/getquote"
                        },
                        {
                            "key": "http.status_code",
                            "type": "string",
                            "value": "200"
                        },
                        {
                            "key": "http.user_agent",
                            "type": "string",
                            "value": ""
                        },
                        {
                            "key": "busy_ns",
                            "type": "int64",
                            "value": 407973
                        },
                        {
                            "key": "idle_ns",
                            "type": "int64",
                            "value": 2657911
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p4",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "a6d8a4f7ad43e18e",
                    "operationName": "oteldemo.ShippingService/GetQuote",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "4838fc9c31d8855e"
                        }
                    ],
                    "startTime": 1703716193171873,
                    "duration": 3240,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "app.shipping.zip_code",
                            "type": "string",
                            "value": "94043"
                        },
                        {
                            "key": "app.shipping.items.count",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "app.shipping.cost.total",
                            "type": "string",
                            "value": "8.90"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193171875,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Processing get quote request"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193175068,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Received Quote"
                                },
                                {
                                    "key": "app.shipping.cost.total",
                                    "type": "string",
                                    "value": "8.90"
                                }
                            ]
                        }
                    ],
                    "processID": "p5",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "bea79f5888b1fc5b",
                    "operationName": "oteldemo.ShippingService/ShipOrder",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "2cca272cc80024fc"
                        }
                    ],
                    "startTime": 1703716193189350,
                    "duration": 16,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "app.shipping.tracking.id",
                            "type": "string",
                            "value": "bb54080b-3e61-4e12-9adc-ee54b76e2f74"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193189352,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Processing shipping order request"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193189365,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Shipping tracking id created, response sent back"
                                }
                            ]
                        }
                    ],
                    "processID": "p5",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "400deee69063ce52",
                    "operationName": "oteldemo.ProductCatalogService/GetProduct",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "2fbb239dad3111a0"
                        }
                    ],
                    "startTime": 1703716193152772,
                    "duration": 418,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.ProductCatalogService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetProduct"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "app.product.id",
                            "type": "string",
                            "value": "0PUK6V6EV0"
                        },
                        {
                            "key": "app.product.name",
                            "type": "string",
                            "value": "Solar System Color Imager"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193153082,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Product Found - ID: 0PUK6V6EV0, Name: Solar System Color Imager"
                                }
                            ]
                        }
                    ],
                    "processID": "p6",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "a747ea9272807b9d",
                    "operationName": "oteldemo.CartService/GetCart",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "73186e5200643840"
                        }
                    ],
                    "startTime": 1703716193118259,
                    "duration": 23233,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.CartService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetCart"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "2fbb239dad3111a0",
                    "operationName": "oteldemo.ProductCatalogService/GetProduct",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "73186e5200643840"
                        }
                    ],
                    "startTime": 1703716193146153,
                    "duration": 8059,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.ProductCatalogService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetProduct"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "dc1d62bcbafb1410",
                    "operationName": "oteldemo.CurrencyService/Convert",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "73186e5200643840"
                        }
                    ],
                    "startTime": 1703716193155756,
                    "duration": 8061,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.CurrencyService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Convert"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "4838fc9c31d8855e",
                    "operationName": "oteldemo.ShippingService/GetQuote",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "73186e5200643840"
                        }
                    ],
                    "startTime": 1703716193165546,
                    "duration": 10098,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.ShippingService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetQuote"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "1e98c7af67d04cd0",
                    "operationName": "oteldemo.CurrencyService/Convert",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "73186e5200643840"
                        }
                    ],
                    "startTime": 1703716193176315,
                    "duration": 5143,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.CurrencyService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Convert"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "9a9eee311cc0da41",
                    "operationName": "oteldemo.PaymentService/Charge",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "324154dc7b1177c7"
                        }
                    ],
                    "startTime": 1703716193182009,
                    "duration": 4205,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.PaymentService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Charge"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": [
                        "invalid parent span IDs=324154dc7b1177c7; skipping clock skew adjustment"
                    ]
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "2cca272cc80024fc",
                    "operationName": "oteldemo.ShippingService/ShipOrder",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "324154dc7b1177c7"
                        }
                    ],
                    "startTime": 1703716193187514,
                    "duration": 2231,
                    "tags": [
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.ShippingService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "ShipOrder"
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p7",
                    "warnings": [
                        "invalid parent span IDs=324154dc7b1177c7; skipping clock skew adjustment"
                    ]
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "73186e5200643840",
                    "operationName": "prepareOrderItemsAndShippingQuoteFromCart",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "324154dc7b1177c7"
                        }
                    ],
                    "startTime": 1703716193111646,
                    "duration": 70036,
                    "tags": [
                        {
                            "key": "app.shipping.amount",
                            "type": "float64",
                            "value": 33
                        },
                        {
                            "key": "app.cart.items.count",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "app.order.items.count",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "internal"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p8",
                    "warnings": [
                        "invalid parent span IDs=324154dc7b1177c7; skipping clock skew adjustment"
                    ]
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "4005b0fa42b94b40",
                    "operationName": "/oteldemo.FeatureFlagService/GetFlag",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "65b23a65de6ee59e"
                        }
                    ],
                    "startTime": 1703716193194376,
                    "duration": 1053,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "app.featureflag.raw_value",
                            "type": "float64",
                            "value": 1
                        },
                        {
                            "key": "app.featureflag.name",
                            "type": "string",
                            "value": "cartServiceFailure"
                        },
                        {
                            "key": "app.featureflag.enabled",
                            "type": "string",
                            "value": "true"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p9",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "0e7ec7965b8ea40a",
                    "operationName": "featureflagservice.repo.query:featureflags",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "4005b0fa42b94b40"
                        }
                    ],
                    "startTime": 1703716193194620,
                    "duration": 729,
                    "tags": [
                        {
                            "key": "total_time_microseconds",
                            "type": "int64",
                            "value": 401
                        },
                        {
                            "key": "source",
                            "type": "string",
                            "value": "featureflags"
                        },
                        {
                            "key": "queue_time_microseconds",
                            "type": "int64",
                            "value": 64
                        },
                        {
                            "key": "query_time_microseconds",
                            "type": "int64",
                            "value": 332
                        },
                        {
                            "key": "idle_time_microseconds",
                            "type": "int64",
                            "value": 1411668
                        },
                        {
                            "key": "decode_time_microseconds",
                            "type": "int64",
                            "value": 4
                        },
                        {
                            "key": "db.url",
                            "type": "string",
                            "value": "ecto://ffs_postgres"
                        },
                        {
                            "key": "db.type",
                            "type": "string",
                            "value": "sql"
                        },
                        {
                            "key": "db.statement",
                            "type": "string",
                            "value": "SELECT f0.\"id\", f0.\"description\", f0.\"enabled\", f0.\"name\", f0.\"inserted_at\", f0.\"updated_at\" FROM \"featureflags\" AS f0 WHERE (f0.\"name\" = $1)"
                        },
                        {
                            "key": "db.name",
                            "type": "string",
                            "value": "ffs"
                        },
                        {
                            "key": "db.instance",
                            "type": "string",
                            "value": "ffs"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p10",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "369dc2951fd18964",
                    "operationName": "oteldemo.CartService/GetCart",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "a747ea9272807b9d"
                        }
                    ],
                    "startTime": 1703716193138310,
                    "duration": 2263,
                    "tags": [
                        {
                            "key": "net.host.name",
                            "type": "string",
                            "value": "cartservice"
                        },
                        {
                            "key": "net.host.port",
                            "type": "int64",
                            "value": 7070
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "http.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.target",
                            "type": "string",
                            "value": "/oteldemo.CartService/GetCart"
                        },
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://cartservice:7070/oteldemo.CartService/GetCart"
                        },
                        {
                            "key": "http.flavor",
                            "type": "string",
                            "value": "2"
                        },
                        {
                            "key": "http.user_agent",
                            "type": "string",
                            "value": "grpc-go/1.59.0"
                        },
                        {
                            "key": "app.user.id",
                            "type": "string",
                            "value": "1f3552da-ee3d-440b-94a9-d4a11223179c"
                        },
                        {
                            "key": "app.cart.items.count",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "http.route",
                            "type": "string",
                            "value": "/oteldemo.CartService/GetCart"
                        },
                        {
                            "key": "http.status_code",
                            "type": "int64",
                            "value": 200
                        },
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "net.peer.ip",
                            "type": "string",
                            "value": "::ffff:172.18.0.23"
                        },
                        {
                            "key": "net.peer.port",
                            "type": "int64",
                            "value": 46452
                        },
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.CartService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetCart"
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193138547,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Fetch cart"
                                }
                            ]
                        }
                    ],
                    "processID": "p11",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "65b23a65de6ee59e",
                    "operationName": "HTTP POST",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "3f710ce931bd4083"
                        }
                    ],
                    "startTime": 1703716193193024,
                    "duration": 2680,
                    "tags": [
                        {
                            "key": "http.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "net.peer.name",
                            "type": "string",
                            "value": "featureflagservice"
                        },
                        {
                            "key": "net.peer.port",
                            "type": "int64",
                            "value": 50053
                        },
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://featureflagservice:50053/oteldemo.FeatureFlagService/GetFlag"
                        },
                        {
                            "key": "http.flavor",
                            "type": "string",
                            "value": "2.0"
                        },
                        {
                            "key": "http.status_code",
                            "type": "int64",
                            "value": 200
                        },
                        {
                            "key": "peer.service",
                            "type": "string",
                            "value": "featureflagservice:50053"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p12",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "3f710ce931bd4083",
                    "operationName": "oteldemo.FeatureFlagService/GetFlag",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "57a9e494dcd59519"
                        }
                    ],
                    "startTime": 1703716193191994,
                    "duration": 3791,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.FeatureFlagService"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "GetFlag"
                        },
                        {
                            "key": "net.peer.name",
                            "type": "string",
                            "value": "featureflagservice"
                        },
                        {
                            "key": "net.peer.port",
                            "type": "int64",
                            "value": 50053
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "peer.service",
                            "type": "string",
                            "value": "featureflagservice:50053"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p13",
                    "warnings": [
                        "invalid parent span IDs=57a9e494dcd59519; skipping clock skew adjustment"
                    ]
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "7f6eddba6baf00b4",
                    "operationName": "HGET",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "369dc2951fd18964"
                        }
                    ],
                    "startTime": 1703716193138847,
                    "duration": 1547,
                    "tags": [
                        {
                            "key": "db.system",
                            "type": "string",
                            "value": "redis"
                        },
                        {
                            "key": "db.redis.flags",
                            "type": "string",
                            "value": "None"
                        },
                        {
                            "key": "db.statement",
                            "type": "string",
                            "value": "HGET 1f3552da-ee3d-440b-94a9-d4a11223179c"
                        },
                        {
                            "key": "net.peer.name",
                            "type": "string",
                            "value": "redis-cart"
                        },
                        {
                            "key": "net.peer.port",
                            "type": "int64",
                            "value": 6379
                        },
                        {
                            "key": "db.redis.database_index",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "peer.service",
                            "type": "string",
                            "value": "redis-cart:6379"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193138865,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Enqueued"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193138886,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "Sent"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193140394,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "ResponseReceived"
                                }
                            ]
                        }
                    ],
                    "processID": "p14",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "a8092c355d148d89",
                    "operationName": "charge",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "8346c777318c0b0b"
                        }
                    ],
                    "startTime": 1703716193184000,
                    "duration": 127,
                    "tags": [
                        {
                            "key": "app.payment.card_type",
                            "type": "string",
                            "value": "visa"
                        },
                        {
                            "key": "app.payment.card_valid",
                            "type": "bool",
                            "value": true
                        },
                        {
                            "key": "app.payment.charged",
                            "type": "bool",
                            "value": true
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "internal"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p15",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "8346c777318c0b0b",
                    "operationName": "grpc.oteldemo.PaymentService/Charge",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "9a9eee311cc0da41"
                        }
                    ],
                    "startTime": 1703716193184000,
                    "duration": 627,
                    "tags": [
                        {
                            "key": "rpc.system",
                            "type": "string",
                            "value": "grpc"
                        },
                        {
                            "key": "rpc.method",
                            "type": "string",
                            "value": "Charge"
                        },
                        {
                            "key": "rpc.service",
                            "type": "string",
                            "value": "oteldemo.PaymentService"
                        },
                        {
                            "key": "app.payment.amount",
                            "type": "float64",
                            "value": 694.314002653
                        },
                        {
                            "key": "rpc.grpc.status_code",
                            "type": "int64",
                            "value": 0
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p16",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "d8c4735d06842c7d",
                    "operationName": "router frontend egress",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "d2d4660e3d1e25ed"
                        }
                    ],
                    "startTime": 1703716193080235,
                    "duration": 15000116,
                    "tags": [
                        {
                            "key": "canceled",
                            "type": "string",
                            "value": "true"
                        },
                        {
                            "key": "http.protocol",
                            "type": "string",
                            "value": "HTTP/1.1"
                        },
                        {
                            "key": "upstream_address",
                            "type": "string",
                            "value": "172.18.0.24:8080"
                        },
                        {
                            "key": "peer.address",
                            "type": "string",
                            "value": "172.18.0.24:8080"
                        },
                        {
                            "key": "component",
                            "type": "string",
                            "value": "proxy"
                        },
                        {
                            "key": "upstream_cluster",
                            "type": "string",
                            "value": "frontend"
                        },
                        {
                            "key": "upstream_cluster.name",
                            "type": "string",
                            "value": "frontend"
                        },
                        {
                            "key": "http.status_code",
                            "type": "string",
                            "value": "0"
                        },
                        {
                            "key": "response_flags",
                            "type": "string",
                            "value": "-"
                        },
                        {
                            "key": "error",
                            "type": "string",
                            "value": "true"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p17",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "d2d4660e3d1e25ed",
                    "operationName": "ingress",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "c0b2e055b2ec26ce"
                        }
                    ],
                    "startTime": 1703716193079721,
                    "duration": 15000833,
                    "tags": [
                        {
                            "key": "node_id",
                            "type": "string",
                            "value": ""
                        },
                        {
                            "key": "zone",
                            "type": "string",
                            "value": ""
                        },
                        {
                            "key": "guid:x-request-id",
                            "type": "string",
                            "value": "a87e2374-2bed-9633-adea-9935d3baeeb5"
                        },
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://localhost:8080/api/checkout?currencyCode=BRL"
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "downstream_cluster",
                            "type": "string",
                            "value": "-"
                        },
                        {
                            "key": "user_agent",
                            "type": "string",
                            "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
                        },
                        {
                            "key": "http.protocol",
                            "type": "string",
                            "value": "HTTP/1.1"
                        },
                        {
                            "key": "peer.address",
                            "type": "string",
                            "value": "172.18.0.1"
                        },
                        {
                            "key": "request_size",
                            "type": "string",
                            "value": "374"
                        },
                        {
                            "key": "response_size",
                            "type": "string",
                            "value": "24"
                        },
                        {
                            "key": "component",
                            "type": "string",
                            "value": "proxy"
                        },
                        {
                            "key": "upstream_cluster",
                            "type": "string",
                            "value": "frontend"
                        },
                        {
                            "key": "upstream_cluster.name",
                            "type": "string",
                            "value": "frontend"
                        },
                        {
                            "key": "http.status_code",
                            "type": "string",
                            "value": "504"
                        },
                        {
                            "key": "response_flags",
                            "type": "string",
                            "value": "UT"
                        },
                        {
                            "key": "error",
                            "type": "string",
                            "value": "true"
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p17",
                    "warnings": null
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "0b9d38930998104c",
                    "operationName": "POST",
                    "references": [
                        {
                            "refType": "CHILD_OF",
                            "traceID": "864f33c21e7388fc27615dd9f45cb499",
                            "spanID": "d8c4735d06842c7d"
                        }
                    ],
                    "startTime": 1703716193081000,
                    "duration": 14999553,
                    "tags": [
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://localhost:8080/api/checkout?currencyCode=BRL"
                        },
                        {
                            "key": "http.host",
                            "type": "string",
                            "value": "localhost:8080"
                        },
                        {
                            "key": "net.host.name",
                            "type": "string",
                            "value": "localhost"
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "http.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.target",
                            "type": "string",
                            "value": "/api/checkout?currencyCode=BRL"
                        },
                        {
                            "key": "http.user_agent",
                            "type": "string",
                            "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
                        },
                        {
                            "key": "http.request_content_length_uncompressed",
                            "type": "int64",
                            "value": 374
                        },
                        {
                            "key": "http.flavor",
                            "type": "string",
                            "value": "1.1"
                        },
                        {
                            "key": "net.transport",
                            "type": "string",
                            "value": "ip_tcp"
                        },
                        {
                            "key": "net.peer.ip",
                            "type": "string",
                            "value": "::ffff:172.18.0.26"
                        },
                        {
                            "key": "net.peer.port",
                            "type": "int64",
                            "value": 36608
                        },
                        {
                            "key": "http.status_code",
                            "type": "int64",
                            "value": 200
                        },
                        {
                            "key": "http.status_text",
                            "type": "string",
                            "value": ""
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "server"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [],
                    "processID": "p18",
                    "warnings": [
                        "clock skew adjustment disabled; not applying calculated delta of -483.647µs"
                    ]
                },
                {
                    "traceID": "864f33c21e7388fc27615dd9f45cb499",
                    "spanID": "c0b2e055b2ec26ce",
                    "operationName": "HTTP POST",
                    "references": [],
                    "startTime": 1703716193073000,
                    "duration": 15014000,
                    "tags": [
                        {
                            "key": "app.session.id",
                            "type": "string",
                            "value": "1f3552da-ee3d-440b-94a9-d4a11223179c"
                        },
                        {
                            "key": "component",
                            "type": "string",
                            "value": "fetch"
                        },
                        {
                            "key": "http.method",
                            "type": "string",
                            "value": "POST"
                        },
                        {
                            "key": "http.url",
                            "type": "string",
                            "value": "http://localhost:8080/api/checkout?currencyCode=BRL"
                        },
                        {
                            "key": "app.synthetic_request",
                            "type": "string",
                            "value": "false"
                        },
                        {
                            "key": "http.status_code",
                            "type": "int64",
                            "value": 504
                        },
                        {
                            "key": "http.status_text",
                            "type": "string",
                            "value": "Gateway Timeout"
                        },
                        {
                            "key": "http.host",
                            "type": "string",
                            "value": "localhost:8080"
                        },
                        {
                            "key": "http.scheme",
                            "type": "string",
                            "value": "http"
                        },
                        {
                            "key": "http.user_agent",
                            "type": "string",
                            "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
                        },
                        {
                            "key": "http.response_content_length",
                            "type": "int64",
                            "value": 24
                        },
                        {
                            "key": "span.kind",
                            "type": "string",
                            "value": "client"
                        },
                        {
                            "key": "internal.span.format",
                            "type": "string",
                            "value": "otlp"
                        }
                    ],
                    "logs": [
                        {
                            "timestamp": 1703716193074000,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "fetchStart"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193074000,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "domainLookupStart"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193074000,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "domainLookupEnd"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193074000,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "connectStart"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193074000,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "connectEnd"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716193078800,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "requestStart"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716208080800,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "responseStart"
                                }
                            ]
                        },
                        {
                            "timestamp": 1703716208083700,
                            "fields": [
                                {
                                    "key": "event",
                                    "type": "string",
                                    "value": "responseEnd"
                                }
                            ]
                        }
                    ],
                    "processID": "p19",
                    "warnings": null
                }
            ],
            "processes": {
                "p1": {
                    "serviceName": "currencyservice",
                    "tags": [
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "currencyservice"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "cpp"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.13.0"
                        }
                    ]
                },
                "p10": {
                    "serviceName": "featureflagservice",
                    "tags": [
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "opentelemetry_ecto"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "1.1.1"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "featureflagservice"
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Erlang/OTP 23 erts-11.2.2.13"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "BEAM"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "11.2.2.13"
                        },
                        {
                            "key": "service.instance.id",
                            "type": "string",
                            "value": "featureflagservice@f6fad7832c2c"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "erlang"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.3.0"
                        }
                    ]
                },
                "p11": {
                    "serviceName": "cartservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "aadec63f51a1b7ed4457f2e599fb4b8c5433f5f5fba00546796d7d4459094967"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "Microsoft.AspNetCore"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "dotnet"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.6.0"
                        }
                    ]
                },
                "p12": {
                    "serviceName": "cartservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "aadec63f51a1b7ed4457f2e599fb4b8c5433f5f5fba00546796d7d4459094967"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "System.Net.Http"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "dotnet"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.6.0"
                        }
                    ]
                },
                "p13": {
                    "serviceName": "cartservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "aadec63f51a1b7ed4457f2e599fb4b8c5433f5f5fba00546796d7d4459094967"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "OpenTelemetry.Instrumentation.GrpcNetClient"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "1.0.0.0"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "dotnet"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.6.0"
                        }
                    ]
                },
                "p14": {
                    "serviceName": "cartservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "aadec63f51a1b7ed4457f2e599fb4b8c5433f5f5fba00546796d7d4459094967"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "OpenTelemetry.Instrumentation.StackExchangeRedis"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "1.0.0.12"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "dotnet"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.6.0"
                        }
                    ]
                },
                "p15": {
                    "serviceName": "paymentservice",
                    "tags": [
                        {
                            "key": "host.arch",
                            "type": "string",
                            "value": "amd64"
                        },
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "d3bb88372669"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "os.version",
                            "type": "string",
                            "value": "6.2.0-39-generic"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "paymentservice"
                        },
                        {
                            "key": "process.command",
                            "type": "string",
                            "value": "/usr/src/app/index.js"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"/usr/local/bin/node\",\"--require\",\"./opentelemetry.js\",\"/usr/src/app/index.js\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "node"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/local/bin/node"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "node"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 23
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Node.js"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "21.5.0"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.18.1"
                        }
                    ]
                },
                "p16": {
                    "serviceName": "paymentservice",
                    "tags": [
                        {
                            "key": "host.arch",
                            "type": "string",
                            "value": "amd64"
                        },
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "d3bb88372669"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "os.version",
                            "type": "string",
                            "value": "6.2.0-39-generic"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "@opentelemetry/instrumentation-grpc"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.45.1"
                        },
                        {
                            "key": "process.command",
                            "type": "string",
                            "value": "/usr/src/app/index.js"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"/usr/local/bin/node\",\"--require\",\"./opentelemetry.js\",\"/usr/src/app/index.js\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "node"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/local/bin/node"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "node"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 23
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Node.js"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "21.5.0"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.18.1"
                        }
                    ]
                },
                "p17": {
                    "serviceName": "frontend-proxy",
                    "tags": []
                },
                "p18": {
                    "serviceName": "frontend",
                    "tags": [
                        {
                            "key": "host.arch",
                            "type": "string",
                            "value": "amd64"
                        },
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "3569e905a783"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "os.version",
                            "type": "string",
                            "value": "6.2.0-39-generic"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "@opentelemetry/instrumentation-http"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.45.1"
                        },
                        {
                            "key": "process.command",
                            "type": "string",
                            "value": "/app/server.js"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"/usr/local/bin/node\",\"--require\",\"./Instrumentation.js\",\"/app/server.js\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "node"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/local/bin/node"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "nextjs"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 17
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Node.js"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "18.19.0"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "nodejs"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.18.1"
                        }
                    ]
                },
                "p19": {
                    "serviceName": "frontend-web",
                    "tags": [
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "@opentelemetry/instrumentation-fetch"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.45.1"
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Web Browser"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "browser"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "webjs"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.18.1"
                        }
                    ]
                },
                "p2": {
                    "serviceName": "quoteservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "87344fc5a968f97b672c1ff1ed217f765eb90500d4d3586e0a59467a71135756"
                        },
                        {
                            "key": "host.arch",
                            "type": "string",
                            "value": "x86_64"
                        },
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "87344fc5a968"
                        },
                        {
                            "key": "os.description",
                            "type": "string",
                            "value": "6.2.0-39-generic"
                        },
                        {
                            "key": "os.name",
                            "type": "string",
                            "value": "Linux"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "os.version",
                            "type": "string",
                            "value": "#40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "manual-instrumentation"
                        },
                        {
                            "key": "process.command",
                            "type": "string",
                            "value": "public/index.php"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"public/index.php\"]"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/local/bin/php"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "www-data"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 7
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "cli"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "8.2.13"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "service.version",
                            "type": "string",
                            "value": "1.0.0+no-version-set"
                        },
                        {
                            "key": "telemetry.distro.name",
                            "type": "string",
                            "value": "opentelemetry-php-instrumentation"
                        },
                        {
                            "key": "telemetry.distro.version",
                            "type": "string",
                            "value": "1.0.0"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "php"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.0.0"
                        }
                    ]
                },
                "p3": {
                    "serviceName": "quoteservice",
                    "tags": [
                        {
                            "key": "container.id",
                            "type": "string",
                            "value": "87344fc5a968f97b672c1ff1ed217f765eb90500d4d3586e0a59467a71135756"
                        },
                        {
                            "key": "host.arch",
                            "type": "string",
                            "value": "x86_64"
                        },
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "87344fc5a968"
                        },
                        {
                            "key": "os.description",
                            "type": "string",
                            "value": "6.2.0-39-generic"
                        },
                        {
                            "key": "os.name",
                            "type": "string",
                            "value": "Linux"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "os.version",
                            "type": "string",
                            "value": "#40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "io.opentelemetry.contrib.php.slim"
                        },
                        {
                            "key": "process.command",
                            "type": "string",
                            "value": "public/index.php"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"public/index.php\"]"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/local/bin/php"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "www-data"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 7
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "cli"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "8.2.13"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "service.version",
                            "type": "string",
                            "value": "1.0.0+no-version-set"
                        },
                        {
                            "key": "telemetry.distro.name",
                            "type": "string",
                            "value": "opentelemetry-php-instrumentation"
                        },
                        {
                            "key": "telemetry.distro.version",
                            "type": "string",
                            "value": "1.0.0"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "php"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.0.0"
                        }
                    ]
                },
                "p4": {
                    "serviceName": "shippingservice",
                    "tags": [
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "opentelemetry-otlp"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.14.0"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"/app/shippingservice\"]"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "rust"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "0.21.1"
                        }
                    ]
                },
                "p5": {
                    "serviceName": "shippingservice",
                    "tags": [
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "shippingservice"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"/app/shippingservice\"]"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "rust"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "0.21.1"
                        }
                    ]
                },
                "p6": {
                    "serviceName": "productcatalogservice",
                    "tags": [
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "6f04a6bd9bce"
                        },
                        {
                            "key": "os.description",
                            "type": "string",
                            "value": "Alpine Linux 3.19.0 (Linux 6f04a6bd9bce 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64)"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.46.0"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"./productcatalogservice\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "productcatalogservice"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/src/app/productcatalogservice"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "root"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "go version go1.21.4 linux/amd64"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "go1.21.4"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.20.0"
                        }
                    ]
                },
                "p7": {
                    "serviceName": "checkoutservice",
                    "tags": [
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "36ff72f20f7c"
                        },
                        {
                            "key": "os.description",
                            "type": "string",
                            "value": "Alpine Linux 3.19.0 (Linux 36ff72f20f7c 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64)"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.46.0"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"./checkoutservice\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "checkoutservice"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/src/app/checkoutservice"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "root"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "go version go1.21.4 linux/amd64"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "go1.21.4"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.20.0"
                        }
                    ]
                },
                "p8": {
                    "serviceName": "checkoutservice",
                    "tags": [
                        {
                            "key": "host.name",
                            "type": "string",
                            "value": "36ff72f20f7c"
                        },
                        {
                            "key": "os.description",
                            "type": "string",
                            "value": "Alpine Linux 3.19.0 (Linux 36ff72f20f7c 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64)"
                        },
                        {
                            "key": "os.type",
                            "type": "string",
                            "value": "linux"
                        },
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "checkoutservice"
                        },
                        {
                            "key": "process.command_args",
                            "type": "string",
                            "value": "[\"./checkoutservice\"]"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "checkoutservice"
                        },
                        {
                            "key": "process.executable.path",
                            "type": "string",
                            "value": "/usr/src/app/checkoutservice"
                        },
                        {
                            "key": "process.owner",
                            "type": "string",
                            "value": "root"
                        },
                        {
                            "key": "process.pid",
                            "type": "int64",
                            "value": 1
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "go version go1.21.4 linux/amd64"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "go1.21.4"
                        },
                        {
                            "key": "service.namespace",
                            "type": "string",
                            "value": "opentelemetry-demo"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "go"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.20.0"
                        }
                    ]
                },
                "p9": {
                    "serviceName": "featureflagservice",
                    "tags": [
                        {
                            "key": "otel.library.name",
                            "type": "string",
                            "value": "opentelemetry_grpcbox"
                        },
                        {
                            "key": "otel.library.version",
                            "type": "string",
                            "value": "0.2.0"
                        },
                        {
                            "key": "process.executable.name",
                            "type": "string",
                            "value": "featureflagservice"
                        },
                        {
                            "key": "process.runtime.description",
                            "type": "string",
                            "value": "Erlang/OTP 23 erts-11.2.2.13"
                        },
                        {
                            "key": "process.runtime.name",
                            "type": "string",
                            "value": "BEAM"
                        },
                        {
                            "key": "process.runtime.version",
                            "type": "string",
                            "value": "11.2.2.13"
                        },
                        {
                            "key": "service.instance.id",
                            "type": "string",
                            "value": "featureflagservice@f6fad7832c2c"
                        },
                        {
                            "key": "telemetry.sdk.language",
                            "type": "string",
                            "value": "erlang"
                        },
                        {
                            "key": "telemetry.sdk.name",
                            "type": "string",
                            "value": "opentelemetry"
                        },
                        {
                            "key": "telemetry.sdk.version",
                            "type": "string",
                            "value": "1.3.0"
                        }
                    ]
                }
            },
            "warnings": null
        }
    ],
    "total": 0,
    "limit": 0,
    "offset": 0,
    "errors": null
}

@mviitane
Copy link
Member

mviitane commented Jan 2, 2024

@julianocosta89 for me this PR fixed the problem. I see the rejects from CartService/EmptyCart when enabling the feature flag with 1.0. I don't see any rejects in frontend-proxy.
image

@julianocosta89
Copy link
Member

@mviitane does that reflect on the trace as well?

@mviitane
Copy link
Member

mviitane commented Jan 2, 2024

@mviitane does that reflect on the trace as well?

Yes
image

@julianocosta89
Copy link
Member

I'll try a fresh build

Copy link
Member

@julianocosta89 julianocosta89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, thanks @mviitane!
It works as expected

@julianocosta89 julianocosta89 merged commit b55b147 into open-telemetry:main Jan 2, 2024
27 checks passed
@puckpuck puckpuck deleted the cart.fix-flag-error branch January 2, 2024 19:18
jmichalak9 pushed a commit to jmichalak9/opentelemetry-demo that referenced this pull request Mar 22, 2024
* fix cartServiceFailure flag

Signed-off-by: Pierre Tessier <pierre@pierretessier.com>

* fix cartServiceFailure flag

Signed-off-by: Pierre Tessier <pierre@pierretessier.com>

---------

Signed-off-by: Pierre Tessier <pierre@pierretessier.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cartServiceFailure not working
3 participants