From b5c5ae17bfd00009112e79f390aab718e220e461 Mon Sep 17 00:00:00 2001 From: Kim Nylander Date: Thu, 1 Feb 2024 14:48:58 -0500 Subject: [PATCH 01/10] Add traces to profile content --- .../configure-server/about-server-api.md | 30 +++++++------- .../ingest-and-analyze-profile-data/_index.md | 36 ++++++++++++++--- .../analyze-profiles/_index.md | 2 + .../flamegraphs.md | 4 +- .../profile-cli.md | 3 +- .../profile-tracing/_index.md} | 3 ++ .../profile-tracing/traces-to-profiles.md | 16 ++++++++ .../profiling-types/_index.md | 2 + .../pyroscope-ui.md | 2 + .../self-vs-total.md | 39 ++++++++++--------- 10 files changed, 95 insertions(+), 42 deletions(-) rename docs/sources/{introduction => ingest-and-analyze-profile-data}/flamegraphs.md (97%) rename docs/sources/{introduction/profiling-tracing.md => ingest-and-analyze-profile-data/profile-tracing/_index.md} (97%) create mode 100644 docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md diff --git a/docs/sources/configure-server/about-server-api.md b/docs/sources/configure-server/about-server-api.md index 0cefad2f8e..63ba6ac8d4 100644 --- a/docs/sources/configure-server/about-server-api.md +++ b/docs/sources/configure-server/about-server-api.md @@ -1,17 +1,17 @@ --- description: Learn about the Pyrocope server API -menuTitle: Server API Overview +menuTitle: Server API overview title: Pyroscope Server HTTP API Reference weight: 20 --- -# Pyroscope Server HTTP API Reference +# Pyroscope server HTTP API reference Pyroscope server exposes an HTTP API for querying profiling data and ingesting profiling data from other sources. ## Authentication -Grafana Pyroscope does not include an authentication layer. Operators should use an authenticating reverse proxy for security. +Grafana Pyroscope doesn't include an authentication layer. Operators should use an authenticating reverse proxy for security. In multi-tenant mode, Pyroscope requires the X-Scope-OrgID HTTP header set to a string identifying the tenant. This responsibility should be handled by the authenticating reverse proxy. For more information, refer to the [multi-tenancy documentation]({{< relref "./about-tenant-ids" >}}). @@ -42,11 +42,11 @@ The request body contains profiling data, and the Content-Type header may be use Some of the query parameters depend on the format of profiling data. Pyroscope currently supports three major ingestion formats. -### Text Formats +### Text formats These formats handle simple ingestion of profiling data, such as `cpu` samples, and typically don't support metadata (e.g., labels) within the format. All necessary metadata is derived from query parameters, and the format is specified by the `format` query parameter. -**Supported Formats:** +**Supported formats:** - **Folded**: Also known as `collapsed`, this is the default format. Each line contains a stacktrace followed by the sample count for that stacktrace. For example: ``` @@ -62,7 +62,7 @@ foo;baz foo;bar ``` -### pprof format +### The `pprof` format The `pprof` format is a widely used binary profiling data format, particularly prevalent in the Go ecosystem. @@ -72,13 +72,13 @@ When using this format, certain query parameters have specific behaviors: - **name**: This parameter contains the _prefix_ of the application name. Since a single request might include multiple profile types, the complete application name is formed by concatenating this prefix with the profile type. For instance, if you send CPU profiling data and set `name` to `my-app{}`, it will be displayed in pyroscope as `my-app.cpu{}`. - **units**, **aggregationType**, and **sampleRate**: These parameters are ignored. The actual values are determined based on the profile types present in the data (refer to the "Sample Type Configuration" section for more details). -#### Sample Type Configuration +#### Sample type configuration -Pyroscope server inherently supports standard Go profile types such as `cpu`, `inuse_objects`, `inuse_space`, `alloc_objects`, and `alloc_space`. When dealing with software that generates data in pprof format, you may need to supply a custom sample type configuration for Pyroscope to interpret the data correctly. +Pyroscope server inherently supports standard Go profile types such as `cpu`, `inuse_objects`, `inuse_space`, `alloc_objects`, and `alloc_space`. When dealing with software that generates data in `pprof` format, you may need to supply a custom sample type configuration for Pyroscope to interpret the data correctly. -For an example Python script to ingest a pprof file with a custom sample type configuration, see **[this Python script](https://github.com/grafana/pyroscope/tree/main/examples/api/ingest_pprof.py).** +For an example Python script to ingest a `pprof` file with a custom sample type configuration, see **[this Python script](https://github.com/grafana/pyroscope/tree/main/examples/api/ingest_pprof.py).** -To ingest pprof data with custom sample type configuration, modify your requests as follows: +To ingest `pprof` data with custom sample type configuration, modify your requests as follows: * Set Content-Type to `multipart/form-data`. * Upload the profile data in a form file field named `profile`. * Include the sample type configuration in a form file field named `sample_type_config`. @@ -264,9 +264,9 @@ This table details the options for passing absolute values. | Option | Example | Notes | |:-----------------------|:----------------------|:-------------------| | Date | `20231223` | Format: `YYYYMMDD` | -| Unix Time seconds | `1577836800` | | -| Unix Time milliseconds | `1577836800000` | | -| Unix Time microseconds | `1577836800000000` | | +| Unix Time seconds | `1577836800` | | +| Unix Time milliseconds | `1577836800000` | | +| Unix Time microseconds | `1577836800000000` | | | Unix Time nanoseconds | `1577836800000000000` | | **Relative time** @@ -301,7 +301,7 @@ See the [Query output](#query-output) section for more information on the respon #### `maxNodes` -The `maxNodes` parameter truncates the number of elements in the profile response, to allow tools (for example, a frontend) to render large profiles efficiently. +The `maxNodes` parameter truncates the number of elements in the profile response, to allow tools (for example, a frontend) to render large profiles efficiently. This is typically used for profiles that are known to have large stack traces. When no value is provided, the default is taken from the `max_flamegraph_nodes_default` configuration parameter. @@ -377,7 +377,7 @@ This example groups by a cluster: } ``` -### Alternative Query Output +### Alternative query output When the `format` query parameter is `dot`, the endpoint responds with a [DOT format](https://en.wikipedia.org/wiki/DOT_(graph_description_language)) data representing the queried profile. This can be used to create an alternative visualization of the profile. diff --git a/docs/sources/ingest-and-analyze-profile-data/_index.md b/docs/sources/ingest-and-analyze-profile-data/_index.md index 1ff7250b79..b54c7ae20b 100644 --- a/docs/sources/ingest-and-analyze-profile-data/_index.md +++ b/docs/sources/ingest-and-analyze-profile-data/_index.md @@ -1,7 +1,9 @@ --- -title: Ingest and analyze profile data -menuTitle: Ingest and analyze profile data -description: How to use Pyroscope to ingest and analyze profile data. +title: View and analyze profile data +menuTitle: View and analyze profile data +description: How to use Pyroscope to view and analyze profile data. +aliases: + - ../ingest-and-analyze-profile-data/ weight: 50 keywords: - pyroscope @@ -11,8 +13,30 @@ keywords: - CLI --- -# Ingest and analyze profile data +# View and analyze profile data -The following topics include sections on how to use the Pyroscope CLI and UI to ingest and analyze profile data. +Profiling data can be presented in a variety of formats presents such as: +- **Flamegraphs**: Visualize call relationships and identify hotspots. +- **Tables**: View detailed statistics for specific functions or time periods. +- **Charts and graphs**: Analyze trends and compare performance across different metrics. -{{< section menuTitle="true" >}} +## Viewing profiles + +Pyroscope offers both a Command Line Interface (CLI) and an Application Programming Interface (API) to interact with and retrieve profiling data. These tools provide flexibility in how you access and manage your profiling information. + +You can export profiling data from Pyroscope in various formats: +- **JSON**: JSON object easy to integrate with other tools and scripts +- **dot**: text containing a DOT representation of the profile + +Integrating Pyroscope with Grafana is a common and recommended approach for visualizing profiling data. Grafana, being a powerful tool for data visualization, can effectively display profiling data in an accessible and insightful manner. + +Options for Visualizing Data in Grafana: +- **Pyroscope App Plugin**: This plugin is specifically designed for Pyroscope data. It allows for easy browsing, analysis, and comparison of multiple profiles across different labels or time periods. This is particularly useful for a comprehensive overview of your application's performance. +- **Explore Tab**: In Grafana, the Explore tab is suited for making targeted queries on your profiling data. This is useful for in-depth analysis of specific aspects of your application's performance. +- **Dashboard**: Grafana dashboards are excellent for integrating profiling data with other metrics. You can display Pyroscope data alongside other dashboard items, creating a unified view of your application’s overall health and performance. + +For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana](https://grafana.com/docs/pyroscope/latest/introduction/pyroscope-in-grafana/#pyroscope-and-profiling-in-grafana). + +Note the Pyroscope app plugin is cloud only... + +For more information on configuring these data sources, refer to the Pyroscope data source documentation in [Grafana Cloud](/docs/grafana-cloud/connect-externally-hosted/data-sources/grafana-pyroscope/) and [Grafana](/docs/grafana/latest/datasources/grafana-pyroscope/). diff --git a/docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md b/docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md index 14c559e4ca..657c4cf5e0 100644 --- a/docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md +++ b/docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md @@ -3,6 +3,8 @@ title: Analyze app performance using Pyroscope menuTitle: Analyze app performance description: How to use the Pyroscope UI to analyze performance of your applications. weight: 40 +aliases: + - ../ingest-and-analyze-profile-data/analyze-profiles/ draft: true keywords: - pyroscope diff --git a/docs/sources/introduction/flamegraphs.md b/docs/sources/ingest-and-analyze-profile-data/flamegraphs.md similarity index 97% rename from docs/sources/introduction/flamegraphs.md rename to docs/sources/ingest-and-analyze-profile-data/flamegraphs.md index b68050dc69..5236f25cc4 100644 --- a/docs/sources/introduction/flamegraphs.md +++ b/docs/sources/ingest-and-analyze-profile-data/flamegraphs.md @@ -2,7 +2,9 @@ title: "Flamegraphs: Visualizing performance data" menuTitle: Flamegraphs description: Learn about flamegraphs to help visualize performance data. -weight: 100 +weight: 33 +aliases: + - ../introduction/flamegraphs/ keywords: - Pyroscope - Profiling diff --git a/docs/sources/ingest-and-analyze-profile-data/profile-cli.md b/docs/sources/ingest-and-analyze-profile-data/profile-cli.md index 3529777260..86fdacfb50 100644 --- a/docs/sources/ingest-and-analyze-profile-data/profile-cli.md +++ b/docs/sources/ingest-and-analyze-profile-data/profile-cli.md @@ -3,10 +3,11 @@ aliases: - /docs/phlare/latest/operators-guide/configuring/profile-cli/ - /docs/phlare/latest/profile-cli/ - /docs/pyroscope/latest/configure-server/profile-cli/ + - ../ingest-and-analyze-profile-data/profile-cli/ description: Getting started with the profile CLI tool. menuTitle: Profile CLI title: Profile CLI -weight: 50 +weight: 60 --- # Profile CLI diff --git a/docs/sources/introduction/profiling-tracing.md b/docs/sources/ingest-and-analyze-profile-data/profile-tracing/_index.md similarity index 97% rename from docs/sources/introduction/profiling-tracing.md rename to docs/sources/ingest-and-analyze-profile-data/profile-tracing/_index.md index 70d36c5020..d77957671f 100644 --- a/docs/sources/introduction/profiling-tracing.md +++ b/docs/sources/ingest-and-analyze-profile-data/profile-tracing/_index.md @@ -3,6 +3,9 @@ title: Profiling and tracing integration menuTitle: Profiling and tracing description: Learning about how profiling and tracing work together. weight: 50 +aliases: + - ../introduction/traces-to-profiles/ + - ../introduction/profile-tracing/ keywords: - pyroscope - continuous profiling diff --git a/docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md b/docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md new file mode 100644 index 0000000000..ce05de2f54 --- /dev/null +++ b/docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md @@ -0,0 +1,16 @@ +--- +title: Traces to profiles +menuTitle: Traces to profiles +description: Learning about traces to profiles integration in Grafana and Grafana Cloud. +weight: 150 +keywords: + - pyroscope + - continuous profiling + - tracing +--- + +# Trace to profiles + +[//]: # 'Shared content for Trace to profiles in the Tempo data source' + +{{< docs/shared source="grafana" lookup="datasources/tempo-traces-to-profiles.md" version="" >}} \ No newline at end of file diff --git a/docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md b/docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md index 7ede9074a5..5d2df38b6d 100644 --- a/docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md +++ b/docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md @@ -3,6 +3,8 @@ title: Understand profiling types and their uses in Pyroscope menuTitle: Understand profiling types description: Learn about the different profiling types available in Pyroscope and how to effectively use them in your application performance analysis. weight: 30 +aliases: + - ../ingest-and-analyze-profile-data/profiling-types/ keywords: - pyroscope - profiling types diff --git a/docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md b/docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md index dd05422581..c399f1131b 100644 --- a/docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md +++ b/docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md @@ -3,6 +3,8 @@ title: Use the Pyroscope UI to explore profiling data menuTitle: Use the Pyroscope UI description: How to use the Pyroscope UI to explore profile data. weight: 40 +aliases: + - ../ingest-and-analyze-profile-data/profile-ui/ keywords: - pyroscope - performance analysis diff --git a/docs/sources/ingest-and-analyze-profile-data/self-vs-total.md b/docs/sources/ingest-and-analyze-profile-data/self-vs-total.md index 4f23b248c1..bec7645398 100644 --- a/docs/sources/ingest-and-analyze-profile-data/self-vs-total.md +++ b/docs/sources/ingest-and-analyze-profile-data/self-vs-total.md @@ -1,8 +1,10 @@ --- title: Understand 'self' vs. 'total' metrics in profiling with Pyroscope menuTitle: Understand 'self' vs. 'total' metrics -description: Learn the differences between 'self' and 'total' metrics in profiling and their specific applications in CPU and Memory profiling with Pyroscope -weight: 35 +description: Learn the differences between 'self' and 'total' metrics in profiling and their specific applications in CPU and Memory profiling with Pyroscope. +weight: 42 +aliases: + - ../ingest-and-analyze-profile-data/self-vs-total/ keywords: - pyroscope - cpu profiling @@ -17,28 +19,29 @@ Profiling in Pyroscope provideds many different ways of analyzing your profiling ![Self vs total annotated](https://grafana.com/media/docs/pyroscope/self_vs_total_annotated.png) -## General Concepts of 'Self' and 'Total' +## Self + +'Self' refers to the resource usage (CPU time, memory allocation, etc.) directly attributed to a specific function or a code segment, excluding the resources used by its sub-functions or calls + +This metric helps isolate the direct impact of a specific code block, making it crucial for pinpointing primary resource consumers ![Self vs total diagram](https://grafana.com/media/docs/pyroscope/self_vs_total_explainer_diagram.png) -### What is 'Self'? -- **Definition**: 'Self' refers to the resource usage (CPU time, memory allocation, etc.) directly attributed to a specific function or a code segment, excluding the resources used by its sub-functions or calls -- **Importance**: This metric helps isolate the direct impact of a specific code block, making it crucial for pinpointing primary resource consumers +## Total -### What is 'Total'? +'Total' encompasses the combined resource usage of a function along with all the functions it calls -- **Definition**: 'Total' encompasses the combined resource usage of a function along with all the functions it calls -- **Importance**: It provides a holistic view of a function's overall resource consumption, essential for understanding cumulative impacts +It provides a holistic view of a function's overall resource consumption, essential for understanding cumulative impacts -## 'Self' and 'Total' in CPU Profiling +## 'Self' and 'Total' in CPU profiling -### CPU Profiling Specifics +In CPU profiling, 'self' indicates the CPU time consumed directly by the function, crucial for identifying functions with high CPU demand. -- **Self in CPU**: In CPU profiling, 'self' indicates the CPU time consumed directly by the function, crucial for identifying functions with high CPU demand. -- **Total in CPU**: The 'total' CPU time includes time spent in the function itself plus time in all called functions, highlighting comprehensive CPU usage +The 'total' CPU time includes time spent in the function itself plus time in all called functions, highlighting comprehensive CPU usage -In the example below, we demonstrate a simplified pseudocode representation of a CPU-intensive process. This illustrates how 'self' and 'total' time would be calculated for different functions in a typical application. The following diagram provides a visual representation of these concepts. +The example below demonstrates a simplified pseudocode representation of a CPU-intensive process. +This illustrates how 'self' and 'total' time would be calculated for different functions in a typical application. The following diagram provides a visual representation of these concepts. ![Self vs total diagram](https://grafana.com/media/docs/pyroscope/self_vs_total_explainer_diagram.png) @@ -68,14 +71,12 @@ def apply(): handle_request() ``` -## 'Self' and 'Total' in Memory Profiling - -### Memory Profiling Specifics +## 'Self' and 'Total' in memory profiling - **Self in Memory**: For memory profiling, 'self' measures the memory allocated by the function itself, vital for detecting direct memory allocation issues. -- **Total in Memory**: 'Total' memory includes allocations by the function and its callee functions, essential for assessing overall memory footprint. +- **Total in Memory**: 'Total' memory includes allocations by the function and its called functions, essential for assessing overall memory footprint. -The same example from the CPU profiling section can be used to illustrate the concepts of 'self' and 'total' in memory profiling, just with memory units instead of cpu. +The same example from the CPU profiling section can be used to illustrate the concepts of 'self' and 'total' in memory profiling, just with memory units instead of CPU. ## Conclusion From 499105e3b2f9f7a2d45cb275d0a6c4b06ea0f03c Mon Sep 17 00:00:00 2001 From: Kim Nylander Date: Thu, 1 Feb 2024 14:50:27 -0500 Subject: [PATCH 02/10] Rename folder to view and analyze --- .../_index.md | 0 .../analyze-profiles/_index.md | 0 .../flamegraphs.md | 0 .../profile-cli.md | 0 .../profile-tracing/_index.md | 0 .../profile-tracing/traces-to-profiles.md | 0 .../profiling-types/_index.md | 0 .../pyroscope-ui.md | 0 .../self-vs-total.md | 0 9 files changed, 0 insertions(+), 0 deletions(-) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/_index.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/analyze-profiles/_index.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/flamegraphs.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/profile-cli.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/profile-tracing/_index.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/profile-tracing/traces-to-profiles.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/profiling-types/_index.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/pyroscope-ui.md (100%) rename docs/sources/{ingest-and-analyze-profile-data => view-and-analyze-profile-data}/self-vs-total.md (100%) diff --git a/docs/sources/ingest-and-analyze-profile-data/_index.md b/docs/sources/view-and-analyze-profile-data/_index.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/_index.md rename to docs/sources/view-and-analyze-profile-data/_index.md diff --git a/docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md b/docs/sources/view-and-analyze-profile-data/analyze-profiles/_index.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/analyze-profiles/_index.md rename to docs/sources/view-and-analyze-profile-data/analyze-profiles/_index.md diff --git a/docs/sources/ingest-and-analyze-profile-data/flamegraphs.md b/docs/sources/view-and-analyze-profile-data/flamegraphs.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/flamegraphs.md rename to docs/sources/view-and-analyze-profile-data/flamegraphs.md diff --git a/docs/sources/ingest-and-analyze-profile-data/profile-cli.md b/docs/sources/view-and-analyze-profile-data/profile-cli.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/profile-cli.md rename to docs/sources/view-and-analyze-profile-data/profile-cli.md diff --git a/docs/sources/ingest-and-analyze-profile-data/profile-tracing/_index.md b/docs/sources/view-and-analyze-profile-data/profile-tracing/_index.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/profile-tracing/_index.md rename to docs/sources/view-and-analyze-profile-data/profile-tracing/_index.md diff --git a/docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md b/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/profile-tracing/traces-to-profiles.md rename to docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md diff --git a/docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md b/docs/sources/view-and-analyze-profile-data/profiling-types/_index.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/profiling-types/_index.md rename to docs/sources/view-and-analyze-profile-data/profiling-types/_index.md diff --git a/docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md b/docs/sources/view-and-analyze-profile-data/pyroscope-ui.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/pyroscope-ui.md rename to docs/sources/view-and-analyze-profile-data/pyroscope-ui.md diff --git a/docs/sources/ingest-and-analyze-profile-data/self-vs-total.md b/docs/sources/view-and-analyze-profile-data/self-vs-total.md similarity index 100% rename from docs/sources/ingest-and-analyze-profile-data/self-vs-total.md rename to docs/sources/view-and-analyze-profile-data/self-vs-total.md From 88c498b1b3511ec1a3c356ec3a8f1dc8a7a70c7b Mon Sep 17 00:00:00 2001 From: Kim Nylander Date: Thu, 1 Feb 2024 15:18:34 -0500 Subject: [PATCH 03/10] Fix links --- docs/sources/configure-server/about-server-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/configure-server/about-server-api.md b/docs/sources/configure-server/about-server-api.md index 63ba6ac8d4..b36f76e3a9 100644 --- a/docs/sources/configure-server/about-server-api.md +++ b/docs/sources/configure-server/about-server-api.md @@ -249,7 +249,7 @@ In a Kubernetes environment, a query could also look like: `process_cpu:cpu:nanoseconds:cpu:nanoseconds{namespace="dev", container="my_application_name"}` {{% admonition type="note" %}} -Refer to the [profiling types documentation]({{< relref "../ingest-and-analyze-profile-data/profiling-types" >}}) for more information and [profile-metrics.json](https://github.com/grafana/pyroscope/blob/main/public/app/constants/profile-metrics.json) for a list of valid profile types. +Refer to the [profiling types documentation]({{< relref "../view-and-analyze-profile-data/profiling-types" >}}) for more information and [profile-metrics.json](https://github.com/grafana/pyroscope/blob/main/public/app/constants/profile-metrics.json) for a list of valid profile types. {{% /admonition %}} #### `from` and `until` @@ -419,4 +419,4 @@ See [this Python script](https://github.com/grafana/pyroscope/tree/main/examples The `profilecli` tool can also be used to interact with the Pyroscope server API. The tool supports operations such as ingesting profiles, querying for existing profiles, and more. -Refer to the [Profile CLI]({{< relref "../ingest-and-analyze-profile-data/profile-cli" >}}) page for more information. +Refer to the [Profile CLI]({{< relref "../view-and-analyze-profile-data/profile-cli" >}}) page for more information. From 85ddd76cc557841bdc4b2b3fb512d9b12f83c6b8 Mon Sep 17 00:00:00 2001 From: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Date: Thu, 1 Feb 2024 15:55:51 -0500 Subject: [PATCH 04/10] Update docs/sources/view-and-analyze-profile-data/_index.md --- docs/sources/view-and-analyze-profile-data/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/view-and-analyze-profile-data/_index.md b/docs/sources/view-and-analyze-profile-data/_index.md index b54c7ae20b..16b2ec80d1 100644 --- a/docs/sources/view-and-analyze-profile-data/_index.md +++ b/docs/sources/view-and-analyze-profile-data/_index.md @@ -35,7 +35,7 @@ Options for Visualizing Data in Grafana: - **Explore Tab**: In Grafana, the Explore tab is suited for making targeted queries on your profiling data. This is useful for in-depth analysis of specific aspects of your application's performance. - **Dashboard**: Grafana dashboards are excellent for integrating profiling data with other metrics. You can display Pyroscope data alongside other dashboard items, creating a unified view of your application’s overall health and performance. -For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana](https://grafana.com/docs/pyroscope/latest/introduction/pyroscope-in-grafana/#pyroscope-and-profiling-in-grafana). +For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana]({{< relref "../introduction/pyroscope-in-grafana/#pyroscope-and-profiling-in-grafana" >}}). Note the Pyroscope app plugin is cloud only... From a9b470736ab8b2a16853e400158cc7df85e1b980 Mon Sep 17 00:00:00 2001 From: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Date: Fri, 2 Feb 2024 10:17:34 -0500 Subject: [PATCH 05/10] Apply suggestions from code review --- docs/sources/configure-server/about-server-api.md | 6 +++--- .../profile-tracing/traces-to-profiles.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/sources/configure-server/about-server-api.md b/docs/sources/configure-server/about-server-api.md index b36f76e3a9..da41509d47 100644 --- a/docs/sources/configure-server/about-server-api.md +++ b/docs/sources/configure-server/about-server-api.md @@ -1,11 +1,11 @@ --- description: Learn about the Pyrocope server API -menuTitle: Server API overview -title: Pyroscope Server HTTP API Reference +menuTitle: Server HTTP API +title: Pyroscope Server HTTP API weight: 20 --- -# Pyroscope server HTTP API reference +# Pyroscope server HTTP API Pyroscope server exposes an HTTP API for querying profiling data and ingesting profiling data from other sources. diff --git a/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md b/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md index ce05de2f54..382b8746f7 100644 --- a/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md +++ b/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md @@ -9,7 +9,7 @@ keywords: - tracing --- -# Trace to profiles +# Traces to profiles [//]: # 'Shared content for Trace to profiles in the Tempo data source' From 5a5610e6d612865cf23de05407f0d6002dd166b3 Mon Sep 17 00:00:00 2001 From: Kim Nylander Date: Fri, 2 Feb 2024 10:17:53 -0500 Subject: [PATCH 06/10] Fix links --- docs/sources/introduction/continuous-profiling/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/introduction/continuous-profiling/_index.md b/docs/sources/introduction/continuous-profiling/_index.md index bd28134f57..78e829dbe6 100644 --- a/docs/sources/introduction/continuous-profiling/_index.md +++ b/docs/sources/introduction/continuous-profiling/_index.md @@ -21,7 +21,7 @@ This is great for debugging but not so great for production. ![example flamegraph](https://grafana.com/static/img/pyroscope/pyroscope-ui-single-2023-11-30.png) {{% admonition type="note" %}} -To learn more about flamegraphs, refer to [Flamegraphs: Visualizing performance data]({{< relref "../flamegraphs" >}}). +To learn more about flamegraphs, refer to [Flamegraphs: Visualizing performance data]({{< relref "../../view-and-analyze-profile-data/flamegraphs" >}}). {{% /admonition %}} Continuous profiling is a modern approach which is safer and more scalable for production environments. @@ -54,7 +54,7 @@ Pyroscope's low-overhead profiling enables precise optimization of resource usag By providing in-depth insights into application performance, Pyroscope allows teams to identify and eliminate inefficiencies, leading to significant savings in areas like observability, incident management, messaging/queuing, deployment tools, and infrastructure. By using sampling profilers, Pyroscope is able to collect data with minimal overhead (~2-5% depending on a few factors). -The [custom storage engine](https://pyroscope.io/docs/storage-design/) compresses and stores the data efficiently. +The [custom storage engine]({{< relref "../../reference-pyroscope-architecture/about-grafana-pyroscope-architecture" >}}) compresses and stores the data efficiently. Some advantages of this are: - Low CPU overhead thanks to sampling profiler technology From 345131f83eceb7421e23450ee5caf9adc28d31ac Mon Sep 17 00:00:00 2001 From: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Date: Fri, 2 Feb 2024 10:43:06 -0500 Subject: [PATCH 07/10] Apply suggestions from code review --- docs/sources/configure-server/about-server-api.md | 2 +- docs/sources/view-and-analyze-profile-data/_index.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/configure-server/about-server-api.md b/docs/sources/configure-server/about-server-api.md index da41509d47..8b996bfc6b 100644 --- a/docs/sources/configure-server/about-server-api.md +++ b/docs/sources/configure-server/about-server-api.md @@ -1,7 +1,7 @@ --- description: Learn about the Pyrocope server API menuTitle: Server HTTP API -title: Pyroscope Server HTTP API +title: Pyroscope server HTTP API weight: 20 --- diff --git a/docs/sources/view-and-analyze-profile-data/_index.md b/docs/sources/view-and-analyze-profile-data/_index.md index 16b2ec80d1..65941cc678 100644 --- a/docs/sources/view-and-analyze-profile-data/_index.md +++ b/docs/sources/view-and-analyze-profile-data/_index.md @@ -35,7 +35,7 @@ Options for Visualizing Data in Grafana: - **Explore Tab**: In Grafana, the Explore tab is suited for making targeted queries on your profiling data. This is useful for in-depth analysis of specific aspects of your application's performance. - **Dashboard**: Grafana dashboards are excellent for integrating profiling data with other metrics. You can display Pyroscope data alongside other dashboard items, creating a unified view of your application’s overall health and performance. -For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana]({{< relref "../introduction/pyroscope-in-grafana/#pyroscope-and-profiling-in-grafana" >}}). +For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana]({{< relref "../introduction/pyroscope-in-grafana#pyroscope-and-profiling-in-grafana" >}}). Note the Pyroscope app plugin is cloud only... From d6cd7f092e85e7e84f926af34716147e52dc8a58 Mon Sep 17 00:00:00 2001 From: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Date: Fri, 2 Feb 2024 11:07:17 -0500 Subject: [PATCH 08/10] Update docs/sources/view-and-analyze-profile-data/_index.md --- docs/sources/view-and-analyze-profile-data/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/view-and-analyze-profile-data/_index.md b/docs/sources/view-and-analyze-profile-data/_index.md index 65941cc678..a09967f7d3 100644 --- a/docs/sources/view-and-analyze-profile-data/_index.md +++ b/docs/sources/view-and-analyze-profile-data/_index.md @@ -37,6 +37,6 @@ Options for Visualizing Data in Grafana: For more information on using profiles in Grafana, refer to [Pyroscope and profiles in Grafana]({{< relref "../introduction/pyroscope-in-grafana#pyroscope-and-profiling-in-grafana" >}}). -Note the Pyroscope app plugin is cloud only... +The Pyroscope app plugin works for Grafana Cloud. For more information on configuring these data sources, refer to the Pyroscope data source documentation in [Grafana Cloud](/docs/grafana-cloud/connect-externally-hosted/data-sources/grafana-pyroscope/) and [Grafana](/docs/grafana/latest/datasources/grafana-pyroscope/). From 69fcb849b075164a6d12cf8eabb1548d70beefe2 Mon Sep 17 00:00:00 2001 From: Kim Nylander Date: Fri, 2 Feb 2024 15:24:19 -0500 Subject: [PATCH 09/10] Add link to PR 2982 content --- .../profile-tracing/traces-to-profiles.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md b/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md index 382b8746f7..4b4e6768c9 100644 --- a/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md +++ b/docs/sources/view-and-analyze-profile-data/profile-tracing/traces-to-profiles.md @@ -11,6 +11,12 @@ keywords: # Traces to profiles +{{< admonition type="note" >}} + +Your application must be instrumented for profiles and traces. For more information, refer to [Span profiles for Traces to profiles]({{< relref "../../configure-client/go-span-profiles" >}}). + +{{< /admonition >}} + [//]: # 'Shared content for Trace to profiles in the Tempo data source' {{< docs/shared source="grafana" lookup="datasources/tempo-traces-to-profiles.md" version="" >}} \ No newline at end of file From 0735380f374d8564e410f99b12841bd5aed74e52 Mon Sep 17 00:00:00 2001 From: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Date: Tue, 6 Feb 2024 15:29:27 -0500 Subject: [PATCH 10/10] Apply suggestions from code review --- docs/sources/view-and-analyze-profile-data/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/view-and-analyze-profile-data/_index.md b/docs/sources/view-and-analyze-profile-data/_index.md index a09967f7d3..0aca33d595 100644 --- a/docs/sources/view-and-analyze-profile-data/_index.md +++ b/docs/sources/view-and-analyze-profile-data/_index.md @@ -25,8 +25,8 @@ Profiling data can be presented in a variety of formats presents such as: Pyroscope offers both a Command Line Interface (CLI) and an Application Programming Interface (API) to interact with and retrieve profiling data. These tools provide flexibility in how you access and manage your profiling information. You can export profiling data from Pyroscope in various formats: +- **pprof**: Support for pprof, gzip compressed pprof, for example, `foo.pprof.gz` - **JSON**: JSON object easy to integrate with other tools and scripts -- **dot**: text containing a DOT representation of the profile Integrating Pyroscope with Grafana is a common and recommended approach for visualizing profiling data. Grafana, being a powerful tool for data visualization, can effectively display profiling data in an accessible and insightful manner.