diff --git a/changelog/12527.doc.md b/changelog/12527.doc.md new file mode 100644 index 000000000000..b21312bbb3d1 --- /dev/null +++ b/changelog/12527.doc.md @@ -0,0 +1 @@ +Added Documentation for Realtime Markers Section. \ No newline at end of file diff --git a/docs/docs/command-line-interface.mdx b/docs/docs/command-line-interface.mdx index e3079b3a797c..48e057e215d6 100644 --- a/docs/docs/command-line-interface.mdx +++ b/docs/docs/command-line-interface.mdx @@ -30,6 +30,7 @@ import RasaProBanner from "@theme/RasaProBanner"; |`rasa data validate` |Checks the domain, NLU and conversation data for inconsistencies. | |`rasa export` |Exports conversations from a tracker store to an event broker. | |`rasa evaluate markers` |Extracts markers from an existing tracker store. | +|`rasa marker upload` |Upload marker configurations to Analytics Data Pipeline | |`rasa license` |Display licensing information. | |`rasa -h` |Shows all available commands. | @@ -663,6 +664,68 @@ Python Logging Options: --quiet Be quiet! Sets logging level to WARNING. (default: None) ``` +## rasa markers upload + + + + + +:::info New in 3.6 + +This command is available from Rasa Pro 3.6.0 and requires [Rasa Analytics Data Pipeline](./monitoring/analytics/getting-started-with-analytics.mdx) +::: + +This command applies to [markers](./markers.mdx) and their [real-time processing](./monitoring/analytics/realtime-markers.mdx). +Running this command validates the marker configuration file against the domain file and uploads the configuration to Analytics Data Pipeline + +``` +usage: rasa markers upload [-h] [-v] [-vv] [--quiet] + [--logging-config-file LOGGING_CONFIG_FILE] + [--config CONFIG] + [--rasa-pro-services-url RASA_PRO_SERVICES_URL] + [-d DOMAIN] + +optional arguments: + -h, --help show this help message and exit + --config CONFIG The marker configuration file(s) containing marker + definitions. This can be a single YAML file, or a + directory that contains several files with marker + definitions in it. The content of these files will be + read and merged together. (default: markers.yml) + --rasa-pro-services-url RASA_PRO_SERVICES_URL + The URL of the Rasa Pro Services instance to upload + markers to.Specified URL should not contain a trailing + slash. (default: ) + -d DOMAIN, --domain DOMAIN + Domain specification. This can be a single YAML file, + or a directory that contains several files with domain + specifications in it. The content of these files will + be read and merged together. (default: domain.yml) + +Python Logging Options: + You can control level of log messages printed. In addition to these + arguments, a more fine grained configuration can be achieved with + environment variables. See online documentation for more info. + + -v, --verbose Be verbose. Sets logging level to INFO. (default: + None) + -vv, --debug Print lots of debugging statements. Sets logging level + to DEBUG. (default: None) + --quiet Be quiet! Sets logging level to WARNING. (default: + None) + --logging-config-file LOGGING_CONFIG_FILE + If set, the name of the logging configuration file + will be set to the given name. (default: None) + +Description: + The `rasa markers upload` command allows you to upload markers to the Rasa Pro Services. Markers are custom conversational events that provide additional context for analysis and insights generation. By uploading markers, you can enable real-time analysis and enhance the performance of your Rasa Assistant. + +Examples: + Upload Markers to Rasa Pro Services: + rasa markers upload --config markers.yml --rasa-pro-services-url https://example.com/rasa-pro -d domain.yml + +``` + ## rasa license diff --git a/docs/docs/markers.mdx b/docs/docs/markers.mdx index 24d4c35059ab..9414314f68de 100644 --- a/docs/docs/markers.mdx +++ b/docs/docs/markers.mdx @@ -14,6 +14,12 @@ This feature is currently experimental and might change or be removed in the fut ::: +:::danger + +In the upcoming release version 3.7 of Rasa Open Source, we’re removing this experimental feature. For documentation on the markers feature in Rasa Pro, please [click here](./monitoring/analytics/realtime-markers.mdx) + +::: + ## Overview Markers are conditions that allow you to describe and mark points of interest in dialogues for evaluating your bot. @@ -173,6 +179,12 @@ You cannot reuse an existing marker name in the definition of another marker. ## Extracting Markers +:::info + +Rasa Pro supports real-time processing of markers, Read more about [Real-Time Markers](monitoring/analytics/realtime-markers.mdx) + +::: + Markers are extracted from dialogues already stored in a tracker store. To learn how to store interactions with your bot in a tracker store, read the [Tracker Store](./tracker-stores.mdx) page. diff --git a/docs/docs/monitoring/analytics/data-structure-reference.mdx b/docs/docs/monitoring/analytics/data-structure-reference.mdx index ded6ba7d568e..9f6fc95435e5 100644 --- a/docs/docs/monitoring/analytics/data-structure-reference.mdx +++ b/docs/docs/monitoring/analytics/data-structure-reference.mdx @@ -781,6 +781,120 @@ Time of the last update of the slot in this session. The timestamp is a UTC. - Type: `DateTime` - Example: `2022-06-21 02:15:49.326936` +### rasa_patterns + +Patterns are marker definitions that have been received from Rasa. This table is called `patterns` +to distinguish them from extracted markers which are stored in `rasa_markers` table. It +stores the configuration of markers (which can be thought of as a pattern +of conversational events) along with their metadata. + +#### `id` pattern identifier + +The unique identifier of the rasa pattern is generated by Analytics. + +- Type: `varchar(36)` +- Example: `bd074dc7-e745-4db6-86d0-75b0af7bc067` + +#### `name` pattern name + +Name of the pattern + +- Type: `varchar()` +- Example: `registration success` + +#### `description` pattern description + +Description of the pattern + +- Type: `varchar()` +- Example: `This marker identifies successful account registration in the chat` + +#### `config` pattern configuration + +Pattern configuration dictionary stored as an escaped string + +- Type: `varchar()` +- Example: `"{'or': [{'intent': 'mood_unhappy'},{'intent': 'mood_great'}]}"` + +#### `is_active` soft-delete flag + +Only patterns with `is_active==True` are processed during real-time analysis + +- Type: `boolean` + +#### `created_at` creation date time + +Time of creation of this pattern. The timestamp is a UTC. + +- Type: `DateTime` +- Example: `2022-06-21 02:15:49.326936` + +#### `updated_at` update date time + +Time of the last update of the pattern in this session. The timestamp is a UTC. + +- Type: `DateTime` +- Example: `2022-06-21 02:15:49.326936` + +### rasa_markers + +Extracted markers from the conversations. Each row in this table corresponds +to a marker along with details of the pattern, sender, session and the last event where it was extracted. + +#### `id` marker identifier + +The unique identifier of the extracted rasa marker is generated by Analytics. + +- Type: `varchar(36)` +- Example: `bd074dc7-e745-4db6-86d0-75b0af7bc067` + +#### `pattern_id` pattern which was applied in this marker + +The unique identifier of the pattern which was applied in this marker. It is +a foreign key to the [`rasa_patterns.id`](#rasa_patterns) column + +- Type: `varchar(36)` +- Example: `9e4ebded-f232-4cc5-af78-d98daa0c1a53` + +#### `sender_id` sender identifier + +The unique identifier of the sender whose conversation this marker is part of. +It is a foreign key to the [`rasa_sender.id`](#rasa_sender) column. + +- Type: `varchar(36)` +- Example: `9e4ebded-f232-4cc5-af78-d98daa0c1a53` + +#### `session_id` session identifier + +The unique identifier of the session this marker is part of. +It is a foreign key to the [`rasa_session.id`](#rasa_session) column. + +- Type: `varchar(36)` +- Example: `63b150a6-21a3-4e6c-bb24-5ab6ddc30cf1` + +#### `event_id` event identifier + +The unique identifier of the event from event broker where this marker was applied. +Note that a marker can be applied across multiple events, this is the ID of the last event in the sequence. + +- Type: `varchar(36)` +- Example: `63b150a6-21a3-4e6c-bb24-5ab6ddc30cf1` + +#### `num_preceding_user_turns` Number of Proeeding User turns + +an integer indicating the number of user turns preceding the event at which the marker applied. + +- Type: `integer` +- Example: `4` + +#### `created_at` creation date time + +Time of creation of this marker. The timestamp is a UTC. + +- Type: `DateTime` +- Example: `2022-06-21 02:15:49.326936` + + ## Internal Tables Internal tables are used to store information about the assistant and diff --git a/docs/docs/monitoring/analytics/realtime-markers.mdx b/docs/docs/monitoring/analytics/realtime-markers.mdx new file mode 100644 index 000000000000..e32efee33cb0 --- /dev/null +++ b/docs/docs/monitoring/analytics/realtime-markers.mdx @@ -0,0 +1,95 @@ +--- +id: realtime-markers +sidebar_label: Real-Time Markers +title: Real-Time Analysis of Markers +description: +abstract: Process markers real-time from Rasa + Assistants to track metrics like solution and abandonment rates, + add metatags to conversations, + or filter conversations for Conversation Driven Development +--- + +import useBaseUrl from "@docusaurus/useBaseUrl"; +import RasaProLabel from "@theme/RasaProLabel"; +import RasaProBanner from "@theme/RasaProBanner"; +import { Grid } from "@theme/Grid"; + + + +:::info Available in Rasa Pro from 3.6.0 +::: + + + +In Rasa Pro, [Markers](../../markers.mdx) are a powerful feature that allows +you to track and extract custom conversational events. With the Analytics +Data Pipeline, you can now process these markers in real-time, enabling +you to gain valuable insights and enhance the performance of your Rasa +Assistant. In this guide, we'll explore how to leverage real-time +analysis of markers to track solution and abandonment rates in your conversations. + +![flow of information with realtime markers](/img/analytics/realtime-markers.png) + +## Defining Markers + +Please consult the [Markers](../../markers.mdx/#defining-markers) section of Rasa documentation +for details about defining markers. + +## Enable Real-time Processing + +To enable real-time analysis of markers in your Rasa project, you need to +upload the markers to Analytics Data Pipeline with the `rasa markers upload` +command. This command validates the marker YAML against the domain file to +make sure all the slots, intents and actions referred in the file also exist +in the domain and uploads the marker configuration YAML to the +Analytics Data Pipeline (where they're persisted in a database) and exits. +To get started, make sure you have the latest +version of Rasa installed. Then, open your command line +interface and navigate to your Rasa project directory. Run the following command: + +``` +rasa markers upload --config= -d= -rasa-pro-services-url= +``` + +By default, this command validates the marker configuration file against +your bot's domain.yml file. To specify a different domain file, use the +optional `-d` argument. + +This command should be run whenever there is a change in the marker +configuration file. The changes might include addition of new markers, +changing an existing marker or removing an existing marker. + +:::note + +This command **uploads** the marker configurations to the data pipeline. The pipeline assumes the configuration +file is the source of truth and only processes the markers defined in it. If you remove a marker from this file, +and run this command, then the processing for that marker is stopped. + +::: + +### Configuring the CLI command + +Visit our [CLI page](./command-line-interface.mdx#rasa-marker-upload) +for more information about the command line arguments available. + +## How are Markers processed? + +The markers YAML file describes the pattern of events for marker extraction. +Once the YAML files are uploaded, the patterns to be used for marker extraction are +stored in the `rasa_patterns` table. As the Kafka Consumer starts receiving events +from the Rasa Assistant, it starts analyzing them for markers. The Pipeline +processes all the events from the Kafka Event Broker and identifies points of +interest in the conversation that match the marker. The extracted markers are then stored +in the `rasa_marker` table. + +The evaluation of Markers in the Pipeline is similar to the `rasa evaluate markers` +command which can be used to process Markers from the conversations in Tracker Store. +Read more about it [here](../../markers.mdx/#extracting-markers) + +Extracted markers are added to the `rasa_markers` table in the database +immediately once they are processed. Each row in this table contains +the foreign key identifiers for the pattern, session, sender and +last event when marker was extracted along with `num_preceding_user_turns` +which tracks the number of turns preceding the event at which the marker applied. +Check out the [Data Structure Reference](data-structure-reference.mdx/#rasa-markers) page for the +database schema of relevant tables. diff --git a/docs/docs/rasa-pro.mdx b/docs/docs/rasa-pro.mdx index 8be08d9982c3..b1ab3353a92f 100644 --- a/docs/docs/rasa-pro.mdx +++ b/docs/docs/rasa-pro.mdx @@ -5,7 +5,7 @@ title: Rasa Pro hide_table_of_contents: true --- -Rasa Pro is the commercial, pro-code offering of Rasa that’s built to address enterprise needs +Rasa Pro is the commercial, pro-code offering of Rasa built to address enterprise needs around security, observability and scale. Rasa Pro integrates seamlessly with enterprise technology stacks and is the recommended platform @@ -44,6 +44,18 @@ Resolve performance issues faster and identify bottlenecks in message handling a [Read more here](./monitoring/tracing.mdx). +### PII Handling + +Anonymize PII (Personal Identifiable Information) in logs and events streamed via the Kafka event broker. + +[Read more here](./pii-management.mdx) + +### Real-Time Markers + +Mark points of interest in conversations to support the targeted analysis of user journeys real time. + +[Read more here](./monitoring/analytics/realtime-markers.mdx) + ### Secrets Management Enhance security with our seamless Vault integration, enabling dynamic credential rotation for Rasa databases without system disruptions. @@ -53,3 +65,9 @@ Enhance security with our seamless Vault integration, enabling dynamic credentia ### Security Scanning for Vulnerability Protection Nightly and proactive security patches on your docker image to make sure dependencies are always up to date. + +### Spaces (Alpha Release) + +Modularize your assistant for better scaling and team collaboration. + +[Read more here](./spaces.mdx) diff --git a/docs/sidebars.js b/docs/sidebars.js index 566abae8caa7..0678d4d9594d 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -87,6 +87,7 @@ module.exports = { collapsed: true, items: [ 'monitoring/analytics/getting-started-with-analytics', + 'monitoring/analytics/realtime-markers', 'monitoring/analytics/example-queries', 'monitoring/analytics/data-structure-reference', ], diff --git a/docs/static/img/analytics/realtime-markers.png b/docs/static/img/analytics/realtime-markers.png new file mode 100644 index 000000000000..267a114ff8e2 Binary files /dev/null and b/docs/static/img/analytics/realtime-markers.png differ