RFC: Flow Action – Encoding rules-based conversations in action-server

[moaazsidat]

RFC: Flow Action – Encoding rules-based conversations in action-server

Problem

We (Dialogue) work with a large number of rules based conversations where we want certain blocks of the conversation to happen in a deterministic manner / same way 100% of the time. We also want to reuse these blocks across different Rasa stories.

We started out by leveraging what Rasa provides out of the box and ran into limitations with each:

1. Memoization Policy

This is meant for learning conversations that will happen the same way all the time – we use checkpoints in combination with actions that set featurized slots to achieve conditional branching to represent the rules in our rules based conversations.

Problems

We've reached the limits of how far we can go with this because of the sheer amount of training data, and subsequently, the time it takes to train, due to the high number of checkpoints and featurized slots.

This either means that our model fails to train given a fair amount of resources (Ubuntu, 16gb memory), and even if it does succeed, it takes over 15-20 minutes to train and build the app every time a change is made, making the feedback loops around testing conversational changes very slow, driving down engineering productivity.

There's probably a couple of specific problems to pull out from the above, we've created some issues around the problems we've run into thus far with what we know:

• Limitation on stories / checkpoints: Training a rather large model (250 StoryBlocks) with multiple checkpoints halts on Linux #5912
• Limitation on slots: Rasa training is very slow due to excessive copy of the domain, fails on machine with low memory. #6044

Beyond that, checkpoints are limited in their ability to compose rules based conversations, because two different Rasa stories cannot converge to a checkpoint and then each diverge to a different path from it – they both need to continue along the checkpoint.

2. Form Action

Next, we went down the route of expressing the rules based conversation as a single form – achieving any branching that we have to do by encapsulating that logic in the required_slots or the run method of the form that are called on every turn.

This approach led to the problems outlined below, but also made us realize that what we were trying to achieve is running a series of forms (or actions), each of which is triggered when a set of conditions were met.

Problems

1. encapsulates too much logic in one form, makes it hard to maintain
2. makes it difficult to achieve modularity offered by forms in the first place – hard to compose conversations that ask the same subset of questions.
3. to an extent lose the ability to express conversations in any other way than just python code – this means content creators for bots have to know and understand python / action-server

---

With those constraints in mind, we set out to build an abstraction we'd like to have the Rasa team's thoughts on.

Flow action

Flow Action is like a Form action. It gets instantiated within a story as a form action would. It also requires a Form policy to be in place.

This is an action that executes a series of actions (actions, forms, or utterances) in a deterministic way. Each action specified as part of a Flow action can have a condition defined based on slot values that determines whether or not to execute the action.

We also built a YAML composer for Flow action to allow ease of writing and modifying conversational flows.

Example

How a Rasa story looks like using a flow

## User only wants food
* trigger_start
  - utter_greeting
  - utter_ask_types_of_food_and_drinks
* wants_food
  - food_preference_flow
  - form{"name": "food_preference_flow"}
  - form{"name": null}

## User wants food and drinks
* trigger_start
  - utter_greeting
  - utter_ask_types_of_food_and_drinks
* wants_food+wants_beverages
  - food_preference_flow
  - form{"name": "food_preference_flow"}
  - form{"name": null}
  - beverage_preference_flow
  - form{"name": "beverage_preference_flow"}
  - form{"name": null}

food_preference_flow written as a FlowAction:

name: food_preference_flow
description: Dispatches the customer to the right restaurant based on preference
actions:

  - name: food_preference_assessment_flow

  - name: burgers_flow
     condition: food_preference_outcome is "burgers"

  - name: pizza_flow
     condition: food_preference_outcome is "pizza"

  - name: sushi_flow
     condition: food_preference_outcome is "sushi"

food_preference_assessment_flow written as a FlowAction:

name: food_preference_assessment_flow 
description: |
	Asks questions until a food preference outcome 
	has been determined (burgers, pizza, sushi)
# this flow action exits as soon as a food_preference_outcome is computed
condition: food_preference_outcome is not None
actions:
  - name: allergies_form

  - name: adult_beverages_form
    condition: not child

  - name: children_beverages_form
    condition: child

  - name: preferred_cuisines_form

  - name: cuisine_details_from
    condition: preferred_cuisines not in ["cusine_not_supported_A", "cusine_not_supported_B"]

  - name: utter_cuisine_not_supported_but_will_still_try_and_help
    condition: preferred_cuisines in ["cusine_not_supported_A", "cusine_not_supported_B"]

  - name: action_determine_food_preference_outcome
  - name: action_predict_restaurants

Illustration of alternative syntax for condition

name: food_preference_assessment_flow 
description: |
    Asks questions until a food preference outcome 
    has been determined (burgers, pizza, sushi)
# this flow action exits as soon as a food_preference_outcome is computed
if: food_preference_outcome is not None
actions:
    - allergies_form
    - if: not child
      - adult_beverages_form
    - if: child
      - parents_approval_form
      - children_beverages_form
    - preferred_cuisines_form
    - cuisine_details_from
    - if: preferred_cuisines in ["cusine_not_supported_A", "cusine_not_supported_B"]
      - action_determine_food_preference_outcome
      - action_predict_restaurants
    - if: preferred_cuisines not in ["cusine_not_supported_A", "cusine_not_supported_B"]
      - utter_cuisine_not_supported_but_will_still_try_and_help

Notes

• for condition: food_preference_outcome is "burgers",
  • food_preference_outcome is a slot
  • condition is a [pypred](https://github.com/armon/pypred) Predicate statement which is evaluted using slot values from the tracker

Results

We've seen some positive preliminary results with this approach

1. faster build times, no reliance on training, faster feedback loops
2. can test rules based conversations more extensively with unit tests
3. yaml format, along with ability to compose conversations with deterministic blocks, using slots to build sets of conditions
4. More expressibility for conditional branching than stories, primarily because condition statements allow for combinations of boolean expressions.
5. cleaner representation of the conversations to learn – looking back at our conversational data that we'd like to turn into training data, there are often a series of forms that are triggered, flow seems to encapsulate that better than just using Rasa stories + checkpoints and forms.

Questions for the Rasa team & community

1. Is this a good approach to solving this problem?
   As Rasa engineers, is this a reasonable extension of Rasa, what problems do you forsee?
2. Even if the approach is good, are we mis-using the Form Action and Form Policy? Is there a better way to introduce a new way of controlling the flow (eg. a new policy?).
3. Is it a good idea to decouple rules based conversations from rasa core and let action-server take on that responsibility?
4. How does this compare with other solutions you may have in the works (Rule Policy)?
5. How does this align with the Rasa roadmap for 2.0 and beyond?
6. Where is the best place this abstraction should live? rasa, rasa-sdk, separate lib, or someplace else?

Authored by Ed and myself

[sara-tagger]

Thanks for submitting this feature request 🚀 @tabergma will get back to you about it soon! ✨

[Ghostvv]

We're working on creating RulePolicy. With it, you can create separate rule snippets that will be executed always if condition in them met. We'll announce it soon

[Ghostvv]

One of the main problems that I see with your proposal is how to deal with user input that don't correspond to the flow

[Ghostvv]

Here is the PR for RulePolicy: #6088

It doesn't contain documentation yet, but we added rules example and formbot example is rewritten as new rules

[moaazsidat]

how to deal with user input that don't correspond to the flow

could you elaborate on what you mean by that? perhaps an example of where that fails? thanks!

[Ghostvv]

similar as forms. there could be an intent like some chitchat that is not handled by the deterministic flow

[moaazsidat]

that can be handled the same way it is handled within forms, no?

## chitchat
* trigger_start
    - food_preference_assessment_flow
    - form{"name": "food_preference_assessment_flow"}
* chitchat
    - utter_chitchat
    - food_preference_assessment_flow
    - form{"name": null}

[Ghostvv]

then I don't see the difference to a form, except that conditions of when to stop, and what to do are different. We added one more abstraction LoopAction: https://github.com/RasaHQ/rasa/blob/master/rasa/core/actions/loops.py. It is specifically designed with this customization in mind

[wochinge]

This can now. be achieved with the RulePolicy