Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format tracker state for logging #6133

Closed
indam23 opened this issue Jul 2, 2020 · 9 comments
Closed

Format tracker state for logging #6133

indam23 opened this issue Jul 2, 2020 · 9 comments
Labels
area:rasa-oss/cli Issues focused on the rasa command-line-interface area:rasa-oss 🎡 Anything related to the open source Rasa framework difficulty:easy 🦋 type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@indam23
Copy link
Contributor

indam23 commented Jul 2, 2020

Description of Problem:
It would be useful if the tracker state was displayed in logs in a way that's easier to parse.
At the moment, it's displayed in a list of dictionaries which can be hard to read when debugging

[{'intent_thankyou': 1.0, 'prev_action_listen': 1.0}, {'prev_utter_noworries': 1.0, 'intent_thankyou': 1.0}, {'intent_goodbye': 1.0, 'prev_action_listen': 1.0}, {'intent_goodbye': 1.0, 'prev_utter_goodbye': 1.0}, {'intent_pay_cc': 1.0, 'prev_action_listen': 1.0}]

Overview of the Solution:
It would be easier for debugging if it was displayed similar to how slots are displayed i.e. in a human-readable list

Definition of Done:

@indam23 indam23 added type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR area:rasa-oss 🎡 Anything related to the open source Rasa framework labels Jul 2, 2020
@indam23 indam23 modified the milestone: 1.10 Rasa Open Source Jul 2, 2020
@Raghibshams456
Copy link

I want to solve this problem but I am not able to understand the problem properly..can you explain me please..

@rajsukanya
Copy link

I would like to work on this issue.

@indam23
Copy link
Contributor Author

indam23 commented Jul 17, 2020

@rajsukanya do you have any questions about it? Feel free to open a PR

@mardntas
Copy link
Contributor

Hi, me and @aleronupe are interesting in this issue and started to wok on it. @melindaloubser1 do you have any example of how you think the logging should look like?

@indam23
Copy link
Contributor Author

indam23 commented Sep 22, 2020

Not sold on a specific format, but it should represent a human-readable version of the state. It'll be important to keep in mind what the state really means: Each item in the list is the last action + last intent (which was not necessarily the previous step, there could be multiple actions). If the action is action_listen, then that intent was the previous step. The example shown in the issue description translates to this state:

- action: action_listen 1.0
- intent: thankyou 1.0
- action: utter_noworries 1.0
- action: action_listen 1.0
- intent: goodbye 1.0
- action: utter_goodbye 1.0
- action: action_listen 1.0
- intent: pay_cc 1.0

However, this was in Rasa 1.x. Since the release candidate for 2.x is currently out, new features/enhancements should target 2.x. The way state is stored and shown is different for 2.x, which means this issue is probably not worth addressing in 1.x.

@aleronupe
Copy link
Contributor

Thank you for your answer @melindaloubser1. We will proceed to work on this issue targeting the 2.x version currently on master.
We noticed that there are a few differences on tracker state between the two versions of rasa, hence the new format is as follows:

Current tracker state [{}, {'user': {'intent': 'greet'}, 'prev_action': {'action_name': 'action_listen'}}, {'user': {'intent': 'greet'}, 'prev_action': {'action_name': 'utter_greet'}}, {'user': {'intent': 'affirm'}, 'prev_action': {'action_name': 'action_listen'}}]

Is it okay if we follow the same approach of human-readable logs of your previous answer?

@indam23
Copy link
Contributor Author

indam23 commented Sep 24, 2020

Cool! As I mentioned, I'm really not particular about how it's done, as long as it makes sense from the human-readability standpoint. If you have questions about the implementation or the new format and what it means, feel free to tag me in your PR and either I or someone else will get back to you.

@mardntas
Copy link
Contributor

Our proposed solution can be found at #7026 pull request

@alwx alwx added the area:rasa-oss/cli Issues focused on the rasa command-line-interface label Jan 29, 2021
@wochinge
Copy link
Contributor

closed by #7026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss/cli Issues focused on the rasa command-line-interface area:rasa-oss 🎡 Anything related to the open source Rasa framework difficulty:easy 🦋 type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

No branches or pull requests

8 participants