Dream emotion dist #568

asleepann · 2023-09-12T17:45:08Z

Made dream_emotion dist with the following annotators: bot-emotion-classifier (determines emotion and mood of the bot) and emotional-bot-response (change hypotheses text according to emotion and mood of the bot). Response selector in dream_emotion selects emotional text of the hypothesis.
(old PR: #563)
DO NOT DELETE THIS BRANCH

annotators/bot_emotion_classifier/service_configs/bot-emotion-classifier/service.yml

dilyararimovna · 2023-09-18T07:22:03Z

annotators/emotional_bot_response/server.py

+            del api_conf[key]
+            break
+
+logger.info(f"Available APIs: {', '.join([api['display_name'] for api in api_conf.values()])}")


you do not use API_CONFIGS and api_conf, so I suggest to remove them (and from yml files too)

dilyararimovna · 2023-09-18T07:26:09Z

components/lsaHJAS68dskjHfslkdj2.yml

@@ -0,0 +1,25 @@
+name: emotion-response_selector


i suggest to follow unofficial rules and name components with "_" not "-".
With "-" we name containers.
(it was important rule when we were using Kubernetes, so we just follow it for now to be able to use kuber in future without breaking changes)

fixed
also, thank you for explanations :)

dilyararimovna · 2023-09-18T07:26:19Z

components/uKasd89kldsIK0Pd.yml

@@ -0,0 +1,23 @@
+name: bot-emotion-classifier


btw why bot emotion cls is annotator not bot uttr annotator?

name fixed
don't know why I didn't fixed type of the annotator before)
now should be correct: changed state manager method and formatters

...ed_response_selector/service_configs/emotion-ranking-based-response-selector/environment.yml

annotators/bot_emotion_classifier/service_configs/bot-emotion-classifier/service.yml

dilyararimovna · 2023-09-28T17:40:06Z

annotators/emotional_bot_response/service_configs/emotional-bot-response/service.yml

+      SERVICE_PORT: 8050
+      SERVICE_NAME: emotional-bot-response
+      WORK_DIR: annotators/emotional_bot_response
+      FLASK_APP: server


no need to write it here

dilyararimovna · 2023-09-28T17:40:14Z

annotators/emotional_bot_response/service_configs/emotional-bot-response/service.yml

+  build:
+    args:
+      SERVICE_PORT: 8050
+      SERVICE_NAME: emotional-bot-response


dilyararimovna · 2023-09-28T17:45:13Z

..._based_response_selector/service_configs/emotion-ranking-based-response-selector/service.yml

+      N_UTTERANCES_CONTEXT: 5
+      FILTER_TOXIC_OR_BADLISTED: 1
+      EMOTIONAL_RESPONSES: 1
+      FLASK_APP: server


you may not put it here- it's below

state_formatters/dp_formatters.py

dilyararimovna · 2023-09-28T17:46:43Z

state_formatters/dp_formatters.py

+    hypots = [h["text"] for h in hypotheses]
+
+    bot_mood_label = (
+        dialog["bot_utterances"][-1]["annotations"].get("bot_emotion_classifier", {}).get("bot_mood_label", "happy")


what if this fucntion will be triggered on the 1st step of the dialog? When no bot uttrs

added handling of this situation

dilyararimovna · 2023-10-11T12:55:29Z

annotators/emotional_bot_response/server.py

+if GENERATIVE_SERVICE_CONFIG:
+    with open(f"common/generative_configs/{GENERATIVE_SERVICE_CONFIG}", "r") as f:
+        GENERATIVE_SERVICE_CONFIG = json.load(f)
+GENERATIVE_TIMEOUT = int(getenv("GENERATIVE_TIMEOUT", 30))


dilyararimovna · 2023-10-11T13:01:58Z

annotators/emotional_bot_response/server.py

+def make_prompt(sentence, emotion="neutral", mood="happy", intensity=7):
+    prompt = f"""You are a writer who needs to rewrite a sentence to express
+    specific emotions, moods, and intensity levels. Your task is to generate
+    a new sentence that conveys the desired emotions, moods, and emotion intensity


put the prompt into a file. you may replace in the following view. In the prompt put:

- Sentence: {SENTENCE}

and then do: prompt.replace("{SENTENCE}", sentence)

dilyararimovna · 2023-10-11T13:39:43Z

annotators/emotional_bot_response/server.py

+    except Exception as e:
+        sentry_sdk.capture_exception(e)
+        logger.exception(e)
+        response = None


None is a bad choice. If response is str, then set default value to ""

dilyararimovna · 2023-10-11T13:39:59Z

annotators/emotional_bot_response/server.py

+    except Exception as exc:
+        logger.exception(exc)
+        sentry_sdk.capture_exception(exc)
+        result = {"hypotheses": "fallback standard response"}


"" empty fallback

dilyararimovna · 2023-10-11T13:41:14Z

annotators/emotional_bot_response/server.py

+    bot_mood_labels = ["angry"]
+    bot_emotions = ["anger"]
+    responses = rewrite_sentences(sentences, bot_mood_labels, bot_emotions)
+    logger.info("TEST. Sentences: {}".format(sentences))


more common to use f-string for now which is:

f"TEST. Sentences: {sentences}"

dilyararimovna · 2023-10-11T13:42:39Z

annotators/emotional_bot_response/server.py

+    bot_emotion = request.json.get("bot_emotion", [])
+
+    results = []
+    for i in range(len(sentences)):


just FYI
you can do:

for sent, mood_lab, emot in zip(sentences, bot_mood_labels, bot_emotions):

thanks! done

dilyararimovna · 2023-10-11T13:42:59Z

annotators/emotional_bot_response/server.py

+
+    sentences = request.json.get("sentences", [])
+    bot_mood_label = request.json.get("bot_mood_label", [])
+    bot_emotion = request.json.get("bot_emotion", [])


multiple bot_emotionS (and mood labelS)

dilyararimovna · 2023-10-11T13:45:08Z

assistant_dists/dream_emotion/docker-compose.override.yml

+    build:
+      args:
+        SERVICE_PORT: 8048
+        SERVICE_NAME: emotion-response_selector


name of component with only _
name of container with only -

dilyararimovna · 2023-10-11T13:48:19Z

assistant_dists/dream_emotion/pipeline_conf.json

+    "metadata": {
+        "display_name": "Dream",
+        "author": "DeepPavlov",
+        "description": "Main version of DeepPavlov Dream Socialbot",


dilyararimovna · 2023-10-11T13:51:11Z

state_formatters/dp_formatters.py

+        {
+            "sentences": hypots,
+            "bot_mood_label": bot_mood_labels,
+            "bot_emotion": bot_emotions,


bot_emotions
bot_mood_labels

dilyararimovna · 2023-10-24T11:15:08Z

annotators/emotional_bot_response/server.py

+if GENERATIVE_SERVICE_CONFIG:
+    with open(f"common/generative_configs/{GENERATIVE_SERVICE_CONFIG}", "r") as f:
+        GENERATIVE_SERVICE_CONFIG = json.load(f)
+GENERATIVE_TIMEOUT = int(getenv("GENERATIVE_TIMEOUT", 30.0))


Suggested change

GENERATIVE_TIMEOUT = int(getenv("GENERATIVE_TIMEOUT", 30.0))

GENERATIVE_TIMEOUT = float(getenv("GENERATIVE_TIMEOUT", 30.0))

dilyararimovna

Good Job! thank you!

dilyararimovna · 2023-10-24T11:16:28Z

annotators/bot_emotion_classifier/server.py

+        print("SENTENCE:", sentence)
+        print("USER EMOTION: ", user_emotion)
+        print("SENTIMENT: ", sentiment)
+        print("OLD BOT MOOD: ", bot_mood)


all prints should be deleted (or changed to logger) -- they will not be shown because of logger

deleted them (I already had the same info in logger)

dilyararimovna · 2023-10-24T11:17:54Z

annotators/bot_emotion_classifier/server.py

+    annotated_utterances = request.json.get("annotated_utterances", [])
+    bot_moods = request.json.get("bot_mood", [])
+
+    for i in range(len(sentences)):


just writing to show you how it is more beautiful to write these 4 lines of code in one:

for sentence, annotated_utterance, bot_mood in zip(sentences, annotated_utterances, bot_moods):

I don't know why but sometimes zip confuses me in code of other people, and I find for i easier to understand, but I will try to use zip instead because it's truly more beautiful :)

dilyararimovna · 2023-10-24T11:19:52Z

annotators/emotional_bot_response/server.py

+    )
+    try:
+        hypotheses = send_request_to_prompted_generative_service(
+            "",


dialog context is a list of utterance not str

dilyararimovna · 2023-10-24T11:22:56Z

assistant_dists/dream_emotion/docker-compose.override.yml

+          sentence-ranker:8128, property-extraction:8136, prompt-selector:8135, openai-api-chatgpt:8145,
+          dff-dream-persona-chatgpt-prompted-skill:8137, dff-dream-faq-prompted-skill:8170,
+          openai-api-chatgpt-16k:8167, bot-emotion-classifier:8051, emotional-bot-response:8050,
+          emotion-ranking-based-response-selector:8048"


i suggest to get rid of extra components in this distribution. For example, factoid compoentns, etc.
So, you me leave components from "in general" prompted distribution (aka https://github.com/deeppavlov/dream/tree/main/assistant_dists/dream_persona_openai_prompted):

sentseg:8011, combined-classification:8087,
sentence-ranker:8128, prompt-selector:8135, openai-api-chatgpt:8145,
dff-dream-persona-chatgpt-prompted-skill:8137, dff-dream-faq-prompted-skill:8170,
openai-api-chatgpt-16k:8167, bot-emotion-classifier:8051, emotional-bot-response:8050,
emotion-ranking-based-response-selector:8048

done. Hope I removed unnecessary components from all config files...

components/fuipwahef749384h2e.yml

dilyararimovna · 2023-10-24T11:26:23Z

components/lsaHJAS68dskjHfslkdj2.yml

+- candidate_annotators
+required_previous_services: null
+state_manager_method: add_bot_utterance
+tags: null


selectors. And in pipeline conf please also fix it. As in skill selector (this is a bug in whole dream -- I am goijng to fix it in a separate pull request for all response selectors. I actually thought I did it previously, but seems like not yet).
https://github.com/deeppavlov/dream/blob/dev/assistant_dists/dream_persona_openai_prompted/pipeline_conf.json#L159-L161 (also find a card for this component and make analogy)

added tags to response selector in component card and in pipeline_conf

dilyararimovna · 2023-10-24T11:27:49Z

components/fuipwahef749384h2e.yml

+group: candidate_annotators
+connector:
+  protocol: http
+  timeout: 15.0


why this timeout is 15 seconds if your generative timeout is 120? fix it. Including in pipeline and othe config files. Maybe you can decrease the generative timeout (it won't take 120 secods because the prompt is yours hard-coded, so it would be a fast response by openai)

changed generative timeout to 30 sec everywhere

dilyararimovna · 2023-10-24T11:28:09Z

response_selectors/emotion_ranking_based_response_selector/server.py

+
+SENTENCE_RANKER_ANNOTATION_NAME = getenv("SENTENCE_RANKER_ANNOTATION_NAME")
+SENTENCE_RANKER_SERVICE_URL = getenv("SENTENCE_RANKER_SERVICE_URL")
+SENTENCE_RANKER_TIMEOUT = int(getenv("SENTENCE_RANKER_TIMEOUT"))


Suggested change

SENTENCE_RANKER_TIMEOUT = int(getenv("SENTENCE_RANKER_TIMEOUT"))

SENTENCE_RANKER_TIMEOUT = float(getenv("SENTENCE_RANKER_TIMEOUT"))

also compare file with the latest version of response selector original (which it is copied from) because I think there were some bug fixes, i want you to check whether you do not have same bugs here

fixed. Also, added new test files as in ranking_based_response_selector: test.py and conftest.py

dilyararimovna · 2023-11-07T09:25:07Z

response_selectors/emotion_ranking_based_response_selector/server.py

+
+            if EMOTIONAL_RESPONSES:
+                emotional_hypotheses = hypotheses[best_id].get("annotations").get("emotional_bot_response")
+                selected_responses.append(emotional_hypotheses.pop("hypotheses"))


из emotional_hypotheses не обязательно извлекать текст как pop.
а вот из hypotheses[best_id] надо выдернуть text в любом случае - чтобы убрать лишнее из дикта и оставить там только атрибуты, без ключа text.
плюс нужна проверка, что предсказание аннотатора, то есть этот эмо респонс, не пустой.

dilyararimovna · 2023-11-07T09:26:41Z

...ed_response_selector/service_configs/emotion-ranking-based-response-selector/environment.yml

@@ -0,0 +1,9 @@
+SERVICE_PORT: 8048
+SERVICE_NAME: emotion_response_selector


ты его в пайплайн конфе используешь как response_selector, а не emotion_response_selector. Так что поменяй тут название обратно (и в docker compose yml тоже)

dilyararimovna · 2023-11-07T09:26:46Z

..._based_response_selector/service_configs/emotion-ranking-based-response-selector/service.yml

+  build:
+    args:
+      SERVICE_PORT: 8048
+      SERVICE_NAME: emotion_response_selector


ты его в пайплайн конфе используешь как response_selector, а не emotion_response_selector. Так что поменяй тут название обратно (и в docker compose yml тоже)

dilyararimovna · 2023-11-07T09:28:13Z

components/fuipwahef749384h2e.yml

+dialog_formatter: state_formatters.dp_formatters:bot_mood_emotion_formatter
+response_formatter: state_formatters.dp_formatters:simple_formatter_service
+previous_services: skills
+required_previous_services: skills


не надо делать это required. пусть тут будет null

dilyararimovna · 2023-11-07T09:29:36Z

assistant_dists/dream_emotion/docker-compose.override.yml

+    build:
+      args:
+        SERVICE_PORT: 8048
+        SERVICE_NAME: emotion_response_selector


ты его в пайплайн конфе используешь как response_selector, а не emotion_response_selector. Так что поменяй тут название обратно (и в docker compose yml тоже)

dilyararimovna · 2023-11-14T08:39:42Z

annotators/bot_emotion_classifier/server.py

+            {
+                "bot_mood": new_bot_mood,
+                "bot_mood_label": new_bot_mood_label,
+                "bot_emotion": bot_emotion,


oops... I have noticed previously.
So, you need to create an empty list result, and append those dicts to it on every iteration of for cycle above. and then jsonify result

so, you input and output batches would be of the same length. because now they are input of, for example, 5 samples, and output of length 1 anyway

assistant_dists/dream_emotion/pipeline_conf.json

dilyararimovna · 2023-11-14T08:56:48Z

components/fuipwahef749384h2e.yml

+model_type: NN-based
+is_customizable: true
+author: publisher@deeppavlov.ai
+description: Propose alternative hypotheses with specified emotional color.


if it proposes a hypotheses, why it is a candidate annotator NOT a skill?
I suggest to make a few changes and move it to skills

This is not a skill, because it uses hypotheses and changes them according to specified emotions. Yes, it sounds confusing, we initially discussed this too, but we need to finish it for the AC report. I can change the description in the component

Changed description

so, it doesn't produce a new one hypotheses, but processes existing hypothesEs?

dilyararimovna · 2023-11-14T11:50:12Z

assistant_dists/dream_emotion/pipeline_conf.json

+            "is_enabled": true,
+            "source": {
+                "component": "components/rFC0YJOoDFvS.yml",
+                "service": "services/agent_services/service_configs/dream"


component above card should be created a new one (in the card in one place the title of the dist is used, so it is unique). And dream -> dream_emotion

dilyararimovna · 2023-11-14T11:50:19Z

assistant_dists/dream_emotion/pipeline_conf.json

+            "is_enabled": true,
+            "source": {
+                "component": "components/sbDcAqiNqxFz.yml",
+                "service": "services/agent_services/service_configs/dream"


component above card should be created a new one (in the card in one place the title of the dist is used, so it is unique). And dream -> dream_emotion

services/agent_services/service_configs/dream_emotion/environment.yml

services/agent_services/service_configs/dream_emotion/service.yml

dilyararimovna

* add dream_emotion dist with annotators * add component cards * fix codestyle flake8 * fix codestyle * another codestyle fix * PR fixes * fixed batches * move preprocessing to annotator * fix codestyle black * minor fixes * removed unnecessary api_configs * changed type of annotator to bot utts annotator * fixed component file * fixed service.yml files * fixed formatters * fixed emotional response server and dist files * moved prompt into separate file * fixed annotators * fixed annotators again * fixed emotion response selector * fixed emotion dist and components * fixed codestyle * fixed codestyle again * fixed codestyle black * fixed naming of emotion response selector * fixed emotional_bot_response component * fixed emotion response selector server * added prev_services in annotator in pipeline_conf * fixed result formatting of bot_emo_cls * fixed codestyle * fixed codestyle again * changed description of emo_bot_response * added necessary component cards * fix: timeout * fix: timeout --------- Co-authored-by: Dilyara Zharikova (Baymurzina) <dilyara.rimovna@gmail.com>

asleepann added 9 commits August 29, 2023 22:06

add dream_emotion dist with annotators

73aa87d

add component cards

9dfd0fc

fix codestyle flake8

e4ed51a

fix codestyle

f98bcb4

another codestyle fix

3d72d70

PR fixes

6fdbd3f

fixed batches

29181b0

move preprocessing to annotator

c62e0c3

fix codestyle black

2fda668

dilyararimovna self-requested a review September 18, 2023 07:06

dilyararimovna requested changes Sep 18, 2023

View reviewed changes

asleepann added 4 commits September 18, 2023 19:05

minor fixes

27a541a

removed unnecessary api_configs

597e363

changed type of annotator to bot utts annotator

c6a8c80

fixed component file

bd0f171

dilyararimovna requested changes Sep 28, 2023

View reviewed changes

asleepann added 2 commits September 29, 2023 17:14

fixed service.yml files

f6bd772

fixed formatters

9a1d533

dilyararimovna requested changes Oct 11, 2023

View reviewed changes

asleepann added 2 commits October 19, 2023 06:13

fixed emotional response server and dist files

aeb452d

moved prompt into separate file

0c007a5

dilyararimovna reviewed Oct 24, 2023

View reviewed changes

dilyararimovna requested changes Oct 24, 2023

View reviewed changes

asleepann added 7 commits November 3, 2023 02:15

fixed annotators

6398d01

fixed annotators again

24ad18a

fixed emotion response selector

67e07ec

fixed emotion dist and components

84f3392

fixed codestyle

32e2da9

fixed codestyle again

e476b67

fixed codestyle black

1377bcd

dilyararimovna requested changes Nov 7, 2023

View reviewed changes

asleepann added 3 commits November 12, 2023 00:47

fixed naming of emotion response selector

4355635

fixed emotional_bot_response component

3c230ed

fixed emotion response selector server

2c261fa

dilyararimovna requested changes Nov 14, 2023

View reviewed changes

asleepann added 5 commits November 14, 2023 12:26

added prev_services in annotator in pipeline_conf

94bdbbb

fixed result formatting of bot_emo_cls

bf945da

fixed codestyle

64c5078

fixed codestyle again

3995623

changed description of emo_bot_response

767a45f

dilyararimovna requested changes Nov 14, 2023

View reviewed changes

added necessary component cards

44dcd4f

dilyararimovna reviewed Nov 16, 2023

View reviewed changes

services/agent_services/service_configs/dream_emotion/environment.yml Outdated Show resolved Hide resolved

fix: timeout

05cf5ba

dilyararimovna reviewed Nov 16, 2023

View reviewed changes

services/agent_services/service_configs/dream_emotion/service.yml Outdated Show resolved Hide resolved

fix: timeout

094a837

dilyararimovna approved these changes Nov 16, 2023

View reviewed changes

dilyararimovna merged commit e12986e into dev Nov 16, 2023
2 checks passed

	GENERATIVE_TIMEOUT = int(getenv("GENERATIVE_TIMEOUT", 30.0))
	GENERATIVE_TIMEOUT = float(getenv("GENERATIVE_TIMEOUT", 30.0))

	SENTENCE_RANKER_TIMEOUT = int(getenv("SENTENCE_RANKER_TIMEOUT"))
	SENTENCE_RANKER_TIMEOUT = float(getenv("SENTENCE_RANKER_TIMEOUT"))

		@@ -0,0 +1,9 @@
		SERVICE_PORT: 8048
		SERVICE_NAME: emotion_response_selector

Dream emotion dist #568

Dream emotion dist #568

Conversation

asleepann commented Sep 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dilyararimovna left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asleepann commented Sep 12, 2023 •

edited

Loading