Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dialoGPT Persona #185

Merged
merged 67 commits into from
Oct 10, 2022
Merged

dialoGPT Persona #185

merged 67 commits into from
Oct 10, 2022

Conversation

dmitrymailk
Copy link
Collaborator

Добавил предтренированную модель dialoGPT на датасете PersonaChat и сервис для ранжирования предложений по косинусной близости.

AndriiHura and others added 27 commits January 24, 2022 11:22
README.md Outdated
#### Dream mini up

```
docker-compose -f docker-compose.yml -f assistant_dists/dream_mini/docker-compose.override.yml -f assistant_dists/dream_mini/proxy.yml up --build
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev yml


Args:
persona_sentences (List[str]): список предложений составляющие полную персону. Defaults to None.
sentence_model (SentenceTransformer): модель для перевода предложения в вектор. Defaults to None.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

англ

dialogpt-persona:
environment:
DEVICE: cpu
CUDA_VISIBLE_DEVICES: ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sentence ranker тоже

WAIT_HOSTS: "convers-evaluator-annotator:8004, dff-program-y-skill:8008, sentseg:8011, convers-evaluation-selector:8009,
dff-intent-responder-skill:8012, intent-catcher:8014, badlisted-words:8018,
spelling-preprocessing:8074, dialogpt:8125"
WAIT_HOSTS: "convers-evaluator-annotator:8004, dff-program-y-skill:8008, sentseg:8011, convers-evaluation-selector:8009, dff-intent-responder-skill:8012, intent-catcher:8014, badlisted-words:8018, spelling-preprocessing:8074, dialogpt:8125, dialogpt-persona:8131, sentence-ranker:8130"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

верни в нескольтко строк - там удобнее смотртеь, пожалуйста

IGNORE_DISLIKED_SKILLS: 0
GREETING_FIRST: 1
GREETING_FIRST: 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

вернуть параметры на старые

@@ -341,12 +341,14 @@ def last_utt_and_history_dialog(dialog: Dict) -> List:
dialog = utils.get_last_n_turns(dialog)
dialog = utils.remove_clarification_turns_from_dialog(dialog)
dialog = utils.replace_with_annotated_utterances(dialog, mode="punct_sent")
logger.info(f"FORMATTER {dialog}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

убрать

state_formatters/dp_formatters.py Outdated Show resolved Hide resolved
services/dialogpt_persona/server.py Outdated Show resolved Hide resolved
services/dialogpt_persona/server.py Outdated Show resolved Hide resolved
services/dialogpt_persona/Dockerfile Outdated Show resolved Hide resolved
annotators/sentence_ranker/README.md Outdated Show resolved Hide resolved
annotators/sentence_ranker/test.py Outdated Show resolved Hide resolved
assistant_dists/dream_mini/docker-compose.override.yml Outdated Show resolved Hide resolved
assistant_dists/dream_mini/docker-compose.override.yml Outdated Show resolved Hide resolved
services/dialogpt_persona/README.md Outdated Show resolved Hide resolved
services/dialogpt_persona/server.py Outdated Show resolved Hide resolved
@dilyararimovna dilyararimovna merged commit c3bb406 into deeppavlov:dev Oct 10, 2022
oserikov pushed a commit that referenced this pull request Nov 21, 2024
* Fix requirements.txt (#84)

* fix itsdangerous requirements

* pin itsdangerous requirements for all flask==1.1.1 servers

* increase timeout to 5s

* add logs

* increase to 100

* add first working version gpt_persona

* add sentence_ranking(not working)

* fix wrong endpoint

* create sentecnce ranker annotator

* rewrite text generation logic

* add comments to code

* fix gpt_persona fallback

* clean code, add train script, write tests

* add get_intents, remove dataset, add hyperparams

* add batch support

* fix: move files

* fix: merge

* fix: codestyle

* fix: remove sentence ranker

* feat: new distribution and rename skill

* feat: new annotator

* feat: relative persona extractor

* fix: codestyle

* fix: proxy

* fix: params

* fix: volumes

* fix: reqs

* fix: tests

* fix: tests relative sents extr

* fix: batching

* fix: codestyle

* fix: persona extractor tests

* fix: persona get

* fix: tests

* fix: imports

* fix: logs

* fix: docs

* fix: add midas

* fix: command

* feat: add to main dist and tests

* feat: remove infilling, add dialogpt persona based, docs

* fix: gpus

* fix: params

* fix: param

* fix: indent

* fix: remove infilling from tests

Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com>
Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com>
Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants